diff --git a/docs/kernel-contract.md b/docs/kernel-contract.md index 7cfdc4e0ff..7c4f16ce72 100644 --- a/docs/kernel-contract.md +++ b/docs/kernel-contract.md @@ -111,7 +111,9 @@ A replacement kernel names the kernel it replaces and gives `replaced`, which ru A gate failure on a kernel already shipped is reported, not hidden. Until its fix lands, its spec can carry a `known` entry naming the gate and what fails. That gate must keep failing (the run prints `KNOWN`); once a fix makes it pass, the run fails until the entry is removed. Today: affine_mm's 4-bit form gives NaN for an infinite input where -MLX's formula keeps the infinity, because its codes enter the MMAs as q + 128 and its bias as b - 128 s. +MLX's formula keeps the infinity, because its codes enter the MMAs as q + 128 and its bias as b - 128 s. GLM-5.3-Flash's +`tf_glm_scores` scores a NaN dot as 0 where float64 keeps the NaN: its relu is `fmaxf(x, 0)`, as `tl.maximum` is in the +Python family's Triton glue it was ported from. HIP’s MLX stream tile keeps its finite dot2 sums unchanged. A non-finite result with an actual non-finite operand is recomputed by full quantization groups using the factored float64 formula. The plain tile is @@ -147,6 +149,10 @@ CUDA also checks `window_attention`'s rows at head dims 256 and 512 (`window_row prompt tiles with P as bf16 or in two halves, a row's last key, windows over a ring or none over a linear cache), each output within float64's range over the spread its fp32 steps and tensor-core sums allow (`core/attention_range.zig`; seven controls, each one planted fault). +CUDA also checks GLM-5.3-Flash's KV page kernels on a page table out of order: the latent write and the pooled keys +(`tf_glm_lat_write`, `tf_glm_pool_keys`), the index scores and the selected tokens' rows (`tf_glm_scores`, +`tf_glm_dsa_tokens`), and the MLA chunks through their merge (`tf_glm_chunks_16` and `tf_glm_chunks_32`, dense and +sparse). HIP: the RMS norm rows, a wave a row (fp32, fp16 and bf16 out) and a block a row (fp16 and bf16 out). ## CUDA records per nvcc release @@ -158,7 +164,8 @@ and its split-K form, prompt16's FP8 tile, qmmf's FP4 32-row cluster tile, five and FP32-output paths, FP8G's 32-row cluster tile with FP32 output, the common EXL3 prompt products, Super's decode-sized latent top-k and NVFP4 up experts, the Mamba scan and Super's NVFP4 prompt down kernels, three of GLM-5.3-Flash's packed matmuls and three of its DFlash2 launches, and the training launches it compiles differently, -and nvcc 13.3 keeps 13.2's and changes `tf_nemo_topk` on sm_120 and sm_121. `kernel-contract` and +and nvcc 13.3 keeps 13.2's and changes `tf_nemo_topk` on sm_120 and sm_121 and four of GLM-5.3-Flash's KV page +launches. `kernel-contract` and `kernel-contract-limits` read the release from each image's `.note.nv.tkinfo`, check the image against the newest set at or below that release (13.0's when an image names none), and print which set they used. A release with no set of its own is checked against the one below it, so its differences show as stale records: re-derive them with diff --git a/docs/receipts/glm-kv-pages-0b094b74/README.md b/docs/receipts/glm-kv-pages-0b094b74/README.md new file mode 100644 index 0000000000..0e0fc3ed04 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/README.md @@ -0,0 +1,101 @@ +# GLM-5.3-Flash on CUDA: KV pages from the core pool, receipt + +Tested implementation: the binaries built from `0b094b74…`, commit 11 of #1163 (commit.txt has the tree and the +commits): commits 1 to 7 are the change, 8 to 10 add the kernel contract's tests, gate 5's records and two host tests, +and 11 fixes a bug the first TP=3 run found (below). After it, `cbe29802` puts a comment on one line and this +receipt's commit adds this folder: neither changes engine or kernel source. The kernel contract ran on `3b41fd7c` +(commit 10); commit 11 changes no kernel. TP=3's ranks ran commit 11's engine source with two local commits for the +ring, which are not part of #1163 (`tp3/ring.patch`): rank 0's NCCL id handed to the other ranks in a file, so that a +ring of /32 links bootstraps (#1178 is the upstream way), and a heat ceiling (`TF_HEAT_CEILING`: a prompt chunk also +waits while its expected rise would pass the ceiling), which changes when chunks run, not what they compute. +Base: main `f8d254c5…` (after 1.1.2); main's binary for the replay and the speed A/B was built the same way +(`bin-f8d254c5`). +Test dates: 2026-10-22 and 10-23. Platform: MSI EdgeXpert (GB10, DGX Spark-compatible; sm_121, 128 GB shared +memory), two at TP=2 over a direct RoCE link, three at TP=3 over a ring of direct links; driver 580.178.04. Ranks in +Ubuntu 24.04 containers pinned to the efficiency cores (`--cpuset-cpus=0-4,10-14`), NCCL 2.30.7. +Toolchain: Zig 0.17.0, nvcc 13.3 (V13.3.73) for sm_121; gate 5's records also with nvcc 13.0 (V13.0.88), both for +sm_80, 86, 89, 120 and 121. binaries.txt has every binary's sha256. +Checkpoint: `nvidia/GLM-5.3-Flash-NVFP4` at `423acf37583782c51c142d145aef733d72943d93`. + +## Scope + +This qualifies GLM's streams on KV pages from the core pool at TP=2 and TP=3: the paged streams against their +whole-window twins byte for byte, the ranks' agreement on one pool, served exactness on pages (concurrent = alone, +drafted = `"draft": false`, resumed = fresh, images), the replies against main's, a long prompt beside short ones, +admission when the pool is short and when more requests arrive at once than there are paged sequences, speed against +main at TP=2, the kernel contract with gate 5's records, the no-GPU CI and the mutations. It does not qualify FP8 +caches on pages (refused by name), DFlash2 drafts (no weights here), Metal or MLX. + +## The bug commit 11 fixes + +The first TP=3 run, on `b7fecb3d` (`tp3/found-b7fecb3d/r6all/`), served `TF_KV_PAGES=all` at `--parallel 8`. With +`all` the engine's own sequence stays idle, so eight lanes have seven paged sequences. Eight requests that arrive +together are admitted before any of them opens (the pool's check then sees every paged sequence free), and the last to +open took slot 8: binding it made a whole-window sequence, 12.88 GiB at a 1M window, before the pages refused it with +`LaneMemory`. Rank 0's host fell from 11 GiB free to 4 GiB in two seconds, its memory guard stopped the server +(`hostwatch-rank0.log`, `KILLED`), the other ranks ended on the closed line, and 53 of the 60 alone comparisons and 40 +of the 72 `"draft": false` ones failed; none was unequal. TP=2's `all` (`--parallel 4`, a 128k window) took the same +path and made 1.77 GiB for slot 4, inside its reserve, so its replies were all equal. Commit 11 takes a stream's pages +before its sequence is bound: a slot past the paged sequences waits for one and nothing is made for it. Every served +run in this folder but that one ran commit 11. + +## Results + +| Check | Result | +| --- | --- | +| `glm-paged-check`, TP=2, first five layers (`tp2/check/`) | Rank 0: 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their whole-window twins' logits, tokens and drafts byte for byte, the pages taken out of order (12 jumps) and NaN before use, the pool idle after; rank 1 made rank 0's pool and mapped its pages through every step | +| `glm-paged-check`, one GPU (`single/glm-paged-check.log`) | The same 7 prompts and 91 rows, byte for byte | +| Synthetic checkpoint, `TF_GLM_LAYERS=2` (`single/reg/`) | `glm-ref`, `glm-synth`, `glm-lanes-cover`, `glm-lanes-concurrent`, `glm-shared-check` and `glm-paged-check` pass, 0 failed | +| TP=2 `all` (`tp2/serve/r2all`) | 3 streams on 504 pages of 512 rows (3.14 GiB), 180 MiB fixed each, the engine's own sequence idle; `bench_concurrent.py`: 12 cells, every reply equal to its alone run (28/28) and to `"draft": false` (30/30); the reuse record's 44 requests all cached as planned; every idle census `free 504 + store 0 == total 504` | +| TP=2 `on` (`tp2/serve/r2on`) | 509 pages (3.17 GiB) beside the engine's own sequence: 28/28 and 30/30, 44 requests as planned, idle `free 509 + store 0 == total 509` | +| The `on` record replayed on this head's windows and on main (`tp2/serve/r2auto`, `r2main`, `tp2/compare/`) | `auto` keeps windows at TP=2 ("4 streams at once, 1.77 GiB each at a 131072-token window"), as main; both replays against the `on` and the `all` records: 43/43 replies equal and as cached as planned in all four, drafted == plain 23/23, no resumed reply slower than its fresh one | +| Images on pages (`tp2/serve/r2vis`) | 305 pages; `media_http.py`: drafted, repeated, `"draft": false` and three at once give the same tokens (prompt 2,050); `media_resume_http.py`: resumed past the image (6,144 of 7,605 tokens cached) = cold, as are `"draft": false` and two at once; idle `free 305 + store 0 == total 305` | +| Speed at TP=2 against main (`tp2/speed/`) | Serves in turn: this head, main, this head, main, this head with `all`. Decode, the 32k chat and cold prompts of 1k to 64k: this head's faster serve over main's slower 0.976 to 1.001, `all` 0.977 to 1.005; at 16k and 32k main's own two serves differ by 1.8% and 1.1%. The same tokens in all five | +| `glm-paged-check`, TP=3, first five layers (`tp3/check/`) | Rank 0: the same 7 prompts and 91 window rows equal their twins byte for byte, the pages out of order (12 jumps) and NaN before use, the pool idle after; ranks 1 and 2 made rank 0's pool and mapped its pages through every step | +| TP=3, pages by default, a 1M window (`tp3/serve/r6auto`) | `TF_KV_PAGES` unset pages at three ranks: "8 streams at once, the engine's own at a 1048576-token window (12.88 GiB, in the model's), 7 more taking 1284 pages of 512 rows (7.99 GiB) as they need them, 131 MiB fixed each, of 24.9 GiB left after a 12.2 GiB reserve"; `bench_concurrent.py`: 16 cells, every reply equal to its alone run (60/60) and to `"draft": false` (72/72); the reuse record's 44 requests all cached as planned; cold prompts of 1k to 64k tokens 0.73 s to 43.3 s (`cold.json`); seven 2k prompts while a 64k conversation held its rows on pages (63,488 of its 65,536 tokens resumed from the cold run's kept state): all seven equal their alone runs and none waited for another request to end (`long-beside-short.json`); every idle census `free 1284 + store 0 == total 1284` | +| TP=3 `all` (`tp3/serve/r6all`) | 1,237 pages of 512 rows (7.70 GiB), 131 MiB fixed each, the engine's own sequence idle, so seven streams at once; `bench_concurrent.py`: 16 cells, every reply equal to its alone run (60/60) and to `"draft": false` (72/72); in the cells of eight the eighth request waited for a paged sequence (their longest first token 12.0 to 21.2 s, against 1.7 to 2.1 s with `auto`); every idle census `free 1237 + store 0 == total 1237` | +| TP=3 `off`, the `auto` record replayed (`tp3/serve/r6off`) | A window a stream with `--parallel auto`: "2 streams at once, 12.88 GiB each at a 1048576-token window, of 24.7 GiB left after a 12.2 GiB reserve" (a named `--parallel 8` is refused); against the `auto` record (`compare-auto.log`): 43/43 replies equal and as cached as planned, drafted == plain 23/23, no resumed reply slower than its fresh one | +| Admission on a short pool (`tp3/serve/r6admit`) | `TF_KV_PAGES=all` with a 22.8 GiB prompt cache: 206 pages (1.28 GiB); `kvp_admit.py --fit` took 64k prompts, 129 pages each, so the pool holds one and not two. Three alone (cold, each kept in the prompt cache), then together: 3/3 equal their alone runs, each resumed 63,488 of its 65,536 tokens, and the second and the third started only after the one before ended; idle `free 206 + store 0 == total 206` | +| The run that found commit 11's bug (`tp3/found-b7fecb3d/r6all/`) | `TF_KV_PAGES=all` at TP=3 on `b7fecb3d` (above): the 8-stream cell's requests ended incomplete when rank 0 was stopped, and every later one could not connect; 7 of 60 alone and 32 of 72 `"draft": false` comparisons equal, 0 unequal | +| Kernel contract (`single/kernel-contract.log`, on `3b41fd7c`) | `SUMMARY 148 kernels: 148 pass gates 1-4, 0 fail, 1 known findings; 32 controls: 32 red as planted, 0 missed; 1018 limit problems` (the known finding: `tf_glm_scores` scores a NaN dot 0); main's (`single-main/kernel-contract.log`): 140 kernels, 140 pass, 0 known, 32 of 32 controls red, 1018; every limit problem line is main's | +| Gate 5's records (`gate5/`) | Fatbins for sm_80, 86, 89, 120 and 121 from nvcc 13.0.88 and 13.3.73: the six launches fit their limits on all five; on this GPU each runs with its 13.3 record's registers and shared memory (`PASS limits live`) | +| No-GPU CI under WSL2 on commit 11 (`ci/`) | `zig build test` 3,463/3,479 (15 skipped; the failure is upstream's `cuda_cost_identity` mtime test, WSL2's coarse file times), `test-kernel-contract` 72/72, `test-golden`, the server and pull security tests 20/20, `fuzz-cortheon` 14/14, the served Flash Next unittest (8), the GLM snapshot, KV pool and CUDA host tests 71/72 (the same failure); lean 439: the one problem past main's 438 is commit 11's two-line comment, which commit 12 puts on one line | +| Mutations (`ci/mutations/`) | Fifteen guards, each broken alone on a copy of commit 11: every one fails a test (`head.log`: the unchanged copy fails only `cuda_cost_identity`); the fifteenth binds a stream's sequence before its pages, as before commit 11 | + +## Reproduction + +Every serve: `tensorfold-native serve CKPT --name glm-tf --host 127.0.0.1 --port 8097 --max-tokens 32768 +--reasoning-effort high --parallel P --context C`, drafts on, one rank a host (`TF_GLM_RANK`, `TF_GLM_WORLD`), +`TF_HEAT_HIGH=92 TF_HEAT_LOW=88 TF_HEAT_CEILING=93`. TP=2: `--context 131072 --parallel 4`; `r2all`, `r2on` and +`r2vis` add `--prompt-cache-gib 4` and `TF_KV_POISON=1` with `TF_KV_PAGES` `all` or `on`, `r2vis` `--vision`; +`r2auto` and `r2main` and every speed serve `--prompt-cache-gib 0`. TP=3: `--context 1048576 --parallel 8` and +`TF_KV_POISON=1`; `r6auto` with `TF_KV_PAGES` unset and the default prompt cache (16 GiB), `r6all` `TF_KV_PAGES=all`, +`r6admit` `TF_KV_PAGES=all` with a 22.8 GiB prompt cache (the `all` serve's room less its streams' fixed parts and 0.9 GiB), `r6off` +`TF_KV_PAGES=off --prompt-cache-gib 0 --parallel auto`; NCCL over the ring's direct links with `NCCL_DEBUG=INFO`, and +each TP=3 host's memory guard (v3's `hostwatch.sh`: the server stopped below 5 GiB free). The checks are the +repository's tools: `bench_concurrent.py --alone --serial --tokens 256 --reps 1`, `reuse_served.py` (`record --planned +--min-prompt 2048`, `replay`, `compare`, `cold`, `ids`), `bench_openai.py`, `bench_cache.py --require-cold`; here +`media_http.py`, `media_resume_http.py`, `kvp_admit.py` and `kvp_sum.py` (tools/; `kvp_sum.py conc` ends non-zero +when a request failed or a reply differed). `tf-cuda-test glm-paged-check CKPT RANK WORLD IDFILE` with +`TF_GLM_LAYERS=5`. runs/ has the chains' own output. + +## Remaining coverage limits + +- TP=3's ranks ran commit 11's engine source with `tp3/ring.patch` (above); TP=2's and one GPU's ran commit 11 alone. +- Ranks 1 and 2 of the TP=3 check print their PASS lines in the middle of NCCL's INFO lines (`NCCL_DEBUG=INFO`). +- Speed against main was measured at TP=2 only; TP=3's cold prompt times are in `tp3/serve/r6auto/cold.json`, with no + main serve beside them. +- `--kv-cache fp8` keeps windows: `auto` does not page and `on` and `all` are refused (`Fp8OnPages`). +- DFlash2 (`--drafter`): no weights here; the drafts on pages are the MTP head's. +- Metal and MLX were not run (no Mac). + +## Publication handling + +CRLF line ends made LF, trailing blanks dropped and an indent with a space before a tab made spaces (as `git diff +--check` asks; four lines of two mutations' stack traces); machine names, internal addresses, home and local paths, and the +work's session, plan, records, container and image names replaced with placeholders (`redaction-report.json` counts +them a file; host-a to host-c run TP=3's ranks 0 to 2, host-d and host-e TP=2's ranks 0 and 1); result values, token +ids and token hashes kept. The TP=3 server logs keep NCCL's INFO lines as they were written. The reuse records keep +their rows without the request bodies (3 MB a record): each row has its body's sha256 and size, and +`records-sha256.txt` the originals'. The speed runs' prompt files are not included: `reuse_served.py cold` and `ids` +make them again. SHA256SUMS pins every published file except itself. diff --git a/docs/receipts/glm-kv-pages-0b094b74/SHA256SUMS b/docs/receipts/glm-kv-pages-0b094b74/SHA256SUMS new file mode 100644 index 0000000000..ee648394e7 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/SHA256SUMS @@ -0,0 +1,157 @@ +53859628bce5dd9c0688b493eaf89493effd9c718793dd2f6b917067a43dd0f8 README.md +7ab4224bf14be7e05655c0db0e96396c36c6b252da311ee84efcbc1c1c81a012 binaries.txt +40ca486838d221c325895202ea121c5971fc2f6911a6dac3a4c81e3511c0952a ci/flashnext-served.log +55e63c9c3792aa30e0f5f27933a52d03775f8b905290b66bbe2e04323f38b919 ci/fuzz-cortheon.log +18244e1dbb0b1bd2f5b52ece812c4801e2078e815456965aff5707110e0178a1 ci/glm-host.log +c146caf8704ad11f494a92a742df4d98844bf1cab9d16da0cbc25bd011ea78a1 ci/lean.txt +c081029197b627e8ad3554dbde0f096e8a6f422204988543bbc93b657a3369a2 ci/mutations/auto-at-two.log +c57bacd22add8d5d95f7e9230e66e7cce0c9d1102cb1b0e0d35ccd2e23236978 ci/mutations/bind-before-pages.log +83dc6862e9b99a79f8bd17cf934846298886bd143ab929185dc3c2a4c28142ca ci/mutations/bytes-on-paged-seq.log +7cc7f2b31e1847b4a6911608be7da1a16411a1896a14f02d8537dcf5177df67e ci/mutations/close-keeps-pages.log +3028b51610a85fdac3f34add964c969035be87c10034988001b117b7c84ddf48 ci/mutations/fp8-kernel-table.log +d2f4c1eb959bdd2a5002e97f325f86eed2f608c7d6f4e3adb6188dfa0ca1adef ci/mutations/fp8-on-pages.log +a024e03eb53ad63ded8cfe925863258c812fc1275c4a40294da0ca28af181caa ci/mutations/head.log +3708b3be101c55fc48fcf222dd2c9f019d2a2fce457b6194299b5cfe89a8dadf ci/mutations/kept-window-cache.log +a20cbc7b7b7ec687c617a3ed8feadeec54e61d0afddaaef2563f63bc17c7018f ci/mutations/map-any-id.log +c358db18a0024499e2cc255d36a86796e81fc87006d14f91fd3b6c7a70b33a3e ci/mutations/no-poison.log +385320138cb53d0e5e8c38a2efef81605d8b21f3ac9e8d4510d973cf70b34cfe ci/mutations/page-bytes-unchecked.log +ea30fbd1b606bc9b79ef6105e3d6d4e258626e0b02ba881895f741c56afe50ab ci/mutations/paged-first-at-cache.log +2be74d70a0446d76e93185c193bbe2fa3181e011fbbfb6c3ce87036d0ad2db78 ci/mutations/paged-table-kept.log +647158f91404df435027c915c74fdd36a026cc8547862dc81879ae723f573564 ci/mutations/piece-first-page.log +9bfe50b8f05bdc7ff995778ba9eee2b43e2afc4eb731e8710d3c8c44e062e9aa ci/mutations/piece-no-offset.log +848045fde420eb60700938300ab4ec81383315e4d74d3013329791afe8d1e06b ci/mutations/split-no-slots.log +41a42a61e37f1027f95aff94de5b6cba17e80c69dd53a30af2bc28739713797f ci/security.log +153d2125cbba424b422ef2ac7a30361f60572a950e102a275e57bb17ae8cdeb6 ci/test-golden.log +1754a6fa9431b03d177ca7d02017c49086d7fbc6afa8c664a6dcb81587171401 ci/test-kernel-contract.log +6762245324d32a5e8e028c780420294ae9220d5b72cdb8546bb8d385cc5e16e1 ci/test.log +e9815b2c3b6c53377216da6b76c3f38a50b836697b7ddd0390cc276cc49bf276 commit.txt +4ef07ace8dcfa4ca3ec730c02d2ed041b0299213b81e1206d4326490e35beafa gate5/build-nvcc13.0.log +27ce6bf357fafe35af42e93c7856946af42a641cec2aed34f999b7f3f1f3ddcb gate5/build-nvcc13.3.log +0a073e1023980be4271a514ba9a15924c7713f3e09f85c2e00aac5d3a07b181b gate5/limits-nvcc13.0.log +244a7227078063a88f00e52b08c4568c42f1d3a94d5ddef16718307bae9111c6 gate5/limits-nvcc13.3.log +3dce0ff9ba767b082974d9d79c4c6e765d0b62e8b5f2e3e54badb8c0bcb90c6c records-sha256.txt +a7d822fbcf7074bf88d85d4f823fb3c3909c9e47f1acc078f6ae541ba941f0e4 redaction-report.json +0e18bbe9a557eaa7546e44ab4b340a1bcf39397ae1911b527ed85d356a295751 runs/tp2-chain.txt +5e30478b4d7fffd485909e54b91a1214b5e57eb3b076a2a937bfa8bf0c6bf80e single-main/kernel-contract.log +8fbee5ca6da4118a71d90c7e2d5955d1f90387677a88f7b7717fb20cba1008b6 single/glm-paged-check.log +d4de6bb4eea45a02d085e4c6f6a301021b32d613978ee08e34eba928f1b42ce4 single/kernel-contract.log +adbf33709a37fbc4324ff069209cfee21365875cdd1f406bbbcf5a6b40c43cb5 single/reg/glm-lanes-concurrent.log +dbd448fe0919788628aba3969e1604ec34be032e4e65e48b9ef8dd1dd6b4adeb single/reg/glm-lanes-cover.log +b21a34a1250dc015643d1f936fd2f8adbfd2c94638af9efbe967c144a263ea60 single/reg/glm-shared-check.log +f0e8e3fe03e4412e2bd6c493c00b4b5867adef9fb28f6defcb1f55e31cad2f45 single/reg/paged.log +cce62e664ca901f5bb9f7fd149c901be4dbcab00a16d55997d10bab4a456a73a single/reg/ref.log +a0ac45dd5c611f1db38e9503f976a3ba621b7af17203ee7259496e022b8b6aec single/reg/summary.log +7dd9ba53e1f98929e7e4ba6944c8aa13ccf1915703dc614c8899b12227a3bdbd single/reg/synth.log +2e47bb5e010aaeef606b482811b0c7e15bd0b288e2ba86dc6f2c93c03dccdcf6 tools/kvp_admit.py +de2fb053f9cb118ba8f4e828944231bec917a4bc4aa049db8c26364f17883bb2 tools/kvp_sum.py +45166e626e21f39a16b5d399476f0a402df19f7993b669fde4838a3542878761 tools/media_http.py +16b709bd8b79d660f03b62ef0f45a1438413cc8516158b7851e06837207dab0b tools/media_resume_http.py +35da8e9d09ac87b8fa55f84e890bf1ccf2c61f974d3603ba8bab1c091d6e76fd tp2/check/glm-paged-check-rank0.log +27a3c10399e022ee115c4c1157977803b7be564ab02b1dfa030a030a41fd5bd0 tp2/check/glm-paged-check-rank1.log +60b7b86e15894388c5d1c9a703c4f528aea4a12b77b2a8e76074e03c1260a431 tp2/compare/auto-against-all.log +eaf1130726c65675b17df1884f2de084ef36967ce60cb62aa72697ae50acaf69 tp2/compare/auto-against-on.log +c8e6dc65416c37751d375b71049ab04cbe013f5c08e89fe6aea0c81bb86909ec tp2/compare/main-against-all.log +0825cb474f3ec95cf8e189c4dc098347d78ba8163c4d0e1b64e6ffcad6d8b48f tp2/compare/main-against-on.log +0defbfeb92772ecff924c2b9e4266f4ced11c5759ab6a7a89ff95a8ea3fbd2c7 tp2/serve/r2all/client.log +a424fe606ad3dca1041ad07c75b293684130e00706bbb31159afbc10e563012d tp2/serve/r2all/conc.json +8c16599ae771d464ba4d4640b5835c54e96d0583c4acec06a281b88fc25d3402 tp2/serve/r2all/rows-without-bodies.jsonl +0509975ed8e174e279b1d35851cc3a077a225c19fafd5ddb0d9fcd961d4fc04f tp2/serve/r2all/server-rank0.log +d39b30cf5ed0431e34c1da9430c834b9ec5f881f135441039fc2bad06d61d9d7 tp2/serve/r2all/server-rank1.log +9ce652f949d06f336da4a49a7d33d71033f07809f043ab49849a8177d858eeea tp2/serve/r2auto/client.log +9189a671020a8b3bf15730647f92a1ec3dcb4908cd328c5b1d5db992c516ddd0 tp2/serve/r2auto/fresh.jsonl +6d1b702f4c09459b9ba7c09e077aed42eeee90ca48135f4dcdc759c28f8460ad tp2/serve/r2auto/server-rank0.log +cec198fd6273e5bb9cb32be7037c93645b20e5b1e24b657ae142ead26f42abd4 tp2/serve/r2auto/server-rank1.log +0d8784e701ad04692fbdafb13492c649d0ed6026254cc308c7463bf404848681 tp2/serve/r2main/client.log +0274a572f9068b55651bc007653f333e009129461c0166a2f7b28b221bc0afe9 tp2/serve/r2main/fresh.jsonl +6514e2db0c935e36ab00db76d46749b9b8658b903d3ec42098ec01244e0cc125 tp2/serve/r2main/server-rank0.log +435d242a8eb701e3785f068059eed994f6f17caaefa9f2aa63ca1dbfa82e51df tp2/serve/r2main/server-rank1.log +52a699af50bde7cc31f414c7b29be8aa90bb97ce5a4742b14d54f8fc1d979a8e tp2/serve/r2on/client.log +22280c789bfc89ba1751956ca2d1cadd6fcc94a9e9e63c983556199c8fd519c4 tp2/serve/r2on/conc.json +f4eabf25497614911f05d5fa6cb3155b27a19e0ca331aebe6faebf858ba25229 tp2/serve/r2on/rows-without-bodies.jsonl +57af5e639abcdbb41a1ec9b9720faeae216821b6b5c10cef1d15063e71b4916a tp2/serve/r2on/server-rank0.log +4cdb8e899193730750af406e518dd5885baea787e7279bfd5f20588c1deb6b7e tp2/serve/r2on/server-rank1.log +77d5e2abe8d4acdfaf0167781aec32666fa75de4f1ee10545acd6d6788879db3 tp2/serve/r2vis/client.log +e7ce04e9a9736b85c56e08176d5192853e60903ab69b4e9bdaaeb062a69a61e0 tp2/serve/r2vis/media-resume.json +c5ff34f4f3d2cd1425da1e8740f8acefccb922bd308e11efdb751f71a5ff0f55 tp2/serve/r2vis/media.json +4f7f810bba77c3fadc6b3ad0e529192aff9fa2d148cd9f4bf0cf375041002999 tp2/serve/r2vis/server-rank0.log +2607a6a2bf2d6ed3d187aa08a82cffeda8645534310fb8e150c4ee557537fea0 tp2/serve/r2vis/server-rank1.log +bfdfe2db6472189fd6b9b96f810839341c40895f4b78658c356fc690bd71db19 tp2/speed/r3k2a/cache.json +d7fd98370fbc1fd58a890f5253f2f37e56d9d236ab0dafb71654e2aa164b6899 tp2/speed/r3k2a/client.log +80ad15ea60f3cb484fed3ffdd3ca3e5f2ea5e98763920f8a93e9f75162e3fe6e tp2/speed/r3k2a/cold.json +f02f331dbb3cfbadfac1322525190282d5b694a164a6a40322f3ce380b71b3c2 tp2/speed/r3k2a/decode.json +fd0be3b6a0083b8c64c6b55608309581dc1e72be678e2dcd9ab4786c0def727a tp2/speed/r3k2a/server-rank0.log +ffae7b93c92a083509ef0338d6763de555aa9ac884f6237b238946d66324e543 tp2/speed/r3k2a/server-rank1.log +3cda854bf8b91cce6d4a0892394d724585664d0682b2a8a61febd3dc7a120446 tp2/speed/r3k2all/cache.json +c8e2e802c0db73e4de25b1862060bee98a54d6b05e04e580f5bddb6fe3d39276 tp2/speed/r3k2all/client.log +cfb77f0db01a9e16ab9348fcfef7fff864afae5dabb24bd473fa351f9dd37705 tp2/speed/r3k2all/cold.json +f651e89548f414b4991d6bc40dbb4010b82c06d4b85c97b0b629bc0ab27f49a0 tp2/speed/r3k2all/decode.json +982dbf0cd28fa48a2ce3118275f4afe5a53be33b95e0b9a7150a5eb854f6f31b tp2/speed/r3k2all/server-rank0.log +c4706b38bd2b2fbcd052836ff22e885550acb0027374c69eb142b77c208d740b tp2/speed/r3k2all/server-rank1.log +bf3b4c8fb8933d032ca8fee35eb8cf393b36bfc07a9b7c9c15b5ad9e8865c3a4 tp2/speed/r3k2b/cache.json +54cdc1d047b13b84a5e2870470d56d49837836d97fda0daf8ddba07e3acce372 tp2/speed/r3k2b/client.log +3561f39410fc332455310bdfff5f72aa5e9f3eaf9835faa7f168aaaee6c52e44 tp2/speed/r3k2b/cold.json +eb95111c01442ba587fed47ae73210fd988687136cfa0049b2f1d4aca8a0e131 tp2/speed/r3k2b/decode.json +c1de3c8b31e53f0badc69af8cebb9efcda2751c780b554cd52ff7cd03716c997 tp2/speed/r3k2b/server-rank0.log +b664491aeea183493d59e5bde6d557413450980ff1bc94480ec513b6335914b0 tp2/speed/r3k2b/server-rank1.log +13f6a23d10c994dc92b1913f023980c13d4ac5d9f7ad4b878265d0dd13b1557e tp2/speed/r3maink2/cache.json +13dd1a4dfbcfc4947cb51a551e1b8765bf431421bd2c9a874c8741fade784b53 tp2/speed/r3maink2/client.log +7f53c6d8353c45a38b2b66eaaab893d18ce2cfe6f02972e25d66d20b87bb8aef tp2/speed/r3maink2/cold.json +2bf7a929ee8429c838de4a304ef3bf3dadf2b57788635af666e52665b851c97b tp2/speed/r3maink2/decode.json +20c50435390edae4ff914f7451bfba842006adb0897791ed021c8916c2e78813 tp2/speed/r3maink2/server-rank0.log +7ddae4daf73b68d15b5a5f543450b5a8ac3ae9e64f94b4bbd078b2c8c44178ea tp2/speed/r3maink2/server-rank1.log +f82327bce9a30b1657bca156c6bcd4edf61a1749a9a9d104f9d2de46b840215d tp2/speed/r3maink22/cache.json +88de99e11e96cdd79a3582895a84afb4177d50129ffebcc24b79885d4e1305da tp2/speed/r3maink22/client.log +d0954e40c4656f82522ccef9fccf1d5247a1696ade5914e81f6db44f8399a0e8 tp2/speed/r3maink22/cold.json +37869c1ab2309e822306d30e2a45132a9cb8baa29efd13b1fb8bbba432dd097b tp2/speed/r3maink22/decode.json +4131c2c3825ed5b6cad14d42cedd8dacb49ecb87ff823ce745053ee5a33b3e29 tp2/speed/r3maink22/server-rank0.log +c4380281c12dc66ed5073ae457693b901d8378040204ea1a45a86a483b2639a5 tp2/speed/r3maink22/server-rank1.log +1575808b853be08a43ac77a0349d2e49ee12e119a2c8b773f53b091bb7cc4686 tp3/build.log +9f2bfeec0e698a07573da09b7ab7f8e7be2101530578b7dc72f83067d64f36ea tp3/chain.log +c20971339ccabe3710149d8447d75546bab8183f67c2c216d78928492e540557 tp3/check/glm-paged-check-rank0.log +e9d6b89fef057fbd692bcaaf2821054ca779abaff0d26ecd4251247ebab4031d tp3/check/glm-paged-check-rank1.log +5b7deadf14e81d879d97d8c3bad29dd4f2a34859e1c4f1fce5c24acf073d2d23 tp3/check/glm-paged-check-rank2.log +589e7d223f2a28d464484045060afb59a6afbfb43b5098c7a93739faa260648d tp3/found-b7fecb3d/r6all/client.log +312a4ace68ad19d851d6f484f686c7d2cfaed3f2cd4c9e1a3ebfc8663006f857 tp3/found-b7fecb3d/r6all/conc.json +53c4b776324155f3f397c444e1096c61b0bad377950e4e6eca0ac56595498c6f tp3/found-b7fecb3d/r6all/hostwatch-rank0.log +03574c5171cc76ddfcfacaaf6e9ac5648945e7891f5636ce1d208207f42cc672 tp3/found-b7fecb3d/r6all/hostwatch-rank1.log +40d36b2432357405cfc54f4a7fbe4eb10627442ce39e2473f4ead3585e3b70c6 tp3/found-b7fecb3d/r6all/hostwatch-rank2.log +6be03a17048304912ecc26db2e3d2de2124cd33c9bd87e913c7dbf2534c0cadc tp3/found-b7fecb3d/r6all/server-rank0.log +55591a4c11331d084fec2df3847a58f6b4e1a834a2278aacade178c8c303cb3f tp3/found-b7fecb3d/r6all/server-rank1.log +d67f75b5709554b393e0adc6450c8aaec829ccbed26fc7abd5d9350e1ba6832b tp3/found-b7fecb3d/r6all/server-rank2.log +956a3b8d4cb6f60cbc600f29d212028cf2d2261b0a9b400ca171a32671c11dac tp3/ring.patch +32b96257ea2e39303305086438c05ae76453bc0db74bfc3e352b9b785378ffdb tp3/serve/r6admit/admit.json +f0b5a01a43c3eadd76bbc632626e7ec72e2c06dd53472025641320c05e1d7ae2 tp3/serve/r6admit/client.log +1b8c0e90ee5af9158c51adcc1332912b318be99dc28d1dc06a8612cc97c5b9c0 tp3/serve/r6admit/hostwatch-rank0.log +06712bb28ce129c3fbfd22ed7d2550bda1cc69e5c883f4e1a6ff608589a3760f tp3/serve/r6admit/hostwatch-rank1.log +a40fbf9825efc5a894b27e58098bb8adc268ee76664bd7b0982158e1b0efe07b tp3/serve/r6admit/hostwatch-rank2.log +cc013b320fe999e201c7e7b79c35477b64c70a55286643b0579d5f4166003375 tp3/serve/r6admit/server-rank0.log +b97bc2b3924f89a9682d6345e2f2109cf341d9a0f8701260c955b385e49ae81c tp3/serve/r6admit/server-rank1.log +9619b1fadbaf65011b8dd50fb712274bba307b820eee608fdf960233244a2b76 tp3/serve/r6admit/server-rank2.log +cab5017427bd763d1c58444bf85ff48e66d192f046311772b3290abe1f777b1f tp3/serve/r6all/client.log +de71fab2048e7efb605d5e53eb0202fcce33532dce7aef7b8ab3b59f8ac85163 tp3/serve/r6all/conc.json +699e9526930edebd1eedd68770264413cadb0eede20b5062fe30ec64de754a59 tp3/serve/r6all/hostwatch-rank0.log +0c720385b33da2bfd2193fa1e05b0bf07e1e8358d4ff239596aab3e4d8c1a834 tp3/serve/r6all/hostwatch-rank1.log +3941a616ff751bd77508ae53b03263b1a34b2cc430c20b3600b79a7bb099501d tp3/serve/r6all/hostwatch-rank2.log +2bb3499499c7fc240cc21133be485b8e238a15a4ec573bf2a7047187ef799986 tp3/serve/r6all/server-rank0.log +cfd6ec04408cb9d744db5597b7e6af49b8b56bad4bf09e4268431d53a72f0e17 tp3/serve/r6all/server-rank1.log +351782990c98d2ef2ecc820265008ad4c05cc4104b04b50cea0125fe4af21731 tp3/serve/r6all/server-rank2.log +9379076eec76877073637b7c50a334a095c78c61dd868b964ef099aebefec3ad tp3/serve/r6auto/client.log +5d84bfe56db2637d6cb7d50171b307c6bab96e847bca00254981e6d4eca63ec6 tp3/serve/r6auto/cold.json +767bacec8831826efe68820a567c62be678f4188d18389614e0d7a42e5c8e397 tp3/serve/r6auto/conc.json +c8b4d7b1826e93dda54fc0ce4a920017c0e1d2e5eab2eb0afc6464ca0eaf5dc6 tp3/serve/r6auto/hostwatch-rank0.log +5d1e6911fa3021b3a63e23653707c59c67867b834286f72201ea318b7da5ef48 tp3/serve/r6auto/hostwatch-rank1.log +107757d291b4ded976a39532a20e0e8f04f3f6170e965c932dbc806c85c951ae tp3/serve/r6auto/hostwatch-rank2.log +ba59818452234a122770d3b99673758ea11d8d98fdcea3405bf7d80bd5c2bdef tp3/serve/r6auto/long-beside-short.json +b8ee21d311250711c90d36bf3b56aeb077059d7961c3b5b4d4bf731a22d944dd tp3/serve/r6auto/rows-without-bodies.jsonl +99c678b8657d2a871f72fceb1ae3db21847e9ef22e3fad382300a0e67ce6b7cf tp3/serve/r6auto/server-rank0.log +aa74474e99cc50ec0a9744c10c6626cf1cdf009e638c55e66a0b59ffc58187de tp3/serve/r6auto/server-rank1.log +db7dc4485771ec3cf40440e8288a42fbb9d579e4685aa30ddede27a1872c1f89 tp3/serve/r6auto/server-rank2.log +9ecfcde3612aa5f834b226caae9afd1f56f28c6f23d36da762dc8d47398bf2a4 tp3/serve/r6off/client.log +178d7c6df17c821b16cda0a8be565750b6171d03d889a736a731314f1c07f4e7 tp3/serve/r6off/compare-auto.log +f1849eaf1efcb80e3dc7e992e90b849fbf8a0c61d8f8545c6bbad107c568bed4 tp3/serve/r6off/fresh.jsonl +3060cf50b6d98807ff11a06d0601721379235ec9b3f649d4b602f266897e7f83 tp3/serve/r6off/hostwatch-rank0.log +7377a0acf997321dda09923fa80c6526a6611221403efbacf1b4319c74a6bec8 tp3/serve/r6off/hostwatch-rank1.log +a03438df0f9464cb09fbc4a90ede443a2c669f139183a549cbe0b00a7d88befe tp3/serve/r6off/hostwatch-rank2.log +07c2743ce9ecfe926a9200aa4c03269a2a73de8341558bff70246cacac408d70 tp3/serve/r6off/server-rank0.log +350b3297d71a74d9afe5999c2640784c651ddd9cd663b4fce3b4804eb266e39b tp3/serve/r6off/server-rank1.log +1df4f720a53b6c7a8d287974f264473f1bb7076e5ee22d76d19e0b56dd44ecb5 tp3/serve/r6off/server-rank2.log diff --git a/docs/receipts/glm-kv-pages-0b094b74/binaries.txt b/docs/receipts/glm-kv-pages-0b094b74/binaries.txt new file mode 100644 index 0000000000..8310b5c0bf --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/binaries.txt @@ -0,0 +1,14 @@ +bin-0b094b74: 0b094b74 built with Zig 0.17.0 and nvcc 13.3 (V13.3.73) for sm_121; TP=2's ranks and the checks +d82043de5d8b1f04c36dec65e0cb9ec7af9c64ef9c4c5cc7b796f0087f1d9b1a tensorfold-native +d82043de5d8b1f04c36dec65e0cb9ec7af9c64ef9c4c5cc7b796f0087f1d9b1a tensorfold-native-k2 +d3042db03485ede9be61e848483674051167fba98f368470f5622cdc6b2cab26 tf-cuda-test + +bin-f8d254c5: main f8d254c5 built the same way; the replay on main and the A/B's main serves +d9181ac075f36ee5c118ef0b38513f6ebf61ab9fba312dbe592f2c434bd02a39 tensorfold-native + +bin-abc83c3d: b7fecb3d with tp3/ring.patch and 0b094b74's change above b7fecb3d, built the same way; TP=3's ranks and check +62649bbd8c2ce0d47e83e54f61dcb9e4dd3efce0f4d1ad4784bf4ad7f0482e14 tensorfold-native +2707a1813f751c12506c224e0f7b25220150c5bb27f00c212f8c664caeb58500 tf-cuda-test + +bin-4c8cd52b: b7fecb3d with tp3/ring.patch, built the same way; the TP=3 serve that found commit 11's bug (tp3/found-b7fecb3d/) +00552dccd11bea1331b44081ba4dc78c745c8e22ac5ef56af8285a3747a8bd67 tensorfold-native diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/flashnext-served.log b/docs/receipts/glm-kv-pages-0b094b74/ci/flashnext-served.log new file mode 100644 index 0000000000..87422c88ad --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/flashnext-served.log @@ -0,0 +1,5 @@ +........ +---------------------------------------------------------------------- +Ran 8 tests in 0.044s + +OK diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/fuzz-cortheon.log b/docs/receipts/glm-kv-pages-0b094b74/ci/fuzz-cortheon.log new file mode 100644 index 0000000000..eb4a1944d2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/fuzz-cortheon.log @@ -0,0 +1,4 @@ +Build Summary: 3/3 steps succeeded; 14/14 tests passed +fuzz-cortheon success ++- run test 14 pass (14 total) 16s MaxRSS:7M + +- compile test safe native success 16s MaxRSS:478M diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/glm-host.log b/docs/receipts/glm-kv-pages-0b094b74/ci/glm-host.log new file mode 100644 index 0000000000..0c2293eafe --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/glm-host.log @@ -0,0 +1,418 @@ +test-cuda-host ++- run test 0 pass, 1 fail (1 total) +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-fx2/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql(u8, &first, &changed)); + ^ +failed command: ./.zig-cache/o/5e598aff1d4f9edbdfba42f5b9bf1824/test --cache-dir=./.zig-cache --seed=0x6117f493 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/bc954b5be8f14a9483a426aa8b36cce6/test --cache-dir=./.zig-cache --seed=0x6117f493 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/9b8430079bb9d01dcc1cf517c720044b/test --cache-dir=./.zig-cache --seed=0x6117f493 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/fae06b03062dcdb2281ded9c24f0a44f/test --cache-dir=./.zig-cache --seed=0x6117f493 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 71/72 tests passed (1 failed) +test-cuda-host transitive failure ++- run test cached +| +- compile test debug native cached 54ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 69ms MaxRSS:38M ++- run test cached +| +- compile test debug native cached 51ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 72ms MaxRSS:39M ++- run exe runtime-discovery-exit success 9ms +| +- compile exe runtime-discovery-exit debug native cached 74ms MaxRSS:39M +| +- options cached +| +- compile lib runtime-discovery-mock debug native cached 71ms MaxRSS:39M ++- run exe rpc-warning-check success 1ms +| +- compile exe rpc-warning-check debug native cached 55ms MaxRSS:39M +| +- options cached +| +- compile lib rpc-mock-complete debug native cached 66ms MaxRSS:39M +| +- compile lib rpc-mock-abi debug native cached 40ms MaxRSS:37M +| +- compile lib rpc-mock-wait debug native cached 41ms MaxRSS:37M +| +- compile lib rpc-mock-store debug native cached 63ms MaxRSS:39M +| +- compile lib rpc-mock-wrong debug native cached 57ms MaxRSS:38M ++- run test cached +| +- compile test debug native cached 93ms MaxRSS:39M +| +- options (+5 more reused dependencies) ++- run test cached +| +- compile test debug native cached 67ms MaxRSS:39M ++- run test 30 pass (30 total) 41ms MaxRSS:10M +| +- compile test debug native cached 73ms MaxRSS:39M +| +- WriteFile h3-compile-only.fatbin cached +| +- options cached ++- run test cached +| +- compile test debug native cached 70ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 73ms MaxRSS:40M +| +- WriteFile attn_train.fatbin cached +| +- WriteFile dsv41_exl3_experts.fatbin cached +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- options cached +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- options cached +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) ++- run test cached +| +- compile test debug native cached 68ms MaxRSS:40M +| +- options cached +| | +- compile exe qwen-round-timing-probe debug native cached 92ms MaxRSS:39M +| | +- WriteFile draft_ids.txt cached +| | +- options (reused) +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 81ms MaxRSS:39M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 50ms MaxRSS:39M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run test cached +| +- compile test debug native cached 65ms MaxRSS:39M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run test cached +| +- compile test debug native cached 91ms MaxRSS:39M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 60ms MaxRSS:38M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- compile obj exl3_host_compile debug native cached 88ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 126ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 66ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 41ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 90ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 56ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 49ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 44ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 55ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 63ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 48ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 56ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test fast native cached 61ms MaxRSS:40M +| +- options (reused) ++- run test cached +| +- compile test fast native cached 62ms MaxRSS:40M +| +- options (reused) ++- compile obj bolt_host_compile debug x86_64-linux-gnu cached 5ms MaxRSS:39M +| +- options cached +| +- WriteFile draft_ids.txt (reused) ++- run test 26 pass (26 total) 2s MaxRSS:8M +| +- compile test debug native cached 84ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 60ms MaxRSS:38M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 68ms MaxRSS:40M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run exe qwen-cost-key-process success 50s +| +- compile exe qwen-cost-key-process debug native cached 64ms MaxRSS:40M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- WriteFile target/config.json cached ++- run test 0 pass, 1 fail (1 total) +| +- compile test debug native cached 78ms MaxRSS:39M ++- compile obj drama_cuda_native_host debug native cached 49ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 47ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 54ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 109ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 114ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 57ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 64ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 89ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 51ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 68ms MaxRSS:40M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 47ms MaxRSS:39M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 54ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 91ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 45ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 73ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 100ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 48ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 50ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 104ms MaxRSS:40M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run test cached +| +- compile test debug native cached 54ms MaxRSS:38M ++- run test cached +| +- compile test debug native cached 109ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 57ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 60ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 59ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 83ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 101ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 116ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 84ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 43ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 62ms MaxRSS:39M ++- run test cached + +- compile test debug native cached 63ms MaxRSS:40M + +- options cached + +- options (reused) + +- WriteFile draft_ids.txt (reused) + +- WriteFile exlr8_alias.zig cached +test-kv-pool success ++- run test cached +| +- compile test debug native cached 69ms MaxRSS:39M ++- run test cached + +- compile test debug native cached 98ms MaxRSS:39M +test-glm-snapshots-host success ++- run test 15 pass (15 total) 4s MaxRSS:65M + +- compile test debug native success 923ms MaxRSS:184M + +- options (reused) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/lean.txt b/docs/receipts/glm-kv-pages-0b094b74/ci/lean.txt new file mode 100644 index 0000000000..36783be6e2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/lean.txt @@ -0,0 +1,440 @@ +zig/build/hip.zig:1: comment past 120 columns +zig/kernels/cuda/experts.cu:1: comment past 120 columns +zig/kernels/cuda/experts_pack.cu:1: comment past 120 columns +zig/kernels/cuda/experts_prefill.cu:1: comment past 120 columns +zig/kernels/cuda/gdn.cu:1: comment past 120 columns +zig/kernels/cuda/qmm_frag.cuh:1: comment past 120 columns +zig/kernels/cuda/qmm_group.cu:1: comment past 120 columns +zig/kernels/cuda/qmm_prefill.cu:1: comment past 120 columns +zig/kernels/hip/affine.hip:1: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:1: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:368: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:473: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:503: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:521: block comment +zig/kernels/metal/core/hc_boundary.metal:5: comment past 120 columns +zig/kernels/metal/core/hc_boundary.metal:13: comment past 120 columns +zig/kernels/metal/core/hc_boundary.metal:82: comment past 120 columns +zig/kernels/metal/core/hc_boundary.metal:109: comment past 120 columns +zig/kernels/metal/core/moe_route.metal:1: comment past 120 columns +zig/kernels/metal/core/moe_route.metal:11: comment past 120 columns +zig/kernels/metal/core/moe_route.metal:68: comment past 120 columns +zig/kernels/metal/core/row_projection.metal:8: comment past 120 columns +zig/kernels/metal/core/row_projection.metal:72: comment past 120 columns +zig/kernels/metal/core/row_projection.metal:161: comment past 120 columns +zig/kernels/metal/decode/fn_gdn.metal:1: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:2: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:3: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:4: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:5: multi-line comment +zig/kernels/metal/glm_absorb_nax.metal:1: comment past 120 columns +zig/kernels/metal/glm_absorb_nax.metal:13: comment past 120 columns +zig/kernels/metal/glm_attn.metal:1: comment past 120 columns +zig/kernels/metal/glm_attn.metal:12: comment past 120 columns +zig/kernels/metal/glm_attn.metal:54: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:1: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:27: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:107: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:116: comment past 120 columns +zig/kernels/metal/glm_kda_prompt.metal:2: comment past 120 columns +zig/kernels/metal/glm_kda_prompt.metal:4: comment past 120 columns +zig/kernels/metal/glm_kda_prompt.metal:65: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:1: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:41: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:170: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:201: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:235: comment past 120 columns +zig/kernels/metal/kimi/dense.metal:10: comment past 120 columns +zig/kernels/metal/kimi/experts.metal:86: comment past 120 columns +zig/kernels/metal/nemotron_experts.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_experts.metal:97: comment past 120 columns +zig/kernels/metal/nemotron_glue.metal:172: comment past 120 columns +zig/kernels/metal/nemotron_head.metal:20: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:7: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:189: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:298: comment past 120 columns +zig/kernels/metal/nemotron_sample.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_sample.metal:145: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:8: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:134: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:149: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:313: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:1: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:28: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:64: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:198: comment past 120 columns +zig/kernels/metal/prefill/fn_prompt.metal:1: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:2: comment past 120 columns +zig/kernels/metal/prefill/fn_prompt.metal:128: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:137: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:200: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:201: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:253: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:254: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:255: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:318: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:319: multi-line comment +zig/kernels/metal/prefill/fn_select.metal:1: comment past 120 columns +zig/kernels/metal/prefill/fn_select.metal:14: comment past 120 columns +zig/kernels/metal/prefill/fn_select.metal:39: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:1: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:44: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:117: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:159: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:1: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:47: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:111: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:271: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:311: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:334: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax_b.metal:102: comment past 120 columns +zig/kernels/metal/prefill/sort.metal:52: comment past 120 columns +zig/kernels/metal/sources.zig:57: multi-line comment +zig/src/cli/cuda_segments.zig:1: comment past 120 columns +zig/src/cli/cuda_segments.zig:22: comment past 120 columns +zig/src/cli/info.zig:13: multi-line comment +zig/src/cli/pull.zig:232: multi-line comment +zig/src/cluster/admission.zig:1: comment past 120 columns +zig/src/cluster/admission_test.zig:1: comment past 120 columns +zig/src/cluster/backend.zig:1: comment past 120 columns +zig/src/cluster/backend.zig:11: comment past 120 columns +zig/src/cluster/backend.zig:101: comment past 120 columns +zig/src/cluster/backend_test.zig:1: comment past 120 columns +zig/src/cluster/budget.zig:1: comment past 120 columns +zig/src/cluster/canon.zig:1: comment past 120 columns +zig/src/cluster/canon.zig:14: comment past 120 columns +zig/src/cluster/canon.zig:75: comment past 120 columns +zig/src/cluster/cli_model.zig:98: comment past 120 columns +zig/src/cluster/cluster.zig:1: comment past 120 columns +zig/src/cluster/config.zig:1: comment past 120 columns +zig/src/cluster/cost.zig:1: comment past 120 columns +zig/src/cluster/estimate.zig:1: comment past 120 columns +zig/src/cluster/exchange.zig:1: comment past 120 columns +zig/src/cluster/exchange.zig:179: comment past 120 columns +zig/src/cluster/fabric_link.zig:1: comment past 120 columns +zig/src/cluster/fabric_link_test.zig:1: comment past 120 columns +zig/src/cluster/launch.zig:1: comment past 120 columns +zig/src/cluster/loader.zig:1: comment past 120 columns +zig/src/cluster/loader.zig:55: comment past 120 columns +zig/src/cluster/loader_test.zig:1: comment past 120 columns +zig/src/cluster/membership.zig:1: comment past 120 columns +zig/src/cluster/membership_test.zig:1: comment past 120 columns +zig/src/cluster/node.zig:83: comment past 120 columns +zig/src/cluster/plan.zig:1: comment past 120 columns +zig/src/cluster/plan.zig:12: comment past 120 columns +zig/src/cluster/plan.zig:15: comment past 120 columns +zig/src/cluster/plan.zig:116: comment past 120 columns +zig/src/cluster/plan.zig:176: comment past 120 columns +zig/src/cluster/probe_fixtures.zig:1: comment past 120 columns +zig/src/cluster/pull.zig:1: comment past 120 columns +zig/src/cluster/pull_test.zig:1: comment past 120 columns +zig/src/cluster/round.zig:1: comment past 120 columns +zig/src/cluster/status.zig:1: comment past 120 columns +zig/src/cluster/toy.zig:1: comment past 120 columns +zig/src/cluster/toy.zig:93: comment past 120 columns +zig/src/cluster/toy.zig:193: comment past 120 columns +zig/src/cluster/traffic.zig:1: comment past 120 columns +zig/src/cluster/traffic.zig:18: comment past 120 columns +zig/src/cluster/traffic.zig:38: comment past 120 columns +zig/src/cluster/traffic.zig:49: comment past 120 columns +zig/src/cluster/wire.zig:1: comment past 120 columns +zig/src/core/affine_mm.zig:1: comment past 120 columns +zig/src/core/affine_mm.zig:25: comment past 120 columns +zig/src/core/affine_mm.zig:33: comment past 120 columns +zig/src/core/affine_mm.zig:54: comment past 120 columns +zig/src/core/affine_mm.zig:60: comment past 120 columns +zig/src/core/affine_mm.zig:75: comment past 120 columns +zig/src/core/affine_mm.zig:99: comment past 120 columns +zig/src/core/checkpoint.zig:1: comment past 120 columns +zig/src/core/checkpoint.zig:11: comment past 120 columns +zig/src/core/checkpoint_metal.zig:65: comment past 120 columns +zig/src/core/checkpoint_metal.zig:178: comment past 120 columns +zig/src/core/copy_index.zig:1: comment past 120 columns +zig/src/core/hc.zig:1: comment past 120 columns +zig/src/core/hc.zig:30: comment past 120 columns +zig/src/core/lanes/fill.zig:26: comment past 120 columns +zig/src/core/lanes/fill.zig:54: comment past 120 columns +zig/src/core/lanes/gpu_full.zig:40: comment past 120 columns +zig/src/core/lanes/gpu_full.zig:114: comment past 120 columns +zig/src/core/lanes/gpu_full_test.zig:1: comment past 120 columns +zig/src/core/moe_route.zig:1: comment past 120 columns +zig/src/core/moe_route.zig:33: comment past 120 columns +zig/src/core/paced_read.zig:5: comment past 120 columns +zig/src/core/prompt_cache.zig:26: comment past 120 columns +zig/src/core/prompt_cache.zig:176: comment past 120 columns +zig/src/core/prompt_cache.zig:300: comment past 120 columns +zig/src/core/prompt_cache.zig:349: comment past 120 columns +zig/src/core/prompt_cache_test.zig:1: comment past 120 columns +zig/src/core/row_projection.zig:21: comment past 120 columns +zig/src/core/row_projection.zig:24: comment past 120 columns +zig/src/core/row_projection.zig:96: comment past 120 columns +zig/src/core/safetensors.zig:64: comment past 120 columns +zig/src/core/segments.zig:1: comment past 120 columns +zig/src/core/segments.zig:26: comment past 120 columns +zig/src/core/segments.zig:61: comment past 120 columns +zig/src/core/stagger.zig:7: comment past 120 columns +zig/src/core/stagger.zig:22: comment past 120 columns +zig/src/core/stagger.zig:144: comment past 120 columns +zig/src/core/template/template.zig:103: comment past 120 columns +zig/src/core/tokenizer/regex.zig:1: comment past 120 columns +zig/src/core/tokenizer/regex.zig:272: comment past 120 columns +zig/src/core/tokenizer/tokenizer.zig:1: comment past 120 columns +zig/src/core/tokenizer/unicode.zig:224: comment past 120 columns +zig/src/cuda/cublaslt.zig:1: comment past 120 columns +zig/src/cuda/segments.zig:142: comment past 120 columns +zig/src/cuda/triton.zig:72: comment past 120 columns +zig/src/fabric/collective.zig:1: comment past 120 columns +zig/src/fabric/collective.zig:130: comment past 120 columns +zig/src/fabric/collective.zig:183: comment past 120 columns +zig/src/fabric/cost.zig:34: comment past 120 columns +zig/src/fabric/cost.zig:43: comment past 120 columns +zig/src/fabric/link_fake.zig:8: comment past 120 columns +zig/src/fabric/mailbox.zig:59: comment past 120 columns +zig/src/fabric/manifest.zig:1: comment past 120 columns +zig/src/fabric/mcdma.zig:160: comment past 120 columns +zig/src/fabric/packet.zig:1: comment past 120 columns +zig/src/fabric/prefill_source.zig:18: comment past 120 columns +zig/src/fabric/verbs_abi.zig:1: comment past 120 columns +zig/src/fabric/verbs_link.zig:1: comment past 120 columns +zig/src/fabric/words.zig:142: comment past 120 columns +zig/src/families/flashnext/call_log.zig:1: comment past 120 columns +zig/src/families/flashnext/follow.zig:51: multi-line comment +zig/src/families/flashnext/follow.zig:51: comment past 120 columns +zig/src/families/flashnext/follow.zig:136: comment past 120 columns +zig/src/families/flashnext/follow.zig:147: comment past 120 columns +zig/src/families/flashnext/follow.zig:150: comment past 120 columns +zig/src/families/flashnext/gdn.zig:1: multi-line comment +zig/src/families/flashnext/gdn.zig:13: multi-line comment +zig/src/families/flashnext/gdn.zig:32: multi-line comment +zig/src/families/flashnext/marks.zig:1: comment past 120 columns +zig/src/families/flashnext/marks.zig:13: comment past 120 columns +zig/src/families/flashnext/marks.zig:40: comment past 120 columns +zig/src/families/flashnext/marks.zig:62: comment past 120 columns +zig/src/families/flashnext/marks.zig:72: comment past 120 columns +zig/src/families/flashnext/marks.zig:103: comment past 120 columns +zig/src/families/flashnext/marks.zig:121: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:17: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:20: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:25: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:49: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:108: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:155: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:205: comment past 120 columns +zig/src/families/flashnext/split.zig:1: comment past 120 columns +zig/src/families/flashnext/split.zig:8: comment past 120 columns +zig/src/families/flashnext/split.zig:62: comment past 120 columns +zig/src/families/flashnext/tp.zig:19: comment past 120 columns +zig/src/families/flashnext/tp.zig:216: multi-line comment +zig/src/families/flashnext/tp.zig:216: comment past 120 columns +zig/src/families/flashnext/tp.zig:247: multi-line comment +zig/src/families/flashnext/tp.zig:263: multi-line comment +zig/src/families/flashnext/tp.zig:283: comment past 120 columns +zig/src/families/flashnext/tp.zig:319: multi-line comment +zig/src/families/flashnext/tp.zig:320: multi-line comment +zig/src/families/flashnext/tp.zig:341: comment past 120 columns +zig/src/families/flashnext/tp.zig:371: comment past 120 columns +zig/src/families/flashnext/tp.zig:392: comment past 120 columns +zig/src/families/flashnext/tp.zig:407: comment past 120 columns +zig/src/families/flashnext/tp.zig:449: multi-line comment +zig/src/families/flashnext/tp.zig:450: multi-line comment +zig/src/families/flashnext/tp_settings.zig:8: comment past 120 columns +zig/src/families/flashnext/tp_settings.zig:39: comment past 120 columns +zig/src/families/glm/attn_check.zig:1: comment past 120 columns +zig/src/families/glm/attn_check.zig:127: comment past 120 columns +zig/src/families/glm/backend.zig:178: comment past 120 columns +zig/src/families/glm/backend.zig:326: comment past 120 columns +zig/src/families/glm/checks.zig:37: comment past 120 columns +zig/src/families/glm/checks.zig:72: comment past 120 columns +zig/src/families/glm/checks.zig:105: comment past 120 columns +zig/src/families/glm/checks.zig:153: comment past 120 columns +zig/src/families/glm/checks.zig:190: comment past 120 columns +zig/src/families/glm/config.zig:222: comment past 120 columns +zig/src/families/glm/cuda_lanes.zig:260: multi-line comment +zig/src/families/glm/engine.zig:1: comment past 120 columns +zig/src/families/glm/engine.zig:42: comment past 120 columns +zig/src/families/glm/engine.zig:105: comment past 120 columns +zig/src/families/glm/engine.zig:238: comment past 120 columns +zig/src/families/glm/engine.zig:455: comment past 120 columns +zig/src/families/glm/ep.zig:1: comment past 120 columns +zig/src/families/glm/ep.zig:25: comment past 120 columns +zig/src/families/glm/ep.zig:50: comment past 120 columns +zig/src/families/glm/ep.zig:204: comment past 120 columns +zig/src/families/glm/ep.zig:218: comment past 120 columns +zig/src/families/glm/ep.zig:309: comment past 120 columns +zig/src/families/glm/ep_control.zig:1: comment past 120 columns +zig/src/families/glm/ep_control.zig:120: comment past 120 columns +zig/src/families/glm/forward.zig:1: comment past 120 columns +zig/src/families/glm/forward.zig:197: comment past 120 columns +zig/src/families/glm/forward.zig:218: comment past 120 columns +zig/src/families/glm/forward.zig:286: comment past 120 columns +zig/src/families/glm/forward.zig:330: comment past 120 columns +zig/src/families/glm/forward.zig:393: comment past 120 columns +zig/src/families/glm/forward.zig:406: comment past 120 columns +zig/src/families/glm/forward.zig:536: comment past 120 columns +zig/src/families/glm/kept.zig:29: comment past 120 columns +zig/src/families/glm/kept.zig:81: comment past 120 columns +zig/src/families/glm/kept.zig:105: comment past 120 columns +zig/src/families/glm/kept.zig:218: comment past 120 columns +zig/src/families/glm/kernels.zig:1: comment past 120 columns +zig/src/families/glm/load_plan.zig:1: comment past 120 columns +zig/src/families/glm/load_plan.zig:99: comment past 120 columns +zig/src/families/glm/moe.zig:18: comment past 120 columns +zig/src/families/glm/moe.zig:150: comment past 120 columns +zig/src/families/glm/mtp.zig:1: comment past 120 columns +zig/src/families/glm/mtp.zig:17: comment past 120 columns +zig/src/families/glm/plain_rounds.zig:14: comment past 120 columns +zig/src/families/glm/plain_rounds.zig:53: comment past 120 columns +zig/src/families/glm/prompt.zig:1: comment past 120 columns +zig/src/families/glm/prompt.zig:21: comment past 120 columns +zig/src/families/glm/prompt.zig:134: comment past 120 columns +zig/src/families/glm/prompt.zig:156: comment past 120 columns +zig/src/families/glm/prompt.zig:218: comment past 120 columns +zig/src/families/glm/prompt.zig:246: comment past 120 columns +zig/src/families/glm/prompt.zig:342: comment past 120 columns +zig/src/families/glm/prompt.zig:393: comment past 120 columns +zig/src/families/glm/prompt.zig:417: comment past 120 columns +zig/src/families/glm/prompt.zig:432: comment past 120 columns +zig/src/families/glm/prompt.zig:553: comment past 120 columns +zig/src/families/glm/snapshot.zig:1: comment past 120 columns +zig/src/families/glm/snapshot.zig:59: comment past 120 columns +zig/src/families/glm/snapshot.zig:91: comment past 120 columns +zig/src/families/glm/weights.zig:1: comment past 120 columns +zig/src/families/kimi_k3/kernels.zig:283: comment past 120 columns +zig/src/families/kimi_k3/prepare.zig:1: comment past 120 columns +zig/src/families/nemotron/backend.zig:301: comment past 120 columns +zig/src/families/nemotron/backend.zig:421: comment past 120 columns +zig/src/families/nemotron/copy_lanes.zig:1: comment past 120 columns +zig/src/families/nemotron/copy_lanes.zig:66: comment past 120 columns +zig/src/families/nemotron/cuda_mtp.zig:372: comment past 120 columns +zig/src/families/nemotron/cuda_segments.zig:1: comment past 120 columns +zig/src/families/nemotron/cuda_segments.zig:64: comment past 120 columns +zig/src/families/nemotron/cuda_triton.zig:109: comment past 120 columns +zig/src/families/nemotron/cuda_triton.zig:116: comment past 120 columns +zig/src/families/nemotron/forward.zig:35: comment past 120 columns +zig/src/families/nemotron/head_block.zig:21: comment past 120 columns +zig/src/families/nemotron/layers.zig:61: comment past 120 columns +zig/src/families/nemotron/mtp.zig:193: comment past 120 columns +zig/src/families/nemotron/prefill_launch.zig:59: comment past 120 columns +zig/src/families/nemotron/prefill_launch.zig:137: comment past 120 columns +zig/src/families/nemotron/prefill_mixers.zig:1: comment past 120 columns +zig/src/families/nemotron/round_depth.zig:1: comment past 120 columns +zig/src/families/nemotron/simd_attention.zig:1: comment past 120 columns +zig/src/families/nemotron/simd_attention.zig:5: comment past 120 columns +zig/src/families/nemotron/simd_attention.zig:88: comment past 120 columns +zig/src/families/qwen3_5/config.zig:1: comment past 120 columns +zig/src/families/qwen3_5/qualify.zig:1: comment past 120 columns +zig/src/families/qwen3_5/weights.zig:1: comment past 120 columns +zig/src/hip/device_arch.c:1: block comment +zig/src/hip/driver.zig:1: comment past 120 columns +zig/src/hip/memory.zig:1: comment past 120 columns +zig/src/main.zig:226: comment past 120 columns +zig/src/native/flashnext_host.zig:445: comment past 120 columns +zig/src/native/metal.zig:318: comment past 120 columns +zig/src/server/chunk_plan.zig:1: comment past 120 columns +zig/src/server/hf_text.zig:23: comment past 120 columns +zig/src/server/log.zig:106: comment past 120 columns +zig/src/server/messages.zig:25: comment past 120 columns +zig/src/server/messages.zig:151: comment past 120 columns +zig/src/server/metrics.zig:325: multi-line comment +zig/src/server/model_text.zig:103: comment past 120 columns +zig/src/server/tool_params.zig:120: comment past 120 columns +zig/src/server/tool_parse.zig:1: comment past 120 columns +zig/src/server/tool_stream.zig:417: comment past 120 columns +zig/src/server/tool_streaming.zig:1: comment past 120 columns +zig/tests/cuda/bench.zig:17: comment past 120 columns +zig/tests/cuda/box/run.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/box/build.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/box/capture.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/box/zrun.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/capture.py:2: docstring past 120 columns +zig/tests/cuda/nemotron/test_aot_manifest.py:2: multi-line docstring +zig/tests/cuda/oracle/oracle.py:1: docstring past 120 columns +zig/tests/cuda/oracle/oracle.py:82: docstring past 120 columns +zig/tests/cuda/oracle/sass_compare.py:1: docstring past 120 columns +zig/tests/cuda/runtime_tests.zig:276: comment past 120 columns +zig/tests/dispatch_bench.zig:1: comment past 120 columns +zig/tests/fabric_handoff.zig:1: comment past 120 columns +zig/tests/fabric_library.zig:1: comment past 120 columns +zig/tests/fabric_vectors.zig:1: comment past 120 columns +zig/tests/flashnext_bench.zig:69: comment past 120 columns +zig/tests/flashnext_bench.zig:119: multi-line comment +zig/tests/flashnext_bench.zig:169: multi-line comment +zig/tests/flashnext_reuse.zig:93: comment past 120 columns +zig/tests/flashnext_reuse.zig:183: comment past 120 columns +zig/tests/flashnext_run.zig: 1690 lines (split past 600) +zig/tests/flashnext_run.zig:1: multi-line comment +zig/tests/flashnext_run.zig:2: multi-line comment +zig/tests/flashnext_run.zig:762: multi-line comment +zig/tests/flashnext_run.zig:828: multi-line comment +zig/tests/flashnext_run.zig:1308: multi-line comment +zig/tests/flashnext_run.zig:1393: multi-line comment +zig/tests/flashnext_run.zig:1441: multi-line comment +zig/tests/flashnext_run.zig:1442: multi-line comment +zig/tests/glm_run.zig:1: comment past 120 columns +zig/tests/grid_sync_bench.zig:1: comment past 120 columns +zig/tests/hip_mock.c:1: block comment +zig/tests/linktest_suite.zig:1: comment past 120 columns +zig/tests/mm_bench.zig:1: comment past 120 columns +zig/tests/moe_bench.zig:1: comment past 120 columns +zig/tests/nemotron_dense.zig:1: comment past 120 columns +zig/tests/nemotron_dense.zig:11: comment past 120 columns +zig/tests/nemotron_dense.zig:100: comment past 120 columns +zig/tests/nemotron_dense.zig:292: comment past 120 columns +zig/tests/nemotron_experts.zig:1: comment past 120 columns +zig/tests/nemotron_experts.zig:133: comment past 120 columns +zig/tests/nemotron_prefill_check.zig:1: comment past 120 columns +zig/tests/nemotron_sample_check.zig:1: comment past 120 columns +zig/tests/nemotron_wide.zig:1: comment past 120 columns +zig/tests/nemotron_wide.zig:244: comment past 120 columns +zig/tests/ordering.zig:1: comment past 120 columns +zig/tests/ordering.zig:29: comment past 120 columns +zig/tests/row_projection_check.zig:1: comment past 120 columns +zig/tests/row_projection_check.zig:19: comment past 120 columns +zig/tests/row_projection_check.zig:163: comment past 120 columns +zig/tests/server/build.sh:2: comment past 120 columns +zig/tests/server/freeze_cases.py:1: docstring past 120 columns +zig/tests/server/freeze_cases.py:150: comment past 120 columns +zig/tests/server/golden_check.zig:79: comment past 120 columns +zig/tests/server/golden_check.zig:173: comment past 120 columns +zig/tests/server/wire.py:52: docstring past 120 columns +zig/tests/server/zig_test.sh:2: comment past 120 columns +zig/tests/tp2_bench.zig:1: comment past 120 columns +zig/tests/wide_heads.zig:46: comment past 120 columns +zig/tests/wide_probe.zig:1: comment past 120 columns +zig/tests/wide_probe.zig:263: comment past 120 columns +zig/tests/wide_trees.zig:1: comment past 120 columns +zig/tests/wide_trees.zig:126: comment past 120 columns +zig/tests/wide_trees.zig:156: comment past 120 columns +tools/zig/check_prefill_layers.py:1: docstring past 120 columns +tools/zig/check_prefill_model.py:1: docstring past 120 columns +tools/zig/check_prefill_ops.py:1: docstring past 120 columns +tools/zig/flashnext_dump.py:1: multi-line docstring +tools/zig/flashnext_hc_coalesce.py:1: multi-line docstring +tools/zig/gen_glm_kernels.py:43: comment past 120 columns +tools/zig/glm_compare.py:1: docstring past 120 columns +tools/zig/glm_ref.py:1: docstring past 120 columns +tools/zig/glm_ref.py:96: docstring past 120 columns +tools/zig/glm_ref.py:122: docstring past 120 columns +tools/zig/oracle_nemotron_rows.py:38: docstring past 120 columns +tools/zig/prefill_dump.py:1: docstring past 120 columns +tools/zig/prefill_glue.py:1: docstring past 120 columns +tools/zig/prefill_kernels.py:34: docstring past 120 columns +tools/zig/prefill_launch.py:1: docstring past 120 columns +tools/zig/prefill_launch.py:173: docstring past 120 columns +tools/zig/prefill_swap.py:1: docstring past 120 columns +tools/zig/reuse_served.py:133: docstring past 120 columns +tools/zig/reuse_served.py:232: comment past 120 columns +tools/zig/reuse_served.py:254: docstring past 120 columns +tools/zig/reuse_served.py:290: docstring past 120 columns +tools/zig/reuse_served.py:375: multi-line docstring +tools/zig/reuse_served.py:401: docstring past 120 columns +tools/zig/reuse_served.py:423: docstring past 120 columns +tools/zig/reuse_served.py:440: docstring past 120 columns +tools/zig/test_fabric.sh:2: comment past 120 columns +tools/zig/test_lanes.sh:2: comment past 120 columns +tools/zig/tokenizer_parity.py:1: docstring past 120 columns +tools/zig/triton_aot_manifest.py:2: docstring past 120 columns +tools/zig/unicode_tables.py:1: docstring past 120 columns +3599 files, 439 problems diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/auto-at-two.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/auto-at-two.log new file mode 100644 index 0000000000..b9d4ad3ca1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/auto-at-two.log @@ -0,0 +1,110 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test 0 pass, 1 fail (1 total) +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql(u8, &first, &changed)); + ^ +failed command: ./.zig-cache/o/5e598aff1d4f9edbdfba42f5b9bf1824/test --cache-dir=./.zig-cache --seed=0x9f516834 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/9764124df67e11b91f84062f64593ae1/test --cache-dir=./.zig-cache --seed=0x9f516834 --listen=- + +test-cuda-host ++- run test 83 pass, 1 fail (84 total) +error: 'cuda_native.test.TF_KV_PAGES: auto pages from three ranks, off and 0 give windows, all pages every stream, other values page' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + expected .off, found .on + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1289c04 in expectEqualInner__func_1028 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x128f867 in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_native.zig:373:9: 0x128f8c0 in test.TF_KV_PAGES: auto pages from three ranks, off and 0 give windows, all pages every stream, other values page (cuda.zig) + try std.testing.expectEqual(Paging.off, try paging(null, w, false)); + ^ +failed command: ./.zig-cache/o/c6d111b79f49b2a17b6edb5ea777e7a3/test --cache-dir=./.zig-cache --seed=0x9f516834 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x9f516834 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/745541c8bb8aa10067244b3b31f01b3a/test --cache-dir=./.zig-cache --seed=0x9f516834 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x9f516834 --listen=- + +Build Summary: 160/163 steps succeeded (2 failed); 253/255 tests passed (2 failed) +test-cuda-host transitive failure ++- run test 0 pass, 1 fail (1 total) ++- run test 83 pass, 1 fail (84 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/bind-before-pages.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/bind-before-pages.log new file mode 100644 index 0000000000..6962064604 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/bind-before-pages.log @@ -0,0 +1,98 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x6ae2f6d --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/54c1ef3d6b87a3fce9f1f7c1305e45c0/test --cache-dir=./.zig-cache --seed=0x6ae2f6d --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/86e148cd8864fbf3af01d6e5f8e28899/test --cache-dir=./.zig-cache --seed=0x6ae2f6d --listen=- + +test-cuda-host ++- run test 83 pass, 1 fail (84 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.TF_KV_PAGES=all: a stream past the paged sequences waits for one, and its slot is never made a whole window' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected 3, found 4 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1275e09 in expectEqualInner__func_954 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x13fc332 in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:170:5: 0x13fc3da in test.TF_KV_PAGES=all: a stream past the paged sequences waits for one, and its slot is never made a whole window (cuda.zig) + try expectEqual(live, F.live); // nothing allocated for it + ^ +failed command: ./.zig-cache/o/b259aa6099dd862e78b6ea60d2015a2b/test --cache-dir=./.zig-cache --seed=0x6ae2f6d --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x6ae2f6d --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 253/254 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 83 pass, 1 fail (84 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/bytes-on-paged-seq.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/bytes-on-paged-seq.log new file mode 100644 index 0000000000..b96d880bfa --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/bytes-on-paged-seq.log @@ -0,0 +1,98 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/6d28bc455f23d336015a66a45a18c781/test --cache-dir=./.zig-cache --seed=0x8a4620cd --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/0b6b80618698da513e512dcd409ba85e/test --cache-dir=./.zig-cache --seed=0x8a4620cd --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x8a4620cd --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x8a4620cd --listen=- + +test-glm-snapshots-host ++- run test 14 pass, 1 fail (15 total) +error: 'glm_snapshot_host.test.GLM snapshots: while a paged stream is current, a prefix's size is the engine's own sequence's, past its pages' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + expected 6291456, found 18446744073709551615 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245553 in expectEqualInner__func_966 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x12461c4 in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:488:5: 0x1246281 in test.GLM snapshots: while a paged stream is current, a prefix's size is the engine's own sequence's, past its pages (glm_snapshot_host.zig) + try t.expectEqual(own, f.e.stateBytes(4096)); + ^ +failed command: ./.zig-cache/o/15b6e6654ef4b8c20230d48b26c606f9/test --cache-dir=./.zig-cache --seed=0x8a4620cd --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 253/254 tests passed (1 failed) +test-glm-snapshots-host transitive failure ++- run test 14 pass, 1 fail (15 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/close-keeps-pages.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/close-keeps-pages.log new file mode 100644 index 0000000000..968e7ad28c --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/close-keeps-pages.log @@ -0,0 +1,97 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xa446069b --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/8f999d1e10fd7824d8274bf18e378bc2/test --cache-dir=./.zig-cache --seed=0xa446069b --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0xa446069b --listen=- + +test-cuda-host ++- run test 83 pass, 1 fail (84 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + /zt/kvp-mut/zig/src/core/kv_pool.zig:203:32: 0x13f9d44 in promise (root.zig) + if (p.available() < n) return error.LaneMemory; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:119:9: 0x13f6fcf in open (cuda.zig) + try p.pool.promise(l, n); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:120:56: 0x155080d in test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back (cuda.zig) + try std.testing.expectEqualSlices(u32, &.{ 1, 0 }, try p.open(1, 600, 100)); // the last page given back, first + ^ +failed command: ./.zig-cache/o/41a58a9d9b41a37679753ddb3ce349fe/test --cache-dir=./.zig-cache --seed=0xa446069b --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/42cd170b45f5c730291b1c56d0deb433/test --cache-dir=./.zig-cache --seed=0xa446069b --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 253/254 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 83 pass, 1 fail (84 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/fp8-kernel-table.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/fp8-kernel-table.log new file mode 100644 index 0000000000..081e011e6d --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/fp8-kernel-table.log @@ -0,0 +1,104 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test 11 pass, 1 fail (12 total) +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +error: 'cuda_kv8_test.test.GLM FP8 KV: an FP8 cache takes no page table, so its writes and reads refuse one before they launch' failed: + expected error.Fp8OnPages, found void + /.local/share/zig/0.17.0/lib/std/testing.zig:634:9: 0x12707db in expectError__func_1068 (std.zig) + return error.TestExpectedError; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_kv8_test.zig:143:5: 0x12715bb in test.GLM FP8 KV: an FP8 cache takes no page table, so its writes and reads refuse one before they launch (cuda.zig) + try t.expectError(error.Fp8OnPages, latent.latentWriteFormat(set, stream, 100, 512, 200, 400, paged, 300, 3, 512)); + ^ +failed command: ./.zig-cache/o/d1aa559c5956637f0a2670ea3e6fc85a/test --cache-dir=./.zig-cache --seed=0x7819bf12 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x7819bf12 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/df575ee2b6723b70a74444009b245f6a/test --cache-dir=./.zig-cache --seed=0x7819bf12 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x7819bf12 --listen=- + +test-cuda-host ++- run test 83 pass, 1 fail (84 total) +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_kv8_test.test.GLM FP8 KV: an FP8 cache takes no page table, so its writes and reads refuse one before they launch' failed: + expected error.Fp8OnPages, found void + /.local/share/zig/0.17.0/lib/std/testing.zig:634:9: 0x1597a5b in expectError__func_2277 (std.zig) + return error.TestExpectedError; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_kv8_test.zig:143:5: 0x159883b in test.GLM FP8 KV: an FP8 cache takes no page table, so its writes and reads refuse one before they launch (cuda.zig) + try t.expectError(error.Fp8OnPages, latent.latentWriteFormat(set, stream, 100, 512, 200, 400, paged, 300, 3, 512)); + ^ +failed command: ./.zig-cache/o/9b3ac620742f5e4666ec94e04a8811c8/test --cache-dir=./.zig-cache --seed=0x7819bf12 --listen=- + +Build Summary: 160/163 steps succeeded (2 failed); 252/254 tests passed (2 failed) +test-cuda-host transitive failure ++- run test 11 pass, 1 fail (12 total) ++- run test 83 pass, 1 fail (84 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/fp8-on-pages.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/fp8-on-pages.log new file mode 100644 index 0000000000..bf8364abc5 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/fp8-on-pages.log @@ -0,0 +1,95 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test 83 pass, 1 fail (84 total) +error: 'cuda_native.test.TF_KV_PAGES with --kv-cache fp8: auto and off keep windows at every rank count, on and all are refused by name' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + expected error.Fp8OnPages, found .on + /.local/share/zig/0.17.0/lib/std/testing.zig:634:9: 0x1289c73 in expectError__func_1030 (std.zig) + return error.TestExpectedError; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_native.zig:395:53: 0x128a14e in test.TF_KV_PAGES with --kv-cache fp8: auto and off keep windows at every rank count, on and all are refused by name (cuda.zig) + for ([_][]const u8{ "on", "1", "all" }) |v| try std.testing.expectError(error.Fp8OnPages, paging(v, w, true)); + ^ +failed command: ./.zig-cache/o/e21baeb43a2a10a5f06790e16ea4b93c/test --cache-dir=./.zig-cache --seed=0x8e84696e --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/5aa559e7730d8b17e1595c18c5b8f2e4/test --cache-dir=./.zig-cache --seed=0x8e84696e --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x8e84696e --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x8e84696e --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/a29397b3a1d61fab05f03ffa3a569475/test --cache-dir=./.zig-cache --seed=0x8e84696e --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 254/255 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 83 pass, 1 fail (84 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/head.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/head.log new file mode 100644 index 0000000000..e5b1ec16a2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/head.log @@ -0,0 +1,109 @@ +test-cuda-host ++- run test 0 pass, 1 fail (1 total) +ectedResult; + ^ +/zt/kvp-mut/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql(u8, &first, &changed)); + ^ +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexp +failed command: ./.zig-cache/o/5e598aff1d4f9edbdfba42f5b9bf1824/test --cache-dir=./.zig-cache --seed=0xd6e558c6 --listen=- + +test-cuda-host ++- run test w +[default] (warn): libcublasLt.so.13 is required for CUDA vision towers (--vision) and CUDA cuBLASLt projections; install the NVIDIA CUDA 13 cuBLAS runtime; on Linux, add its directory to LD_LIBRARY_PATH; a driver-only host does not include it +[default] (warn): libnccl.so.2 is required for CUDA ranks (including the GLM pair); install the NVIDIA NCCL 2 runtime compatible with CUDA 13; on Linux, add its directory to LD_LIBRARY_PATH +[default] (warn): libibverbs.so.1 is required for RoCE/RDMA gathers; install rdma-core's libibverbs runtime and the provider for the network adapter +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +failed command: ./.zig-cache/o/4c4ab1edef8b3cb8520de09e0a44dfc7/test --cache-dir=./.zig-cache --seed=0xd6e558c6 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xd6e558c6 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/077c47a870d1092b64323c69a48960e1/test --cache-dir=./.zig-cache --seed=0xd6e558c6 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/dd228f20f08563a14f5e85099f524636/test --cache-dir=./.zig-cache --seed=0xd6e558c6 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/5136546b9cf70b2931a6ff301208c4be/test --cache-dir=./.zig-cache --seed=0xd6e558c6 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/f255879e004629137a696482f5f8dffc/test --cache-dir=./.zig-cache --seed=0xd6e558c6 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 265/266 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 0 pass, 1 fail (1 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/kept-window-cache.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/kept-window-cache.log new file mode 100644 index 0000000000..db69ebf213 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/kept-window-cache.log @@ -0,0 +1,305 @@ +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/522a3c714e805e2829439e1b48b5fdd8/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/66ebc0ef26864792f1e985d29dc6f948/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/7c7824519733d227e57316fb16a39829/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/0eefa14075b1347761dd7309dc31d967/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test 71 pass, 1 fail (72 total) +error: 'cuda_kv_pool.test.kv pages: a family without snapshot slots keeps the paged plan's cache, not what window-sized streams leave' failed: + expected 17179869184, found 4294967296 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1897fc3 in expectEqualInner__func_3087 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x18ad76f in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:164:5: 0x18ad7c8 in test.kv pages: a family without snapshot slots keeps the paged plan's cache, not what window-sized streams leave (cuda.zig) + try std.testing.expectEqual(@as(u64, 16 << 30), k.copied); + ^ +failed command: ./.zig-cache/o/f2fa0ce8e2eced89c9ca076b7b9f6a62/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +failed command: ./.zig-cache/o/ff4eb066a98f732a91f8d17d7542dd9b/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/28962cb2042773d6ebe849ecc7d02046/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test 0 pass, 1 fail (1 total) + the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql(u8, &first, &changed)); + ^ +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at +failed command: ./.zig-cache/o/5e598aff1d4f9edbdfba42f5b9bf1824/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/18a0eaa687a52b0c5174fce22122003e/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/bc5cb96ccdee01707cb3ff47636e83d0/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/d46e2e5a20a08f2375c004c824c68d90/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/dd228f20f08563a14f5e85099f524636/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-cuda-host ++- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/e3742d1c6ce0ffacd1072cfaf9e1dcf1/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/5ba9dbc67c1769ccc937a1788b76fe9c/test --cache-dir=./.zig-cache --seed=0x4b3410f5 --listen=- + +Build Summary: 160/163 steps succeeded (2 failed); 1044/1050 tests passed (4 skipped, 2 failed) +test-cuda-host transitive failure ++- run test 0 pass, 1 fail (1 total) ++- run test 71 pass, 1 fail (72 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/map-any-id.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/map-any-id.log new file mode 100644 index 0000000000..a37a7f077a --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/map-any-id.log @@ -0,0 +1,95 @@ +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x6f8ea482 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/a9b0f1894c9e03da72d3ba7114bd65c3/test --cache-dir=./.zig-cache --seed=0x6f8ea482 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x6f8ea482 --listen=- + +test-cuda-host ++- run test 83 pass, 1 fail (84 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected error.BadStep, found void + /.local/share/zig/0.17.0/lib/std/testing.zig:634:9: 0x154e1cb in expectError__func_2149 (std.zig) + return error.TestExpectedError; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:137:5: 0x154e8f5 in test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged (cuda.zig) + try std.testing.expectError(error.BadStep, p.map(1, &.{8})); + ^ +failed command: ./.zig-cache/o/47f87d657ea9d0a6501e5601ce873f57/test --cache-dir=./.zig-cache --seed=0x6f8ea482 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/ad5e36e23d0f57552a3e551926a39fea/test --cache-dir=./.zig-cache --seed=0x6f8ea482 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 253/254 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 83 pass, 1 fail (84 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/no-poison.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/no-poison.log new file mode 100644 index 0000000000..ddeff95142 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/no-poison.log @@ -0,0 +1,98 @@ +test-cuda-host ++- run test 83 pass, 1 fail (84 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected 3, found 0 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1275e09 in expectEqualInner__func_954 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x13f733b in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:188:5: 0x13f73b6 in test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written (cuda.zig) + try expectEqual(@as(usize, 3), F.fills); + ^ +failed command: ./.zig-cache/o/3b117c59bb49a6eaff388b7bc01a6a4b/test --cache-dir=./.zig-cache --seed=0x40bb90fc --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/54270bd8b4e8fd830b7e7b8c06de42c6/test --cache-dir=./.zig-cache --seed=0x40bb90fc --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x40bb90fc --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x40bb90fc --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/ed1aba956d0c03cf9e8ec9a65764db7f/test --cache-dir=./.zig-cache --seed=0x40bb90fc --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 253/254 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 83 pass, 1 fail (84 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/page-bytes-unchecked.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/page-bytes-unchecked.log new file mode 100644 index 0000000000..61255e3fcf --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/page-bytes-unchecked.log @@ -0,0 +1,135 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test 83 pass, 1 crash (84 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool' terminated with signal ABRT with stderr: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected error.RankMismatch, found .{ .gpa = .{ .ptr = anyopaque@160ec80, .vtable = .{ .alloc = fn (*anyopaque, usize, mem.Alignment, usize) ?[*]u8@12527d0, .resize = fn (*anyopaque, []u8, mem.Alignment, usize, usize) bool@125d520, .remap = fn (*anyopaque, []u8, mem.Alignment, usize, usize) ?[*]u8@12532c0, .free = fn (*anyopaque, []u8, mem.Alignment, usize) void@125ddf0 } }, .e = .{ .gpa = .{ .ptr = anyopaque@160ec80, .vtable = .{ ... } }, .io = .{ .userdata = anyopaque@aaaaaaaaaaaaaaaa, .vtable = General protection exception (no address available) + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1488:71: 0x151d60e in printValue__func_2043 (std.zig) + .@"enum", .@"union", .@"struct" => return w.printValue(fmt, options, value.*, max_depth), + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1481:33: 0x151cf22 in printValue__func_1736 (std.zig) + try w.printValue(ANY, options, @field(value, f_name), max_depth - 1); + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1481:33: 0x14033c3 in printValue__func_1735 (std.zig) + try w.printValue(ANY, options, @field(value, f_name), max_depth - 1); + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1488:71: 0x14025f2 in printValue__func_1729 (std.zig) + .@"enum", .@"union", .@"struct" => return w.printValue(fmt, options, value.*, max_depth), + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1481:33: 0x14006b9 in printValue__func_1728 (std.zig) + try w.printValue(ANY, options, @field(value, f_name), max_depth - 1); + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1488:71: 0x13ffdea in printValue__func_1727 (std.zig) + .@"enum", .@"union", .@"struct" => return w.printValue(fmt, options, value.*, max_depth), + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:805:25: 0x13ff9b3 in print__func_1726 (std.zig) + try w.printValue( + ^ + /.local/share/zig/0.17.0/lib/std/debug.zig:330:39: 0x13ff5ba in print__func_1725 (std.zig) + stderr.file_writer.interface.print(fmt, args) catch return; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:47:24: 0x13ff463 in failPrint__func_1724 (std.zig) + std.debug.print(fmt, args); + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:633:18: 0x13fec01 in expectError__func_1723 (std.zig) + failPrint("expected error.{s}, found {any}\n", .{ @errorName(expected_error), actual_payload }); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:149:32: 0x13fef44 in test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool (cuda.zig) + try std.testing.expectError(error.RankMismatch, pages_mod.Pages.follow(std.testing.allocator, &f.e, 2, 8, more)); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7384fea2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x7384fea2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x15c60e0 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/a2dc34497a1af0129cd47dc0aa18eb6c/test --cache-dir=./.zig-cache --seed=0x6a94415d --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x6a94415d --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x6a94415d --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/9410324896af0b1cbafb330e08eb1941/test --cache-dir=./.zig-cache --seed=0x6a94415d --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/1228707ec4548e67b5c19eac0b4b868d/test --cache-dir=./.zig-cache --seed=0x6a94415d --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 253/254 tests passed (1 crashed) +test-cuda-host transitive failure ++- run test 83 pass, 1 crash (84 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/paged-first-at-cache.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/paged-first-at-cache.log new file mode 100644 index 0000000000..cf81001349 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/paged-first-at-cache.log @@ -0,0 +1,100 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/a673ceb6ef8a0ab1e642a08d4cf6b255/test --cache-dir=./.zig-cache --seed=0x4fc7633c --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/db85c1e49b820903017527cc1d2f8047/test --cache-dir=./.zig-cache --seed=0x4fc7633c --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x4fc7633c --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x4fc7633c --listen=- + +test-glm-snapshots-host ++- run test 14 pass, 1 fail (15 total) +error: 'glm_snapshot_host.test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + /zt/kvp-mut/zig/src/families/glm/cuda_snapshot_layout.zig:71:13: 0x1248aee in add (cuda.zig) + return error.SnapshotShape; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_snapshot_layout.zig:24:9: 0x124a47c in init (cuda.zig) + try p.add(st, st.arena.buf.ptr, @intCast(first - st.arena.buf.ptr)); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_snapshots.zig:65:18: 0x125a7cf in restore__func_1009 (cuda.zig) + const plan = try layout.Plan.init(&e.cur.st, &e.c, s.pos, if (e.mtp) s.mtp_len else null); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:435:13: 0x1261d4e in test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window (glm_snapshot_host.zig) + try f.e.restoreState(id); + ^ +failed command: ./.zig-cache/o/f7227f2742284b96901cbaf15ef57563/test --cache-dir=./.zig-cache --seed=0x4fc7633c --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 253/254 tests passed (1 failed) +test-glm-snapshots-host transitive failure ++- run test 14 pass, 1 fail (15 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/paged-table-kept.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/paged-table-kept.log new file mode 100644 index 0000000000..7c18bbaeae --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/paged-table-kept.log @@ -0,0 +1,94 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x5f7a3767 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/b84f44240dd8f1199613ea53e108ac9d/test --cache-dir=./.zig-cache --seed=0x5f7a3767 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0x5f7a3767 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/5d143d9c911efcdccae9490a03a59636/test --cache-dir=./.zig-cache --seed=0x5f7a3767 --listen=- + +test-glm-snapshots-host ++- run test 14 pass, 1 fail (15 total) +error: 'glm_snapshot_host.test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + /zt/kvp-mut/zig/src/families/glm/cuda_snapshots.zig:66:28: 0x125a8d0 in restore__func_1009 (cuda.zig) + if (plan.len != s.len) return error.SnapshotSize; + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:435:13: 0x1261dbe in test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window (glm_snapshot_host.zig) + try f.e.restoreState(id); + ^ +failed command: ./.zig-cache/o/14c940ddf9e547048f8d1cdc758084b9/test --cache-dir=./.zig-cache --seed=0x5f7a3767 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 253/254 tests passed (1 failed) +test-glm-snapshots-host transitive failure ++- run test 14 pass, 1 fail (15 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/piece-first-page.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/piece-first-page.log new file mode 100644 index 0000000000..5e0cc407ad --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/piece-first-page.log @@ -0,0 +1,123 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xa9585a23 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/842eb7b081220f35d188fa51d731a5f9/test --cache-dir=./.zig-cache --seed=0xa9585a23 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/dfa28f5f5c5bce09d63efddbb5d8abe7/test --cache-dir=./.zig-cache --seed=0xa9585a23 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0xa9585a23 --listen=- + +test-glm-snapshots-host ++- run test 13 pass, 2 fail (15 total) +error: 'glm_snapshot_host.test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + slices differ. first difference occurs at index 0 (0x0) + + ============ expected this output: ============= len: 16 (0x10) + + EE 05 38 5F 72 A9 CC E3 06 3D 50 77 AA C1 E4 1B ..8_r....=Pw.... + + ============= instead found this: ============== len: 16 (0x10) + + E6 1D 30 57 8A A1 C4 FB 1E 35 68 8F A2 D9 FC 13 ..0W.....5h..... + + ================================================ + + /.local/share/zig/0.17.0/lib/std/testing.zig:346:5: 0x125e78d in expectEqualSlices__func_1010 (std.zig) + return error.TestExpectedEqual; + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:383:5: 0x125e96c in same (glm_snapshot_host.zig) + try t.expectEqualSlices(u8, rec.bytes(a, len), rec.bytes(b, len)); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:404:49: 0x12602a4 in expectSamePrefix (glm_snapshot_host.zig) + for (0..@min(n / 4 + 2, x.pk_rows)) |p| try same(rowAt(a, x, x.pk, p, c.i_dim * 2, 128), rowAt(b, y, y.pk, p, c.i_dim * 2, 128), c.i_dim * 2); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:436:13: 0x1261ee1 in test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window (glm_snapshot_host.zig) + try expectSamePrefix(c, &window.st, &paged.st, at, head); + ^ +error: 'glm_snapshot_host.test.GLM snapshots: a paged part's piece from inside a page's span keeps its offset, on the page the order names' failed: + expected 1048606, found 1060894 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245553 in expectEqualInner__func_966 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x125533e in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:461:5: 0x1255397 in test.GLM snapshots: a paged part's piece from inside a page's span keeps its offset, on the page the order names (glm_snapshot_host.zig) + try t.expectEqual(@as(u64, (1 << 20) + 30), a.live); + ^ +failed command: ./.zig-cache/o/1d37973abcf60118e3a4c3ffaba3ed37/test --cache-dir=./.zig-cache --seed=0xa9585a23 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 252/254 tests passed (2 failed) +test-glm-snapshots-host transitive failure ++- run test 13 pass, 2 fail (15 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/piece-no-offset.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/piece-no-offset.log new file mode 100644 index 0000000000..b95cd512e3 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/piece-no-offset.log @@ -0,0 +1,98 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/b23f4776e45c71f302d4bb75e01f1732/test --cache-dir=./.zig-cache --seed=0xc7e9e979 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xc7e9e979 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/03f27579295a599769f8c15e2a5c8c7e/test --cache-dir=./.zig-cache --seed=0xc7e9e979 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/d8160a9eeb31363374b14e77637836bf/test --cache-dir=./.zig-cache --seed=0xc7e9e979 --listen=- + +test-glm-snapshots-host ++- run test 14 pass, 1 fail (15 total) +error: 'glm_snapshot_host.test.GLM snapshots: a paged part's piece from inside a page's span keeps its offset, on the page the order names' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + expected 1048606, found 1048576 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245553 in expectEqualInner__func_966 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x125533e in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:461:5: 0x1255397 in test.GLM snapshots: a paged part's piece from inside a page's span keeps its offset, on the page the order names (glm_snapshot_host.zig) + try t.expectEqual(@as(u64, (1 << 20) + 30), a.live); + ^ +failed command: ./.zig-cache/o/bae4440fc081f5ec4777ca746b548cfd/test --cache-dir=./.zig-cache --seed=0xc7e9e979 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 253/254 tests passed (1 failed) +test-glm-snapshots-host transitive failure ++- run test 14 pass, 1 fail (15 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/split-no-slots.log b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/split-no-slots.log new file mode 100644 index 0000000000..a531293168 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/mutations/split-no-slots.log @@ -0,0 +1,354 @@ +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/af2f1899e29caf0f0d35a7ad420a733a/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-kv-pool ++- run test 9 pass, 1 crash (10 total) +error: 'kv_pool_test.test.a family without snapshot slots (snap_bytes 0) keeps its whole cache as copied rows beside the pages' terminated with signal ABRT with stderr: + thread 8362 panic: division by zero + /zt/kvp-mut/zig/src/core/kv_pool.zig:57:59: 0x124a11f in split (kv_pool.zig) + const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:59:23: 0x124a980 in test.a family without snapshot slots (snap_bytes 0) keeps its whole cache as copied rows beside the pages (kv_pool.zig) + const s = kv.split(16 * g, 0, null); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x11e78ea in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x11e871d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x11e5428 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x76317142a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x76317142a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x1273b80 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/60d8dfd727b922786662e0a5baa04c7d/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/4f2d433fcc9f2b86294d64b55d2f872f/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/f9d42695841c3e3d201b2463bb670593/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/15ea11f54fcb3a4bb59605b68d3c7f0a/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/0403d4a2a54efc2d373bc55a4e88963d/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/e388b5b93b41f0ed26e6c82f093859ec/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test 350 pass, 1 crash (351 total) +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +error: 'kv_pool_test.test.a family without snapshot slots (snap_bytes 0) keeps its whole cache as copied rows beside the pages' terminated with signal ABRT with stderr: + thread 8372 panic: division by zero + /zt/kvp-mut/zig/src/core/kv_pool.zig:57:59: 0x13a4c3f in split (root.zig) + const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:59:23: 0x13a54a0 in test.a family without snapshot slots (snap_bytes 0) keeps its whole cache as copied rows beside the pages (root.zig) + const s = kv.split(16 * g, 0, null); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x127c9ba in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x127d7ed in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x127a4f8 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7246b5a2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x7246b5a2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x18c4770 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/95e1f11febf0419dc0bbd475a795c043/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test 71 pass, 1 crash (72 total) +error: 'cuda_kv_pool.test.kv pages: a family without snapshot slots keeps the paged plan's cache, not what window-sized streams leave' terminated with signal ABRT with stderr: + thread 10572 panic: division by zero + /zt/kvp-mut/zig/src/core/kv_pool.zig:57:59: 0x189a2bf in split (root.zig) + const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); + ^ + /zt/kvp-mut/zig/src/core/kv_pool.zig:70:20: 0x189bd55 in plan (root.zig) + const c = split(@min(b.cache, left - fit * l.pageBytes()), l.snap_bytes, b.snaps); + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:161:31: 0x18ad53c in test.kv pages: a family without snapshot slots keeps the paged plan's cache, not what window-sized streams leave (cuda.zig) + const p = try kv_pool.plan(l, .{ .room = 40 << 30, .engine = 0, .lanes = 4, .cache = 16 << 30 }); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x12faada in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x12fb90d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x12f8618 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x73fedda2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x73fedda2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x1b74c90 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/f2f83e982948e01e227ce8de6414cdba/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/7f34815f5702eb8897d709525851b9cc/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/1df021322b311fa73a17f8b16dacdce9/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/e12be92274c552437395f0aec43c9e2c/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/7e4f9534d0f74f08330480343c7edc9c/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/deea4e792a2c23a39f127f9bd1027d24/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/334a1825bc6b9675d3dee627452afe17/test --cache-dir=./.zig-cache --seed=0xe4775c52 --listen=- + +Build Summary: 158/163 steps succeeded (3 failed); 1043/1050 tests passed (4 skipped, 3 crashed) +test-cuda-host transitive failure ++- run test 350 pass, 1 crash (351 total) ++- run test 71 pass, 1 crash (72 total) +test-kv-pool transitive failure ++- run test 9 pass, 1 crash (10 total) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/security.log b/docs/receipts/glm-kv-pages-0b094b74/ci/security.log new file mode 100644 index 0000000000..058c1f487c --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/security.log @@ -0,0 +1,38 @@ +PASS key required for /systemone +PASS valid key admits /systemone +PASS key required for /alternative/systemone +PASS valid key admits /alternative/systemone +PASS key required for /systemone/?x=1 +PASS valid key admits /systemone/?x=1 +PASS key required for /v1/systemone +PASS valid key admits /v1/systemone +PASS GET body cannot become another request +PASS GET chunked body stays framed +PASS DELETE body cannot become another request +PASS DELETE chunked body stays framed +PASS malformed header 0 closes before routing +PASS malformed header 1 closes before routing +PASS malformed header 2 closes before routing +PASS malformed header 3 closes before routing +PASS malformed header 4 closes before routing +PASS malformed header 5 closes before routing +PASS malformed header 6 closes before routing +PASS malformed header 7 closes before routing +PASS truncated request head cannot dispatch +PASS GET conflicting framing is rejected +PASS GET body preserves a genuine following request +PASS ordinary pipelining remains supported +SUMMARY server-security: 24 passed, 0 failed, 0 skipped +Build Summary: 12/12 steps succeeded; 20/20 tests passed +test-pull-security success ++- run test 1 pass (1 total) 18ms MaxRSS:6M +| +- compile test debug native success 805ms MaxRSS:147M ++- run test 4 pass (4 total) 3s MaxRSS:9M +| +- compile test debug native success 1s MaxRSS:203M ++- run test 1 pass (1 total) 13ms MaxRSS:6M + +- compile test debug native success 628ms MaxRSS:155M +test-server-security success ++- run python3 success 351ms +| +- compile exe fake_serve debug native cached 63ms MaxRSS:38M ++- run test 14 pass (14 total) 9ms MaxRSS:6M + +- compile test debug native success 748ms MaxRSS:169M diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/test-golden.log b/docs/receipts/glm-kv-pages-0b094b74/ci/test-golden.log new file mode 100644 index 0000000000..8d0c9b48d3 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/test-golden.log @@ -0,0 +1,82 @@ +KNOWN-DIFFER tools/two_tools-stream + reply 0 body differs at 313: golden [content": "\n"}, "finish_reason": null}]} + +data: {"id": "", "type": "function"] +KNOWN-DIFFER tools/two_tools-single-stream + reply 0 body differs at 313: golden [content": "\n"}, "finish_reason": null}]} + +data: {"id": "", "type": "function"] +KNOWN-DIFFER tools/choice-"required" + reply 0 status: golden HTTP/1.1 200 OK, zig HTTP/1.1 400 Bad Request + reply 0 body differs at 2: golden [id": "", "object": "chat.completion", "created": , ] zig [error": {"message": "backend 'lanes' cannot enforce tool_cho] + reply 0 headers differ +KNOWN-DIFFER tools/choice-{"type": "function", "function": {"name": "lookup"}} + reply 0 status: golden HTTP/1.1 200 OK, zig HTTP/1.1 400 Bad Request + reply 0 body differs at 2: golden [id": "", "object": "chat.completion", "created": , ] zig [error": {"message": "backend 'lanes' cannot enforce tool_cho] + reply 0 headers differ +KNOWN-DIFFER errors/format-json + reply 0 body differs at 41: golden [needs the checkpoint's config.json vocab_size", "type": "inv] zig [(response_format and the guided_* fields) is not supported b] + reply 0 headers differ +KNOWN-DIFFER errors/stream-structured + connection after: golden closed, zig open + reply 0 status: golden HTTP/1.1 200 OK, zig HTTP/1.1 400 Bad Request + reply 0 body differs at 0: golden [data: {"id": "", "object": "chat.completion.chunk", "cr] zig [{"error": {"message": "structured output (response_format an] +KNOWN-DIFFER anthropic/tool-use-stream + reply 0 body differs at 704: golden [\n"}} + +event: content_block_stop +data: {"type": "content_blo] zig ["}} + +event: content_block_stop +data: {"type": "content_block] +KNOWN-DIFFER anthropic/tool-result + reply 0 status: golden HTTP/1.1 200 OK, zig HTTP/1.1 400 Bad Request + reply 0 body differs at 2: golden [id": "", "type": "message", "role": "assistant", "model] zig [type": "error", "error": {"type": "invalid_request_error", "] + reply 0 headers differ +KNOWN-DIFFER anthropic/refuse-format + reply 0 body differs at 91: golden [needs the checkpoint's config.json vocab_size"}}] zig [(response_format and the guided_* fields) is not supported b] + reply 0 headers differ +KNOWN-DIFFER cancel/metrics-after + reply 0 body differs at 307: golden [prompt_tokens_total Prompt tokens of finished requests. +# TY] zig [generation_tokens_running Generated tokens held by live stre] + reply 0 headers differ +KNOWN-DIFFER metrics-open/metrics + reply 0 body differs at 307: golden [prompt_tokens_total Prompt tokens of finished requests. +# TY] zig [generation_tokens_running Generated tokens held by live stre] + reply 0 headers differ +KNOWN-DIFFER metrics-open/v1-metrics + reply 0 body differs at 487: golden [prompt_tokens_total Prompt tokens of finished requests. +# TY] zig [generation_tokens_running Generated tokens held by live stre] + reply 0 headers differ +KNOWN-DIFFER keys/metrics-counted + reply 0 body differs at 887: golden [prompt_tokens_total Prompt tokens of finished requests. +# TY] zig [generation_tokens_running Generated tokens held by live stre] + reply 0 headers differ + chat 16 equal 0 differ 0 known + thinking 16 equal 0 differ 0 known + stops 13 equal 0 differ 0 known + usage 10 equal 0 differ 0 known + tools 48 equal 0 differ 4 known + errors 37 equal 0 differ 2 known + completions 15 equal 0 differ 0 known + responses 29 equal 0 differ 0 known + anthropic 26 equal 0 differ 3 known + tokens 19 equal 0 differ 0 known + status 11 equal 0 differ 0 known + framing 42 equal 0 differ 0 known + cancel 1 equal 0 differ 1 known + lifecycle 2 equal 0 differ 0 known +metrics-open 2 equal 0 differ 2 known + keys 19 equal 0 differ 1 known +306/319 equal to server goldens, 0 unexpected differences + keys 6 equal 0 differ 0 known + origin 20 equal 0 differ 0 known +26/26 equal to server goldens, 0 unexpected differences +Build Summary: 5/5 steps succeeded +test-golden success ++- run exe golden_check success 1m +| +- compile exe golden_check debug native success 619ms MaxRSS:149M +| +- compile exe fake_serve debug native success 3s MaxRSS:322M ++- run exe golden_check success 4s + +- compile exe golden_check debug native (reused) + +- compile exe fake_serve debug native (reused) diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/test-kernel-contract.log b/docs/receipts/glm-kv-pages-0b094b74/ci/test-kernel-contract.log new file mode 100644 index 0000000000..b5ef0eed5d --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/test-kernel-contract.log @@ -0,0 +1,12 @@ +Build Summary: 11/11 steps succeeded; 72/72 tests passed +test-kernel-contract success ++- run test 53 pass (53 total) 845ms MaxRSS:10M +| +- compile test debug native success 5s MaxRSS:168M ++- test-gemma4-contract-host success +| +- run test 9 pass (9 total) 26s MaxRSS:14M +| +- compile test debug native success 1s MaxRSS:224M +| +- WriteFile gemma4-host-empty.metallib success ++- run test 3 pass (3 total) 17ms MaxRSS:6M +| +- compile test debug native success 1s MaxRSS:150M ++- run test 7 pass (7 total) 1s MaxRSS:55M + +- compile test debug native success 806ms MaxRSS:149M diff --git a/docs/receipts/glm-kv-pages-0b094b74/ci/test.log b/docs/receipts/glm-kv-pages-0b094b74/ci/test.log new file mode 100644 index 0000000000..daac8b53f3 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/ci/test.log @@ -0,0 +1,900 @@ +test ++- run test w +[default] (warn): XPU: a command list did not drain at close; its memory and context stay until the process ends +failed command: ./.zig-cache/o/c4054c8ba5e1acde7faea349e8073ec3/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- run test w +[default] (warn): flash next: rotary cache table unavailable: OutOfMemory +failed command: ./.zig-cache/o/5b544bf982f782ae0f922f2817138319/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- run test w +[default] (warn): a round could not be recorded: OutOfDeviceMemory +failed command: ./.zig-cache/o/46705ef66f5a7c36d5e3d4d863ca7115/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +failed command: ./.zig-cache/o/08801bb786532554ae398b11b98db3f2/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/c79d0f27b0b3d8460f716fe6b4e343cd/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/ac6b6a1b5825518c3eaad55eec9e5c6d/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/9b8430079bb9d01dcc1cf517c720044b/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/c0f97414abe76845be3004e2e29056d5/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/dd228f20f08563a14f5e85099f524636/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/bc5cb96ccdee01707cb3ff47636e83d0/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/077c47a870d1092b64323c69a48960e1/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (err): rank 0: another rank's process ended (its line closed); this rank stops instead of waiting on it +[default] (err): rank 0: rank 1 stopped (peer OOM); this rank stops too +failed command: ./.zig-cache/o/8000cd26070d78c69d965bddff4a4b1e/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/a3add4cbb59446f6a79267c7e7253d47/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/e3742d1c6ce0ffacd1072cfaf9e1dcf1/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- run test w +[default] (warn): lane host: stopping at a round boundary: 2 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): 4 prompts together: SamplingRefused; each goes alone +[default] (warn): lane host: a round failed (the planner refused): every admitted stream fails +[default] (warn): lane host: a round failed (PlannerFull): every admitted stream fails +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (drain deadline) +[default] (warn): lane host: stopping at a round boundary: 0 running and 1 queued request(s) end (NoContext) +[default] (warn): lane host: a round failed (OutOfMemory): every admitted stream fails +[default] (warn): lane host: a round failed (AdmittedFailed): every admitted stream fails +[default] (warn): lane host: a round failed (FinalsFailed): every admitted stream fails +failed command: ./.zig-cache/o/fb5469cafe64d369186201f11d5dc056/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/8a68c6993f6d4db0cf1e2f3490554764/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): libcublasLt.so.13 is required for CUDA vision towers (--vision) and CUDA cuBLASLt projections; install the NVIDIA CUDA 13 cuBLAS runtime; on Linux, add its directory to LD_LIBRARY_PATH; a driver-only host does not include it +[default] (warn): libnccl.so.2 is required for CUDA ranks (including the GLM pair); install the NVIDIA NCCL 2 runtime compatible with CUDA 13; on Linux, add its directory to LD_LIBRARY_PATH +[default] (warn): libibverbs.so.1 is required for RoCE/RDMA gathers; install rdma-core's libibverbs runtime and the provider for the network adapter +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +failed command: ./.zig-cache/o/672aacbf22b1ad8f800681808246ba38/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test ++- test-cuda-host + +- run test 0 pass, 1 fail (1 total) +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-fx2/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql(u8, &first, &changed)); + ^ +failed command: ./.zig-cache/o/5e598aff1d4f9edbdfba42f5b9bf1824/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): slide pocket: a deleted fact's 1 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a deleted fact's 1 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a delete failed (UploadFailed), and its put-back did not reach the device (PutBackNotOnDevice) +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a delete failed (UploadFailed), and its put-back did not reach the device (PutBackNotOnDevice) +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +failed command: ./.zig-cache/o/e3d115154491929e494aec103397c87f/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/bc954b5be8f14a9483a426aa8b36cce6/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/cdc1568487a0447a4a2b576f26f7ea43/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): lane host: stopping at a round boundary: 2 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): 4 prompts together: SamplingRefused; each goes alone +[default] (warn): lane host: a round failed (the planner refused): every admitted stream fails +[default] (warn): lane host: a round failed (PlannerFull): every admitted stream fails +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (drain deadline) +[default] (warn): lane host: stopping at a round boundary: 0 running and 1 queued request(s) end (NoContext) +[default] (warn): lane host: a round failed (OutOfMemory): every admitted stream fails +[default] (warn): lane host: a round failed (AdmittedFailed): every admitted stream fails +[default] (warn): lane host: a round failed (FinalsFailed): every admitted stream fails +failed command: ./.zig-cache/o/9d16dac97cd5dc37476baef383e62487/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +failed command: ./.zig-cache/o/39948fb54177a7eb0a6235aad72245ce/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +DEPTH_REPLAY n=2 intercept=50 slope=3 head=1 probe=false deepest=6 crossed={ 62, 62 } last={ 6, 6 } trimmed=0 +DEPTH_REPLAY n=2 intercept=50 slope=3 head=1 probe=true deepest=8 crossed={ 59, 57 } last={ 7, 8 } trimmed=11 +DEPTH_REPLAY n=4 intercept=50 slope=3 head=1 probe=false deepest=4 crossed={ 83, 83, 83, 83 } last={ 4, 4, 4, 4 } trimmed=0 +DEPTH_REPLAY n=4 intercept=50 slope=3 head=1 probe=true deepest=5 crossed={ 75, 83, 83, 76 } last={ 5, 5, 5, 5 } trimmed=9 +DEPTH_REPLAY n=2 intercept=55 slope=2.8 head=1 probe=false deepest=6 crossed={ 61, 61 } last={ 6, 6 } trimmed=0 +DEPTH_REPLAY n=2 intercept=55 slope=2.8 head=1 probe=true deepest=8 crossed={ 54, 56 } last={ 8, 8 } trimmed=2 +DEPTH_REPLAY n=4 intercept=55 slope=2.8 head=1 probe=false deepest=4 crossed={ 82, 82, 82, 82 } last={ 4, 4, 4, 4 } trimmed=0 +DEPTH_REPLAY n=4 intercept=55 slope=2.8 head=1 probe=true deepest=6 crossed={ 75, 71, 72, 74 } last={ 5, 6, 5, 5 } trimmed=9 +DEPTH_REPLAY n=8 intercept=100 slope=0 head=0 probe=false deepest=3 crossed={ 100, 100, 100, 100, 100, 100, 100, 100 } last={ 3, 3, 3, 3, 3, 3, 3, 3 } trimmed=0 +DEPTH_REPLAY n=8 intercept=100 slope=0 head=0 probe=true deepest=3 crossed={ 100, 100, 100, 100, 100, 100, 100, 100 } last={ 3, 3, 3, 3, 3, 3, 3, 3 } trimmed=0 +failed command: ./.zig-cache/o/e1835a97d62c9729c6e0d24440f0ea35/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- run test w +[default] (warn): glm: restoring 1500 tokens failed (PoolMoveNotDrained); pool relocation aborted +failed command: ./.zig-cache/o/ed3605c5d23da03cbf65d8413accabc5/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +failed command: ./.zig-cache/o/92b7bd14bd120d46ca9774cd3436a573/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +test ++- test-cuda-host + +- run test w +PROVENANCE_FIELDS fixtures=3 fields=14 plus decision labels +PROVENANCE_TIMING mode=raw one=0 bytes=409600 added=1000 rounds=12 ns_per=23080573 +PROVENANCE_TIMING mode=masked one=0 bytes=409600 added=1000 rounds=12 ns_per=22903699 +PROVENANCE_MASK_WORK one=0 spans=3 matcher=3 output=409603 +PROVENANCE_TIMING mode=raw one=1 bytes=409600 added=1000 rounds=12 ns_per=23551242 +PROVENANCE_TIMING mode=masked one=1 bytes=409600 added=1000 rounds=12 ns_per=22165630 +PROVENANCE_MASK_WORK one=1 spans=3 matcher=3 output=409603 +[default] (warn): TENSORFOLD_REQUEST_LOG .zig-cache/tmp/FLUPxKHNJWRNq3Bj/taken.jsonl gets no requests: it is not a regular file only its owner can read +VISION_MARKERS all 8 fields refuse plain vocabulary; unflagged additions pass +failed command: ./.zig-cache/o/8c26e0cde058da70f2dbee57f29a6773/test --cache-dir=./.zig-cache --seed=0x74791934 --listen=- + +Build Summary: 273/276 steps succeeded (1 failed); 3463/3479 tests passed (15 skipped, 1 failed) +test transitive failure ++- test-kernel-symbols-host success +| +- run test 3 pass (3 total) 10s MaxRSS:77M +| +- compile test debug native success 3s MaxRSS:146M ++- test-checkpoint-security success +| +- run test 32 pass (32 total) 62ms MaxRSS:16M +| | +- compile test safe native success 37s MaxRSS:545M +| +- run test 5 pass (5 total) 4ms MaxRSS:16M +| | +- compile test safe native success 32s MaxRSS:465M +| +- run test 6 pass (6 total) 3ms MaxRSS:15M +| +- compile test safe native success 25s MaxRSS:423M ++- run test 37 pass (37 total) 145ms MaxRSS:11M +| +- compile test debug native success 1s MaxRSS:154M ++- run test 4 pass (4 total) 84ms MaxRSS:7M +| +- compile test fast x86_64-linux-gnu success 1m MaxRSS:397M ++- run test 24 pass (24 total) 3s MaxRSS:59M +| +- compile test debug native success 2s MaxRSS:156M ++- run test 13 pass (13 total) 1s MaxRSS:9M +| +- compile test debug native success 3s MaxRSS:149M +| +- options success +| +- compile exe digest-writer debug native success 2s MaxRSS:142M ++- run test 6 pass (6 total) 11ms MaxRSS:5M +| +- compile test debug native success 646ms MaxRSS:144M ++- test-flashnext-scheduling-host success +| +- run test 30 pass, 1 skip (31 total) 32s MaxRSS:19M +| +- compile test debug native success 1s MaxRSS:195M +| +- WriteFile flashnext-test-empty.zig success ++- run test 15 pass (15 total) 28ms MaxRSS:6M +| +- compile test debug native success 984ms MaxRSS:160M +| +- WriteFile flashnext-test-empty.zig (reused) ++- test-flashnext-batch-host success +| +- run test 14 pass (14 total) 8s MaxRSS:34M +| +- compile test debug native success 2s MaxRSS:182M +| +- WriteFile flashnext-test-empty.zig (reused) ++- run test 2 pass (2 total) 49ms MaxRSS:6M +| +- compile test debug native success 933ms MaxRSS:144M ++- run test 4 pass (4 total) 13ms MaxRSS:16M +| +- compile test debug native success 537ms MaxRSS:145M ++- run test 4 pass (4 total) 31ms MaxRSS:15M +| +- compile test debug native success 1s MaxRSS:148M ++- run test 3 pass (3 total) 25ms MaxRSS:16M +| +- compile test debug native success 1s MaxRSS:143M ++- run test 23 pass (23 total) 10ms MaxRSS:6M +| +- compile test debug native success 1s MaxRSS:147M ++- run test 29 pass (29 total) 14s MaxRSS:24M +| +- compile test debug native success 2s MaxRSS:161M ++- run test 2 pass (2 total) 2ms MaxRSS:7M +| +- compile test fast native success 57s MaxRSS:420M ++- run test 59 pass, 1 skip (60 total) 1m MaxRSS:51M +| +- compile test debug native success 1s MaxRSS:164M +| | +- options cached +| +- compile obj tp_host_compile debug x86_64-linux-gnu success 781ms MaxRSS:109M +| +- options cached ++- run exe tf-tp-test success 7ms +| +- compile exe tf-tp-test fast native success 1m MaxRSS:621M +| +- WriteFile tp_kernel.zig success +| +- options success ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 10ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 12ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 3ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 6ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- test-cuda-host transitive failure +| +- run test 1 pass (1 total) 33ms MaxRSS:13M +| | +- compile test debug native success 2s MaxRSS:156M +| +- run test 1 pass (1 total) 34ms MaxRSS:13M +| | +- compile test debug native success 1s MaxRSS:144M +| +- run test 10 pass (10 total) 50ms MaxRSS:14M +| | +- compile test debug native success 1s MaxRSS:155M +| +- run test 2 pass (2 total) 21ms MaxRSS:13M +| | +- compile test debug native success 1s MaxRSS:145M +| +- run exe runtime-discovery-exit success 44ms +| | +- compile exe runtime-discovery-exit debug native success 1s MaxRSS:143M +| | +- options success +| | +- compile lib runtime-discovery-mock debug native success 1s MaxRSS:71M +| +- run exe rpc-warning-check success 4ms +| | +- compile exe rpc-warning-check debug native success 1s MaxRSS:140M +| | +- options success +| | +- compile lib rpc-mock-complete debug native success 278ms MaxRSS:61M +| | +- compile lib rpc-mock-abi debug native success 365ms MaxRSS:62M +| | +- compile lib rpc-mock-wait debug native success 295ms MaxRSS:61M +| | +- compile lib rpc-mock-store debug native success 282ms MaxRSS:61M +| | +- compile lib rpc-mock-wrong debug native success 296ms MaxRSS:61M +| +- run test 11 pass (11 total) 26ms MaxRSS:13M +| | +- compile test debug native success 2s MaxRSS:151M +| | +- options (+5 more reused dependencies) +| +- run test 1 pass (1 total) 34ms MaxRSS:14M +| | +- compile test debug native success 1s MaxRSS:144M +| +- run test 30 pass (30 total) 99ms MaxRSS:15M +| | +- compile test debug native success 2s MaxRSS:232M +| | +- WriteFile h3-compile-only.fatbin success +| | +- options cached +| +- run test 3 pass (3 total) 2s MaxRSS:73M +| | +- compile test debug native success 2s MaxRSS:157M +| +- run test 3 pass (3 total) 34ms MaxRSS:15M +| | +- compile test debug native success 11s MaxRSS:168M +| | +- WriteFile attn_train.fatbin success +| | +- WriteFile dsv41_exl3_experts.fatbin success +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- options success +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- options success +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| +- run test 3 pass (3 total) 346ms MaxRSS:10M +| | +- compile test debug native success 2s MaxRSS:177M +| | +- options success +| | | +- compile exe qwen-round-timing-probe debug native success 2s MaxRSS:178M +| | | +- WriteFile draft_ids.txt success +| | | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 10 pass (10 total) 30ms MaxRSS:11M +| | +- compile test debug native success 1s MaxRSS:166M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 7 pass (7 total) 23ms MaxRSS:11M +| | +- compile test debug native success 1s MaxRSS:179M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test 6 pass (6 total) 49ms MaxRSS:11M +| | +- compile test debug native success 1s MaxRSS:177M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test 10 pass (10 total) 928ms MaxRSS:10M +| | +- compile test debug native success 3s MaxRSS:198M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 9 pass (9 total) 166ms MaxRSS:11M +| | +- compile test debug native success 1s MaxRSS:171M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- compile obj exl3_host_compile debug native success 1s MaxRSS:120M +| | +- options (reused) +| +- run test 6 pass (6 total) 38ms MaxRSS:9M +| | +- compile test debug native success 2s MaxRSS:166M +| | +- options (reused) +| +- run test 6 pass (6 total) 331ms MaxRSS:9M +| | +- compile test debug native success 1s MaxRSS:162M +| | +- options (reused) +| +- run test 5 pass (5 total) 112ms MaxRSS:7M +| | +- compile test debug native success 2s MaxRSS:160M +| | +- options (reused) +| +- run test 3 pass (3 total) 35ms MaxRSS:8M +| | +- compile test debug native success 1s MaxRSS:160M +| | +- options (reused) +| +- run test 3 pass (3 total) 79ms MaxRSS:7M +| | +- compile test debug native success 2s MaxRSS:158M +| | +- options (reused) +| +- run test 3 pass (3 total) 118ms MaxRSS:7M +| | +- compile test debug native success 5s MaxRSS:156M +| | +- options (reused) +| +- run test 8 pass (8 total) 27ms MaxRSS:11M +| | +- compile test debug native success 898ms MaxRSS:156M +| +- run test 3 pass (3 total) 358ms MaxRSS:14M +| | +- compile test debug native success 1s MaxRSS:155M +| +- run test 8 pass (8 total) 47ms MaxRSS:13M +| | +- compile test debug native success 1s MaxRSS:156M +| +- run test 4 pass (4 total) 45ms MaxRSS:7M +| | +- compile test debug native success 5s MaxRSS:160M +| | +- options (reused) +| +- run test 1 pass (1 total) 196ms MaxRSS:7M +| | +- compile test debug native success 3s MaxRSS:160M +| | +- options (reused) +| +- run test 4 pass (4 total) 2ms MaxRSS:8M +| | +- compile test fast native success 38s MaxRSS:456M +| | +- options (reused) +| +- run test 1 pass (1 total) 16ms MaxRSS:9M +| | +- compile test fast native success 23s MaxRSS:443M +| | +- options (reused) +| +- compile obj bolt_host_compile debug x86_64-linux-gnu success 1s MaxRSS:143M +| | +- options cached +| | +- WriteFile draft_ids.txt (reused) +| +- run test 26 pass (26 total) 10s MaxRSS:8M +| | +- compile test debug native success 4s MaxRSS:181M +| | +- options (reused) +| +- run test 4 pass (4 total) 24ms MaxRSS:10M +| | +- compile test debug native success 1s MaxRSS:169M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 3 pass (3 total) 42s MaxRSS:42M +| | +- compile test debug native success 2s MaxRSS:217M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run exe qwen-cost-key-process success 25s +| | +- compile exe qwen-cost-key-process debug native success 1s MaxRSS:181M +| | | +- WriteFile draft_ids.txt (reused) +| | | +- options (reused) +| | +- WriteFile target/config.json success +| +- run test 0 pass, 1 fail (1 total) +| | +- compile test debug native success 1s MaxRSS:162M +| +- compile obj drama_cuda_native_host debug native success 1s MaxRSS:131M +| | +- options (reused) +| +- run test 5 pass (5 total) 22ms MaxRSS:12M +| | +- compile test debug native success 1s MaxRSS:145M +| +- run test 5 pass (5 total) 380ms MaxRSS:14M +| | +- compile test debug native success 1s MaxRSS:165M +| +- run test 4 pass (4 total) 38ms MaxRSS:15M +| | +- compile test debug native success 1s MaxRSS:162M +| +- run test 604 pass (604 total) 1m MaxRSS:57M +| | +- compile test debug native success 10s MaxRSS:444M +| +- run test 5 pass (5 total) 24ms MaxRSS:12M +| | +- compile test debug native success 941ms MaxRSS:148M +| +- run test 218 pass (218 total) 666ms MaxRSS:10M +| | +- compile test debug native success 2s MaxRSS:188M +| | +- options (reused) +| +- run test 351 pass (351 total) 21s MaxRSS:136M +| | +- compile test debug native success 12s MaxRSS:233M +| +- run test 212 pass (212 total) 17s MaxRSS:45M +| | +- compile test debug native success 2s MaxRSS:189M +| +- run test 74 pass, 1 skip (75 total) 414ms MaxRSS:14M +| | +- compile test debug native success 3s MaxRSS:218M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 58 pass, 2 skip (60 total) 14s MaxRSS:28M +| | +- compile test debug native success 2s MaxRSS:184M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 12 pass (12 total) 38ms MaxRSS:8M +| | +- compile test debug native success 2s MaxRSS:173M +| | +- options (reused) +| +- run test 97 pass (97 total) 994ms MaxRSS:79M +| | +- compile test debug native success 2s MaxRSS:227M +| | +- options (reused) +| +- run test 15 pass (15 total) 10s MaxRSS:13M +| | +- compile test debug native success 2s MaxRSS:188M +| | +- options (reused) +| +- run test 29 pass (29 total) 17s MaxRSS:1G +| | +- compile test debug native success 4s MaxRSS:176M +| | +- options (reused) +| +- run test 5 pass (5 total) 38ms MaxRSS:14M +| | +- compile test debug native success 1s MaxRSS:149M +| +- run test 36 pass (36 total) 433ms MaxRSS:15M +| | +- compile test debug native success 1s MaxRSS:167M +| +- run test 238 pass (238 total) 17s MaxRSS:45M +| | +- compile test debug native success 3s MaxRSS:209M +| +- run test 72 pass (72 total) 352ms MaxRSS:33M +| | +- compile test debug native success 5s MaxRSS:291M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test 3 pass (3 total) 38ms MaxRSS:13M +| | +- compile test debug native success 2s MaxRSS:144M +| +- run test 1 pass (1 total) 31ms MaxRSS:12M +| | +- compile test debug native success 1s MaxRSS:154M +| +- run test 12 pass (12 total) 120ms MaxRSS:9M +| | +- compile test debug native success 1s MaxRSS:169M +| | +- options (reused) +| +- run test 9 pass (9 total) 26ms MaxRSS:10M +| | +- compile test debug native success 1s MaxRSS:174M +| | +- options (reused) +| +- run test 2 pass (2 total) 39ms MaxRSS:9M +| | +- compile test debug native success 2s MaxRSS:213M +| | +- options (reused) +| +- run test 84 pass (84 total) 173ms MaxRSS:18M +| | +- compile test debug native success 2s MaxRSS:210M +| | +- options (reused) +| +- run test 19 pass, 1 skip (20 total) 151ms MaxRSS:10M +| | +- compile test debug native success 2s MaxRSS:179M +| | +- options (reused) +| +- run test 5 pass (5 total) 29ms MaxRSS:13M +| | +- compile test debug native success 1s MaxRSS:146M +| +- run test 34 pass (34 total) 22s MaxRSS:14M +| | +- compile test debug native success 2s MaxRSS:170M +| +- run test 55 pass (55 total) 4s MaxRSS:14M +| | +- compile test debug native success 2s MaxRSS:175M +| +- run test 50 pass, 1 skip (51 total) 3s MaxRSS:15M +| | +- compile test debug native success 1s MaxRSS:171M +| +- run test 1 pass (1 total) 17ms MaxRSS:12M +| +- compile test debug native success 1s MaxRSS:181M +| +- options success +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile exlr8_alias.zig cached ++- run test 4 pass (4 total) 39ms MaxRSS:6M +| +- compile test debug native success 3s MaxRSS:146M +| +- options cached ++- run test 13 pass, 1 skip (14 total) 90ms MaxRSS:7M +| +- compile test debug native success 2s MaxRSS:164M +| +- options (reused) ++- run test 3 pass (3 total) 54ms MaxRSS:6M +| +- compile test debug native success 1s MaxRSS:156M +| +- options (reused) ++- run test 1 pass (1 total) 11ms MaxRSS:6M +| +- compile test debug native success 1s MaxRSS:144M ++- run test 238 pass (238 total) 5s MaxRSS:15M +| +- compile test fast native success 2m MaxRSS:1G ++- test-kv-pool success +| +- run test 10 pass (10 total) 34ms MaxRSS:6M +| | +- compile test debug native success 2s MaxRSS:145M +| +- run test 6 pass (6 total) 2s MaxRSS:9M +| +- compile test debug native success 3s MaxRSS:176M ++- run test 51 pass (51 total) 104ms MaxRSS:16M +| +- compile test debug native success 1s MaxRSS:165M +| +- options success +| | +- compile lib hip-mock-success debug native success 388ms MaxRSS:62M +| | +- compile lib hip-mock-failed debug native success 338ms MaxRSS:62M +| | +- compile lib hip-mock-missing debug native success 284ms MaxRSS:62M +| +- WriteFile kernels.zig cached ++- run test 23 pass, 1 skip (24 total) 202ms MaxRSS:8M +| +- compile test debug native success 2s MaxRSS:169M ++- run test 49 pass (49 total) 109ms MaxRSS:13M +| +- compile test debug native success 997ms MaxRSS:186M +| +- WriteFile kernels.zig success ++- run test 35 pass, 1 skip (36 total) 168ms MaxRSS:9M +| +- compile test debug native success 8s MaxRSS:222M +| +- WriteFile kernels.zig (reused) ++- run test 19 pass (19 total) 2s MaxRSS:12M +| +- compile test debug native success 10s MaxRSS:248M +| +- WriteFile kernels.zig (reused) ++- run test 4 pass (4 total) 74ms MaxRSS:6M +| +- compile test debug native success 1s MaxRSS:158M +| +- WriteFile kernels.zig (reused) ++- run test 4 pass (4 total) 43ms MaxRSS:6M +| +- compile test debug native success 1s MaxRSS:167M +| +- WriteFile kernels.zig (reused) ++- compile exe tf-qwen35-test debug native success 10s MaxRSS:168M +| +- WriteFile kernels.zig (reused) ++- compile exe tensorfold-native safe native success 18s MaxRSS:259M +| +- options (reused) +| +- WriteFile exlr8_alias.zig success +| +- WriteFile kernels.zig (reused) ++- run test 31 pass, 5 skip (36 total) 28ms MaxRSS:16M +| +- compile test safe native success 39s MaxRSS:585M ++- run test 4 pass (4 total) 20ms MaxRSS:6M +| +- compile test debug native success 1s MaxRSS:145M ++- run test 121 pass (121 total) 11s MaxRSS:42M +| +- compile test debug native success 1s MaxRSS:170M ++- run test 22 pass (22 total) 261ms MaxRSS:16M +| +- compile test debug native success 1s MaxRSS:171M ++- run exe learned-warning-test success 60ms + +- compile exe learned-warning-test debug native success 871ms MaxRSS:152M diff --git a/docs/receipts/glm-kv-pages-0b094b74/commit.txt b/docs/receipts/glm-kv-pages-0b094b74/commit.txt new file mode 100644 index 0000000000..3b35e941d5 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/commit.txt @@ -0,0 +1,20 @@ +tested implementation: 0b094b74546d6ca06f84a729ea8c9e7bfbdb80b9 (commit 11) +implementation tree: aa6b53d45e0a57b0a1808ff33ca211dd6c1b555e +base: main after v1.1.2 f8d254c5e0f9ab91eb325a2e7c28d2e5cb07c94d +the kernel contract: on 3b41fd7c7e4a7dfe9a92b876c35f1a05a57a6e9c (commit 10); commit 11 changes no kernel +the no-GPU CI and the mutations: on commit 11 +after it: cbe29802b0133aefb3dfe9f59671f5b450ff910a (commit 12) puts a comment on one line, and the receipt's commit adds this folder +TP=3's ranks: abc83c3d2dacfb33ef8473ebec6f6c2c82c991d7, b7fecb3d with the two commits in tp3/ring.patch and commit 11's change + +2671de386d3d67788e92d7b851d98b81c9821631 kv pool: a family without snapshot slots keeps its whole prompt cache as copied rows +2ee34d6bd559c3c53ba63a5f4cf15c861afcc56c glm cuda: the latent and pooled-key kernels take a stream's page table +ff3078188342cb3bcc804b6821dcd8bae9a2d65d glm cuda: a sequence's latent rows and pooled keys can live on a pool's pages +804c4da39a629901c8299463b6647559541f40ec glm cuda: streams past the engine's own take their KV pages from the core pool +8e279c850e947217f903906cbd7e384067fb5630 glm cuda tests: a paged stream against its whole-window twin, byte for byte +ae8a5a622dd1a8d61959c863fe2d74ce5f0fe5b3 glm cuda: a follower makes rank 0's pool through Pages.follow, under a host test +b7fecb3dd726b0c22572b4e0f2b86ec8a499b5f0 glm cuda: on pages too the chunk kernels read keys in main's loop: a dense chunk's page once a block, a sparse key's row from the token list +786fabe8950cf37bdb1416f303ecef8188feefb0 glm cuda tests: the page kernels under the kernel contract, on a page table out of order +c38f6d0b1e9f3beb588763f9df5627d78c12362b glm cuda: gate 5's records for the six KV page launches, from nvcc 13.0 and 13.3 on every SM +3b41fd7c7e4a7dfe9a92b876c35f1a05a57a6e9c glm cuda tests: a kept prefix's piece inside a page's span, and its size while a paged stream is current +0b094b74546d6ca06f84a729ea8c9e7bfbdb80b9 glm cuda: a stream takes its pages before its sequence is bound, so a slot past the paged sequences waits +cbe29802b0133aefb3dfe9f59671f5b450ff910a glm cuda: Lanes.seat's comment on one line, as the lean check asks diff --git a/docs/receipts/glm-kv-pages-0b094b74/gate5/build-nvcc13.0.log b/docs/receipts/glm-kv-pages-0b094b74/gate5/build-nvcc13.0.log new file mode 100644 index 0000000000..3a24737407 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/gate5/build-nvcc13.0.log @@ -0,0 +1,518 @@ +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/flashnext_fact.fatbin + +- run /usr/local/cuda-13.0/bin/nvcc (flashnext_fact.fatbin) w +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/fp8_experts.fatbin + +- run /usr/local/cuda-13.0/bin/nvcc (fp8_experts.fatbin) w +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/drama3_fp8.fatbin + +- run /usr/local/cuda-13.0/bin/nvcc (drama3_fp8.fatbin) w +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/flashnext_attn.fatbin + +- run /usr/local/cuda-13.0/bin/nvcc (flashnext_attn.fatbin) w +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/prefill_attention.fatbin + +- run /usr/local/cuda-13.0/bin/nvcc (prefill_attention.fatbin) w +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/glm_mm.fatbin + +- run /usr/local/cuda-13.0/bin/nvcc (glm_mm.fatbin) w +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ diff --git a/docs/receipts/glm-kv-pages-0b094b74/gate5/build-nvcc13.3.log b/docs/receipts/glm-kv-pages-0b094b74/gate5/build-nvcc13.3.log new file mode 100644 index 0000000000..8ec4e88dcb --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/gate5/build-nvcc13.3.log @@ -0,0 +1,518 @@ +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/fp8_experts.fatbin + +- run /usr/local/cuda/bin/nvcc (fp8_experts.fatbin) w +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/prefill_attention.fatbin + +- run /usr/local/cuda/bin/nvcc (prefill_attention.fatbin) w +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/drama3_fp8.fatbin + +- run /usr/local/cuda/bin/nvcc (drama3_fp8.fatbin) w +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/flashnext_fact.fatbin + +- run /usr/local/cuda/bin/nvcc (flashnext_fact.fatbin) w +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/glm_mm.fatbin + +- run /usr/local/cuda/bin/nvcc (glm_mm.fatbin) w +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + + +kernel-contract-fatbins ++- fatbins + +- install generated to fatbin/flashnext_attn.fatbin + +- run /usr/local/cuda/bin/nvcc (flashnext_attn.fatbin) w +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " diff --git a/docs/receipts/glm-kv-pages-0b094b74/gate5/limits-nvcc13.0.log b/docs/receipts/glm-kv-pages-0b094b74/gate5/limits-nvcc13.0.log new file mode 100644 index 0000000000..f72b94f3bc --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/gate5/limits-nvcc13.0.log @@ -0,0 +1,6982 @@ +INFO limits fatbins: images built by nvcc 13.0, checked against nvcc 13.0's records +PASS limits sm_80 flashnext_gdn flashnext_gdn_chain: 51 registers, 2060 static shared bytes, bound 1024 (record current) +PASS limits sm_86 flashnext_gdn flashnext_gdn_chain: 56 registers, 2060 static shared bytes, bound 1024 (record current) +PASS limits sm_89 flashnext_gdn flashnext_gdn_chain: 56 registers, 2060 static shared bytes, bound 1024 (record current) +PASS limits sm_120 flashnext_gdn flashnext_gdn_chain: 62 registers, 2060 static shared bytes, bound 1024 (record current) +PASS limits sm_121 flashnext_gdn flashnext_gdn_chain: 62 registers, 2060 static shared bytes, bound 1024 (record current) +RECORD flashnext_gdn flashnext_gdn_chain: .usage = .{ .{ .regs = 51, .static = 2060, .bound = 1024 }, .{ .regs = 56, .static = 2060, .bound = 1024 }, .{ .regs = 56, .static = 2060, .bound = 1024 }, .{ .regs = 62, .static = 2060, .bound = 1024 }, .{ .regs = 62, .static = 2060, .bound = 1024 }, } +PASS limits sm_80 flashnext_hc fn_add_streams: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_add_streams: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_add_streams: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_add_streams: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_add_streams: 35 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_add_streams: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_attn_chunks: .usage = .{ .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, } +PASS limits sm_80 flashnext_attn fn_attn_chunks_ahead: 70 registers, 768 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_attn_chunks_ahead: 70 registers, 768 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_attn_chunks_ahead: 70 registers, 768 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_attn_chunks_ahead: 73 registers, 768 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_attn_chunks_ahead: 73 registers, 768 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_attn_chunks_ahead: .usage = .{ .{ .regs = 70, .static = 768, .bound = 128 }, .{ .regs = 70, .static = 768, .bound = 128 }, .{ .regs = 70, .static = 768, .bound = 128 }, .{ .regs = 73, .static = 768, .bound = 128 }, .{ .regs = 73, .static = 768, .bound = 128 }, } +PASS limits sm_80 flashnext_attn fn_attn_gate: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_attn_gate: 23 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_attn_gate: 23 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_attn_gate: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn fn_attn_gate: 22 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_attn fn_attn_gate: .usage = .{ .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 23, .static = 0, .bound = 256 }, .{ .regs = 23, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_attn fn_attn_merge: 26 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_attn_merge: 28 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_attn_merge: 28 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_attn_merge: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn fn_attn_merge: 26 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_attn fn_attn_merge: .usage = .{ .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 28, .static = 0, .bound = 256 }, .{ .regs = 28, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 26, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_attn fn_attn_prep: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_attn_prep: 34 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_attn_prep: 34 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_attn_prep: 30 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn fn_attn_prep: 30 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_attn fn_attn_prep: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 34, .static = 128, .bound = 256 }, .{ .regs = 34, .static = 128, .bound = 256 }, .{ .regs = 30, .static = 128, .bound = 256 }, .{ .regs = 30, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_gdn_front: 26 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_front: 26 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_front: 26 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_front: 32 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_front: 32 registers, 1024 static shared bytes, bound 256 (record current) +RECORD flashnext_batch fn_gdn_front: .usage = .{ .{ .regs = 26, .static = 1024, .bound = 256 }, .{ .regs = 26, .static = 1024, .bound = 256 }, .{ .regs = 26, .static = 1024, .bound = 256 }, .{ .regs = 32, .static = 1024, .bound = 256 }, .{ .regs = 32, .static = 1024, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_gdn_norm: 24 registers, 528 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_norm: 26 registers, 528 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_norm: 26 registers, 528 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_norm: 24 registers, 528 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_norm: 24 registers, 528 static shared bytes, bound 128 (record current) +RECORD flashnext_batch fn_gdn_norm: .usage = .{ .{ .regs = 24, .static = 528, .bound = 128 }, .{ .regs = 26, .static = 528, .bound = 128 }, .{ .regs = 26, .static = 528, .bound = 128 }, .{ .regs = 24, .static = 528, .bound = 128 }, .{ .regs = 24, .static = 528, .bound = 128 }, } +PASS limits sm_80 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD flashnext_batch fn_gdn_rows: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD flashnext_batch fn_gdn_rows_to: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 flashnext_hc fn_hc_dense_mix: 22 registers, 128 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_hc_dense_mix: 36 registers, 128 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_hc_dense_mix: 36 registers, 128 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_hc_dense_mix: 36 registers, 128 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_hc_dense_mix: 36 registers, 128 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_hc_dense_mix: .usage = .{ .{ .regs = 22, .static = 128, .bound = 0 }, .{ .regs = 36, .static = 128, .bound = 0 }, .{ .regs = 36, .static = 128, .bound = 0 }, .{ .regs = 36, .static = 128, .bound = 0 }, .{ .regs = 36, .static = 128, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_hc_down: 56 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_hc fn_hc_down: 54 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_hc fn_hc_down: 54 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_hc fn_hc_down: 60 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_hc fn_hc_down: 60 registers, 64 static shared bytes, bound 128 (record current) +RECORD flashnext_hc fn_hc_down: .usage = .{ .{ .regs = 56, .static = 64, .bound = 128 }, .{ .regs = 54, .static = 64, .bound = 128 }, .{ .regs = 54, .static = 64, .bound = 128 }, .{ .regs = 60, .static = 64, .bound = 128 }, .{ .regs = 60, .static = 64, .bound = 128 }, } +PASS limits sm_80 flashnext_hc fn_hc_normalize: 32 registers, 16 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_hc_normalize: 40 registers, 16 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_hc_normalize: 40 registers, 16 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_hc_normalize: 40 registers, 16 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_hc_normalize: 38 registers, 16 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_hc_normalize: .usage = .{ .{ .regs = 32, .static = 16, .bound = 0 }, .{ .regs = 40, .static = 16, .bound = 0 }, .{ .regs = 40, .static = 16, .bound = 0 }, .{ .regs = 40, .static = 16, .bound = 0 }, .{ .regs = 38, .static = 16, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_hc_reduce_act: 32 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_86 flashnext_hc fn_hc_reduce_act: 63 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_89 flashnext_hc fn_hc_reduce_act: 63 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_120 flashnext_hc fn_hc_reduce_act: 58 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_121 flashnext_hc fn_hc_reduce_act: 58 registers, 4096 static shared bytes, bound 1024 (record current) +RECORD flashnext_hc fn_hc_reduce_act: .usage = .{ .{ .regs = 32, .static = 4096, .bound = 1024 }, .{ .regs = 63, .static = 4096, .bound = 1024 }, .{ .regs = 63, .static = 4096, .bound = 1024 }, .{ .regs = 58, .static = 4096, .bound = 1024 }, .{ .regs = 58, .static = 4096, .bound = 1024 }, } +PASS limits sm_80 flashnext_hc fn_hc_upmix: 69 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_upmix: 64 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_upmix: 64 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_upmix: 78 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_upmix: 78 registers, 2048 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_upmix: .usage = .{ .{ .regs = 69, .static = 2048, .bound = 256 }, .{ .regs = 64, .static = 2048, .bound = 256 }, .{ .regs = 64, .static = 2048, .bound = 256 }, .{ .regs = 78, .static = 2048, .bound = 256 }, .{ .regs = 78, .static = 2048, .bound = 256 }, } +PASS limits sm_80 flashnext_hc fn_hc_writeback0: 30 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_writeback0: 30 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_writeback0: 30 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_writeback0: 33 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_writeback0: 37 registers, 256 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_writeback0: .usage = .{ .{ .regs = 30, .static = 256, .bound = 256 }, .{ .regs = 30, .static = 256, .bound = 256 }, .{ .regs = 30, .static = 256, .bound = 256 }, .{ .regs = 33, .static = 256, .bound = 256 }, .{ .regs = 37, .static = 256, .bound = 256 }, } +PASS limits sm_80 flashnext_hc fn_hc_writeback_moe: 32 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_writeback_moe: 40 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_writeback_moe: 40 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_writeback_moe: 40 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_writeback_moe: 40 registers, 256 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_writeback_moe: .usage = .{ .{ .regs = 32, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, } +PASS limits sm_80 flashnext_hc fn_hc_writeback_slices: 32 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_writeback_slices: 38 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_writeback_slices: 38 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_writeback_slices: 40 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_writeback_slices: 40 registers, 256 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_writeback_slices: .usage = .{ .{ .regs = 32, .static = 256, .bound = 256 }, .{ .regs = 38, .static = 256, .bound = 256 }, .{ .regs = 38, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, } +PASS limits sm_80 flashnext_hc fn_ple_conv: 23 registers, 0 static shared bytes, bound 512 (record current) +PASS limits sm_86 flashnext_hc fn_ple_conv: 35 registers, 0 static shared bytes, bound 512 (record current) +PASS limits sm_89 flashnext_hc fn_ple_conv: 35 registers, 0 static shared bytes, bound 512 (record current) +PASS limits sm_120 flashnext_hc fn_ple_conv: 35 registers, 0 static shared bytes, bound 512 (record current) +PASS limits sm_121 flashnext_hc fn_ple_conv: 35 registers, 0 static shared bytes, bound 512 (record current) +RECORD flashnext_hc fn_ple_conv: .usage = .{ .{ .regs = 23, .static = 0, .bound = 512 }, .{ .regs = 35, .static = 0, .bound = 512 }, .{ .regs = 35, .static = 0, .bound = 512 }, .{ .regs = 35, .static = 0, .bound = 512 }, .{ .regs = 35, .static = 0, .bound = 512 }, } +PASS limits sm_80 flashnext_hc fn_ple_gate: 32 registers, 128 static shared bytes, bound 512 (record current) +PASS limits sm_86 flashnext_hc fn_ple_gate: 34 registers, 128 static shared bytes, bound 512 (record current) +PASS limits sm_89 flashnext_hc fn_ple_gate: 34 registers, 128 static shared bytes, bound 512 (record current) +PASS limits sm_120 flashnext_hc fn_ple_gate: 35 registers, 128 static shared bytes, bound 512 (record current) +PASS limits sm_121 flashnext_hc fn_ple_gate: 35 registers, 128 static shared bytes, bound 512 (record current) +RECORD flashnext_hc fn_ple_gate: .usage = .{ .{ .regs = 32, .static = 128, .bound = 512 }, .{ .regs = 34, .static = 128, .bound = 512 }, .{ .regs = 34, .static = 128, .bound = 512 }, .{ .regs = 35, .static = 128, .bound = 512 }, .{ .regs = 35, .static = 128, .bound = 512 }, } +PASS limits sm_80 flashnext_attn fn_qsa_pool: 32 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_qsa_pool: 40 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_qsa_pool: 40 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_qsa_pool: 40 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_qsa_pool: 39 registers, 1152 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_qsa_pool: .usage = .{ .{ .regs = 32, .static = 1152, .bound = 128 }, .{ .regs = 40, .static = 1152, .bound = 128 }, .{ .regs = 40, .static = 1152, .bound = 128 }, .{ .regs = 40, .static = 1152, .bound = 128 }, .{ .regs = 39, .static = 1152, .bound = 128 }, } +PASS limits sm_80 flashnext_attn fn_qsa_scores: 40 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_qsa_scores: 46 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_qsa_scores: 46 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_qsa_scores: 48 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn fn_qsa_scores: 48 registers, 4096 static shared bytes, bound 256 (record current) +RECORD flashnext_attn fn_qsa_scores: .usage = .{ .{ .regs = 40, .static = 4096, .bound = 256 }, .{ .regs = 46, .static = 4096, .bound = 256 }, .{ .regs = 46, .static = 4096, .bound = 256 }, .{ .regs = 48, .static = 4096, .bound = 256 }, .{ .regs = 48, .static = 4096, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_qsa_select: 32 registers, 1168 static shared bytes, bound 1024 (record current) +PASS limits sm_86 flashnext_batch fn_qsa_select: 50 registers, 1168 static shared bytes, bound 1024 (record current) +PASS limits sm_89 flashnext_batch fn_qsa_select: 50 registers, 1168 static shared bytes, bound 1024 (record current) +PASS limits sm_120 flashnext_batch fn_qsa_select: 44 registers, 1168 static shared bytes, bound 1024 (record current) +PASS limits sm_121 flashnext_batch fn_qsa_select: 40 registers, 1168 static shared bytes, bound 1024 (record current) +RECORD flashnext_batch fn_qsa_select: .usage = .{ .{ .regs = 32, .static = 1168, .bound = 1024 }, .{ .regs = 50, .static = 1168, .bound = 1024 }, .{ .regs = 50, .static = 1168, .bound = 1024 }, .{ .regs = 44, .static = 1168, .bound = 1024 }, .{ .regs = 40, .static = 1168, .bound = 1024 }, } +PASS limits sm_80 flashnext_mtp fn_row_best: 17 registers, 392 static shared bytes, bound 1024 (record current) +PASS limits sm_86 flashnext_mtp fn_row_best: 17 registers, 392 static shared bytes, bound 1024 (record current) +PASS limits sm_89 flashnext_mtp fn_row_best: 17 registers, 392 static shared bytes, bound 1024 (record current) +PASS limits sm_120 flashnext_mtp fn_row_best: 21 registers, 392 static shared bytes, bound 1024 (record current) +PASS limits sm_121 flashnext_mtp fn_row_best: 21 registers, 392 static shared bytes, bound 1024 (record current) +RECORD flashnext_mtp fn_row_best: .usage = .{ .{ .regs = 17, .static = 392, .bound = 1024 }, .{ .regs = 17, .static = 392, .bound = 1024 }, .{ .regs = 17, .static = 392, .bound = 1024 }, .{ .regs = 21, .static = 392, .bound = 1024 }, .{ .regs = 21, .static = 392, .bound = 1024 }, } +PASS limits sm_80 flashnext_hc fn_shared_scatter: 12 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_shared_scatter: 12 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_shared_scatter: 12 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_shared_scatter: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_shared_scatter: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_shared_scatter: .usage = .{ .{ .regs = 12, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_shared_swiglu: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_shared_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_shared_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_shared_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_shared_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_shared_swiglu: .usage = .{ .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_batch fn_shift_window: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_batch fn_shift_window: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_batch fn_shift_window: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_batch fn_shift_window: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_batch fn_shift_window: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_batch fn_shift_window: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_batch fn_sum_slices: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_batch fn_sum_slices: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_batch fn_sum_slices: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_batch fn_sum_slices: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_batch fn_sum_slices: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_batch fn_sum_slices: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 embed_rows tf_embed_rows_bf16_copies: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 embed_rows tf_embed_rows_bf16_copies: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 embed_rows tf_embed_rows_bf16_copies: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 embed_rows tf_embed_rows_bf16_copies: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 embed_rows tf_embed_rows_bf16_copies: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD embed_rows tf_embed_rows_bf16_copies: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 norms tf_rms_rows_sums: 31 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_86 norms tf_rms_rows_sums: 37 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_89 norms tf_rms_rows_sums: 37 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_120 norms tf_rms_rows_sums: 34 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_121 norms tf_rms_rows_sums: 34 registers, 132 static shared bytes, bound 1024 (record current) +RECORD norms tf_rms_rows_sums: .usage = .{ .{ .regs = 31, .static = 132, .bound = 1024 }, .{ .regs = 37, .static = 132, .bound = 1024 }, .{ .regs = 37, .static = 132, .bound = 1024 }, .{ .regs = 34, .static = 132, .bound = 1024 }, .{ .regs = 34, .static = 132, .bound = 1024 }, } +PASS limits sm_80 route_rows tf_route_logits: 32 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 route_rows tf_route_logits: 39 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 route_rows tf_route_logits: 39 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 route_rows tf_route_logits: 35 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 route_rows tf_route_logits: 34 registers, 0 static shared bytes, bound 128 (record current) +RECORD route_rows tf_route_logits: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 39, .static = 0, .bound = 128 }, .{ .regs = 39, .static = 0, .bound = 128 }, .{ .regs = 35, .static = 0, .bound = 128 }, .{ .regs = 34, .static = 0, .bound = 128 }, } +PASS limits sm_80 route_rows tf_route_logits_few: 31 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 route_rows tf_route_logits_few: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 route_rows tf_route_logits_few: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 route_rows tf_route_logits_few: 35 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 route_rows tf_route_logits_few: 37 registers, 0 static shared bytes, bound 128 (record current) +RECORD route_rows tf_route_logits_few: .usage = .{ .{ .regs = 31, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 35, .static = 0, .bound = 128 }, .{ .regs = 37, .static = 0, .bound = 128 }, } +PASS limits sm_80 route_rows tf_route_topk: 39 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_86 route_rows tf_route_topk: 39 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_89 route_rows tf_route_topk: 39 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_120 route_rows tf_route_topk: 36 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_121 route_rows tf_route_topk: 36 registers, 64 static shared bytes, bound 128 (record current) +RECORD route_rows tf_route_topk: .usage = .{ .{ .regs = 39, .static = 64, .bound = 128 }, .{ .regs = 39, .static = 64, .bound = 128 }, .{ .regs = 39, .static = 64, .bound = 128 }, .{ .regs = 36, .static = 64, .bound = 128 }, .{ .regs = 36, .static = 64, .bound = 128 }, } +PASS limits sm_80 norms tf_rms_rows_bf16: 79 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_86 norms tf_rms_rows_bf16: 79 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_89 norms tf_rms_rows_bf16: 79 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_120 norms tf_rms_rows_bf16: 80 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_121 norms tf_rms_rows_bf16: 80 registers, 32 static shared bytes, bound 256 (record current) +RECORD norms tf_rms_rows_bf16: .usage = .{ .{ .regs = 79, .static = 32, .bound = 256 }, .{ .regs = 79, .static = 32, .bound = 256 }, .{ .regs = 79, .static = 32, .bound = 256 }, .{ .regs = 80, .static = 32, .bound = 256 }, .{ .regs = 80, .static = 32, .bound = 256 }, } +PASS limits sm_80 norms tf_rms_rows: 32 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_86 norms tf_rms_rows: 46 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_89 norms tf_rms_rows: 46 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_120 norms tf_rms_rows: 45 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_121 norms tf_rms_rows: 45 registers, 132 static shared bytes, bound 1024 (record current) +RECORD norms tf_rms_rows: .usage = .{ .{ .regs = 32, .static = 132, .bound = 1024 }, .{ .regs = 46, .static = 132, .bound = 1024 }, .{ .regs = 46, .static = 132, .bound = 1024 }, .{ .regs = 45, .static = 132, .bound = 1024 }, .{ .regs = 45, .static = 132, .bound = 1024 }, } +PASS limits sm_80 router tf_nemo_router: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 router tf_nemo_router: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 router tf_nemo_router: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 router tf_nemo_router: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 router tf_nemo_router: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD router tf_nemo_router: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 router tf_nemo_topk: 94 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 router tf_nemo_topk: 95 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 router tf_nemo_topk: 95 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 router tf_nemo_topk: 91 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 router tf_nemo_topk: 91 registers, 0 static shared bytes, bound 32 (record current) +RECORD router tf_nemo_topk: .usage = .{ .{ .regs = 94, .static = 0, .bound = 32 }, .{ .regs = 95, .static = 0, .bound = 32 }, .{ .regs = 95, .static = 0, .bound = 32 }, .{ .regs = 91, .static = 0, .bound = 32 }, .{ .regs = 91, .static = 0, .bound = 32 }, } +PASS limits sm_80 router tf_nemo_topk_wide: 168 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 router tf_nemo_topk_wide: 168 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 router tf_nemo_topk_wide: 168 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 router tf_nemo_topk_wide: 168 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 router tf_nemo_topk_wide: 168 registers, 0 static shared bytes, bound 32 (record current) +RECORD router tf_nemo_topk_wide: .usage = .{ .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, } +PASS limits sm_80 latent tf_super_topk_few: 32 registers, 4352 static shared bytes, bound 1024 (record current) +PASS limits sm_86 latent tf_super_topk_few: 38 registers, 4352 static shared bytes, bound 1024 (record current) +PASS limits sm_89 latent tf_super_topk_few: 38 registers, 4352 static shared bytes, bound 1024 (record current) +PASS limits sm_120 latent tf_super_topk_few: 30 registers, 4352 static shared bytes, bound 1024 (record current) +PASS limits sm_121 latent tf_super_topk_few: 30 registers, 4352 static shared bytes, bound 1024 (record current) +RECORD latent tf_super_topk_few: .usage = .{ .{ .regs = 32, .static = 4352, .bound = 1024 }, .{ .regs = 38, .static = 4352, .bound = 1024 }, .{ .regs = 38, .static = 4352, .bound = 1024 }, .{ .regs = 30, .static = 4352, .bound = 1024 }, .{ .regs = 30, .static = 4352, .bound = 1024 }, } +N/A limits qmm_group cluster of 8 sm_80: below_minimum +N/A limits qmm_group cluster of 8 sm_86: below_minimum +N/A limits qmm_group cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmm_group cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmm_group cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmm_group cluster of 8: .usage = .{ .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmm_group one CTA a tile: 54 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmm_group one CTA a tile: 54 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmm_group one CTA a tile: 54 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmm_group one CTA a tile: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmm_group one CTA a tile: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmm_group one CTA a tile: .usage = .{ .{ .regs = 54, .static = 0, .bound = 128 }, .{ .regs = 54, .static = 0, .bound = 128 }, .{ .regs = 54, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 lane_gemv: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 lane_gemv: 57 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 lane_gemv: 57 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 lane_gemv: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 lane_gemv: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD lane_gemv: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 lane_gemv split-K: 62 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_86 lane_gemv split-K: 56 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_89 lane_gemv split-K: 56 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_120 lane_gemv split-K: 44 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_121 lane_gemv split-K: 44 registers, 128 static shared bytes, bound 128 (record current) +RECORD lane_gemv split-K: .usage = .{ .{ .regs = 62, .static = 128, .bound = 128 }, .{ .regs = 56, .static = 128, .bound = 128 }, .{ .regs = 56, .static = 128, .bound = 128 }, .{ .regs = 44, .static = 128, .bound = 128 }, .{ .regs = 44, .static = 128, .bound = 128 }, } +PASS limits sm_80 prompt16 FP4: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 prompt16 FP4: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 prompt16 FP4: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 prompt16 FP4: 212 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 prompt16 FP4: 212 registers, 0 static shared bytes, bound 128 (record current) +RECORD prompt16 FP4: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, } +PASS limits sm_80 prompt16 FP8: 243 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 prompt16 FP8: 243 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 prompt16 FP8: 243 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 prompt16 FP8: 218 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 prompt16 FP8: 218 registers, 0 static shared bytes, bound 128 (record current) +RECORD prompt16 FP8: .usage = .{ .{ .regs = 243, .static = 0, .bound = 128 }, .{ .regs = 243, .static = 0, .bound = 128 }, .{ .regs = 243, .static = 0, .bound = 128 }, .{ .regs = 218, .static = 0, .bound = 128 }, .{ .regs = 218, .static = 0, .bound = 128 }, } +PASS limits sm_80 prompt16 FP4 fp32 out: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 prompt16 FP4 fp32 out: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 prompt16 FP4 fp32 out: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 prompt16 FP4 fp32 out: 212 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 prompt16 FP4 fp32 out: 212 registers, 0 static shared bytes, bound 128 (record current) +RECORD prompt16 FP4 fp32 out: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP4 fp32 out 16-row tile: 88 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP4 fp32 out 16-row tile: 88 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP4 fp32 out 16-row tile: 96 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP4 fp32 out 16-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 16-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 16-row tile: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, .{ .regs = 88, .static = 0, .bound = 128 }, .{ .regs = 96, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +N/A limits qmmf FP4 fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP4 fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP4 fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP4 fp32 out 16-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 16-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP4 fp32 out 32-row tile: 106 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP4 fp32 out 32-row tile: 106 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP4 fp32 out 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP4 fp32 out 32-row tile: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 32-row tile: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 32-row tile: .usage = .{ .{ .regs = 106, .static = 0, .bound = 128 }, .{ .regs = 106, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +N/A limits qmmf FP4 fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP4 fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP4 fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP4 fp32 out 32-row tile, cluster of 8: 63 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 32-row tile, cluster of 8: 63 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 63, .static = 0, .bound = 128 }, .{ .regs = 63, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP4 fp32 out 64-row tile: 148 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP4 fp32 out 64-row tile: 148 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP4 fp32 out 64-row tile: 180 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP4 fp32 out 64-row tile: 78 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 64-row tile: 78 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 64-row tile: .usage = .{ .{ .regs = 148, .static = 0, .bound = 128 }, .{ .regs = 148, .static = 0, .bound = 128 }, .{ .regs = 180, .static = 0, .bound = 128 }, .{ .regs = 78, .static = 0, .bound = 128 }, .{ .regs = 78, .static = 0, .bound = 128 }, } +N/A limits qmmf FP4 fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP4 fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP4 fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP4 fp32 out 64-row tile, cluster of 8: 78 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 64-row tile, cluster of 8: 78 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 78, .static = 0, .bound = 128 }, .{ .regs = 78, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP4 fp32 out fused slices: 182 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP4 fp32 out fused slices: 182 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP4 fp32 out fused slices: 228 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP4 fp32 out fused slices: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out fused slices: 168 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out fused slices: .usage = .{ .{ .regs = 182, .static = 0, .bound = 128 }, .{ .regs = 182, .static = 0, .bound = 128 }, .{ .regs = 228, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 16-row tile: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 16-row tile: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 16-row tile: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 16-row tile: 60 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 16-row tile: 60 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 16-row tile: .usage = .{ .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 60, .static = 0, .bound = 128 }, .{ .regs = 60, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 16-row tile, cluster of 8: .usage = .{ .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 32-row tile: 57 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 32-row tile: 57 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 32-row tile: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 32-row tile, cluster of 8: 66 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 32-row tile, cluster of 8: 66 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 32-row tile, cluster of 8: .usage = .{ .{ .regs = 66, .static = 0, .bound = 128 }, .{ .regs = 66, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 64-row tile: .usage = .{ .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 64-row tile, cluster of 8: 103 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 64-row tile, cluster of 8: 103 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 64-row tile, cluster of 8: .usage = .{ .{ .regs = 103, .static = 0, .bound = 128 }, .{ .regs = 103, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 64-row tile, fused slices: 159 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 64-row tile, fused slices: 159 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 64-row tile, fused slices: .usage = .{ .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 159, .static = 0, .bound = 128 }, .{ .regs = 159, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf reduce_kernel: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qmmf reduce_kernel: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qmmf reduce_kernel: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qmmf reduce_kernel: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qmmf reduce_kernel: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD qmmf reduce_kernel: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 qmmf FP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8 fp32 out 16-row tile: 50 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 16-row tile: 50 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 16-row tile: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 50, .static = 0, .bound = 128 }, .{ .regs = 50, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8 fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8 fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8 fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8 fp32 out 16-row tile, cluster of 8: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 16-row tile, cluster of 8: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8 fp32 out 32-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8 fp32 out 32-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8 fp32 out 32-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8 fp32 out 32-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 32-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 32-row tile: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8 fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8 fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8 fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8 fp32 out 32-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 32-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8 fp32 out 64-row tile: 126 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8 fp32 out 64-row tile: 126 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8 fp32 out 64-row tile: 126 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8 fp32 out 64-row tile: 72 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 64-row tile: 72 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 64-row tile: .usage = .{ .{ .regs = 126, .static = 0, .bound = 128 }, .{ .regs = 126, .static = 0, .bound = 128 }, .{ .regs = 126, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8 fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8 fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8 fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8 fp32 out 64-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 64-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8 fp32 out 64-row tile, fused slices: 179 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8 fp32 out 64-row tile, fused slices: 179 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8 fp32 out 64-row tile, fused slices: 179 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8 fp32 out 64-row tile, fused slices: 141 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 64-row tile, fused slices: 141 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 64-row tile, fused slices: .usage = .{ .{ .regs = 179, .static = 0, .bound = 128 }, .{ .regs = 179, .static = 0, .bound = 128 }, .{ .regs = 179, .static = 0, .bound = 128 }, .{ .regs = 141, .static = 0, .bound = 128 }, .{ .regs = 141, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 fp32 out 16-row tile: 60 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 16-row tile: 60 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 16-row tile: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 60, .static = 0, .bound = 128 }, .{ .regs = 60, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 fp32 out 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 fp32 out 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 fp32 out 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 fp32 out 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 fp32 out 32-row tile: 57 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 32-row tile: 57 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 32-row tile: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 fp32 out 32-row tile, cluster of 8: 66 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 32-row tile, cluster of 8: 66 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 66, .static = 0, .bound = 128 }, .{ .regs = 66, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 fp32 out 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 fp32 out 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 fp32 out 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 fp32 out 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 64-row tile: .usage = .{ .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 fp32 out 64-row tile, cluster of 8: 103 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 64-row tile, cluster of 8: 103 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 103, .static = 0, .bound = 128 }, .{ .regs = 103, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 fp32 out 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 fp32 out 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 fp32 out 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 fp32 out 64-row tile, fused slices: 159 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 64-row tile, fused slices: 159 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 64-row tile, fused slices: .usage = .{ .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 159, .static = 0, .bound = 128 }, .{ .regs = 159, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8G fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8G fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8G fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8G fp32 out 16-row tile: 58 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 16-row tile: 58 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 16-row tile: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 58, .static = 0, .bound = 128 }, .{ .regs = 58, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8G fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8G fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8G fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8G fp32 out 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8G fp32 out 32-row tile: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8G fp32 out 32-row tile: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8G fp32 out 32-row tile: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8G fp32 out 32-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 32-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 32-row tile: .usage = .{ .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8G fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8G fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8G fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8G fp32 out 32-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 32-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8G fp32 out 64-row tile: 164 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8G fp32 out 64-row tile: 164 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8G fp32 out 64-row tile: 164 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8G fp32 out 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 64-row tile: .usage = .{ .{ .regs = 164, .static = 0, .bound = 128 }, .{ .regs = 164, .static = 0, .bound = 128 }, .{ .regs = 164, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8G fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8G fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8G fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8G fp32 out 64-row tile, cluster of 8: 122 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 64-row tile, cluster of 8: 122 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 122, .static = 0, .bound = 128 }, .{ .regs = 122, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8G fp32 out 64-row tile, fused slices: 196 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8G fp32 out 64-row tile, fused slices: 196 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8G fp32 out 64-row tile, fused slices: 196 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8G fp32 out 64-row tile, fused slices: 157 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 64-row tile, fused slices: 157 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 64-row tile, fused slices: .usage = .{ .{ .regs = 196, .static = 0, .bound = 128 }, .{ .regs = 196, .static = 0, .bound = 128 }, .{ .regs = 196, .static = 0, .bound = 128 }, .{ .regs = 157, .static = 0, .bound = 128 }, .{ .regs = 157, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf BF16 fp32 out 16-row tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf BF16 fp32 out 16-row tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf BF16 fp32 out 16-row tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf BF16 fp32 out 16-row tile: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 16-row tile: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 16-row tile: .usage = .{ .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +N/A limits qmmf BF16 fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf BF16 fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf BF16 fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf BF16 fp32 out 16-row tile, cluster of 8: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 16-row tile, cluster of 8: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf BF16 fp32 out 32-row tile: 93 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf BF16 fp32 out 32-row tile: 93 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf BF16 fp32 out 32-row tile: 93 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf BF16 fp32 out 32-row tile: 55 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 32-row tile: 55 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 32-row tile: .usage = .{ .{ .regs = 93, .static = 0, .bound = 128 }, .{ .regs = 93, .static = 0, .bound = 128 }, .{ .regs = 93, .static = 0, .bound = 128 }, .{ .regs = 55, .static = 0, .bound = 128 }, .{ .regs = 55, .static = 0, .bound = 128 }, } +N/A limits qmmf BF16 fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf BF16 fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf BF16 fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf BF16 fp32 out 32-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 32-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf BF16 fp32 out 64-row tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf BF16 fp32 out 64-row tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf BF16 fp32 out 64-row tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf BF16 fp32 out 64-row tile: 72 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 64-row tile: 72 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 64-row tile: .usage = .{ .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, } +N/A limits qmmf BF16 fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf BF16 fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf BF16 fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf BF16 fp32 out 64-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 64-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf BF16 fp32 out 64-row tile, fused slices: 180 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf BF16 fp32 out 64-row tile, fused slices: 180 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf BF16 fp32 out 64-row tile, fused slices: 180 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf BF16 fp32 out 64-row tile, fused slices: 143 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 64-row tile, fused slices: 143 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 64-row tile, fused slices: .usage = .{ .{ .regs = 180, .static = 0, .bound = 128 }, .{ .regs = 180, .static = 0, .bound = 128 }, .{ .regs = 180, .static = 0, .bound = 128 }, .{ .regs = 143, .static = 0, .bound = 128 }, .{ .regs = 143, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf reduce_kernel: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qmmf reduce_kernel: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qmmf reduce_kernel: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qmmf reduce_kernel: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qmmf reduce_kernel: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD qmmf reduce_kernel: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits ck tf_ck4_up_relu2_few sm_80: below_minimum +N/A limits ck tf_ck4_up_relu2_few sm_86: below_minimum +N/A limits ck tf_ck4_up_relu2_few sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_up_relu2_few: 125 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 ck tf_ck4_up_relu2_few: 125 registers, 0 static shared bytes, bound 128 (record current) +RECORD ck tf_ck4_up_relu2_few: .usage = .{ .{ .regs = 125, .static = 0, .bound = 128 }, .{ .regs = 125, .static = 0, .bound = 128 }, } +N/A limits ck tf_ck4_down_f32_few sm_80: below_minimum +N/A limits ck tf_ck4_down_f32_few sm_86: below_minimum +N/A limits ck tf_ck4_down_f32_few sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_down_f32_few: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 ck tf_ck4_down_f32_few: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD ck tf_ck4_down_f32_few: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, } +PASS limits sm_80 mamba tf_nemo_scan: 138 registers, 41088 static shared bytes, bound 128 (record current) +PASS limits sm_86 mamba tf_nemo_scan: 144 registers, 41088 static shared bytes, bound 128 (record current) +PASS limits sm_89 mamba tf_nemo_scan: 144 registers, 41088 static shared bytes, bound 128 (record current) +PASS limits sm_120 mamba tf_nemo_scan: 116 registers, 41088 static shared bytes, bound 128 (record current) +PASS limits sm_121 mamba tf_nemo_scan: 116 registers, 41088 static shared bytes, bound 128 (record current) +RECORD mamba tf_nemo_scan: .usage = .{ .{ .regs = 138, .static = 41088, .bound = 128 }, .{ .regs = 144, .static = 41088, .bound = 128 }, .{ .regs = 144, .static = 41088, .bound = 128 }, .{ .regs = 116, .static = 41088, .bound = 128 }, .{ .regs = 116, .static = 41088, .bound = 128 }, } +PASS limits sm_80 rank_sum tf_rank_sum: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 rank_sum tf_rank_sum: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 rank_sum tf_rank_sum: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 rank_sum tf_rank_sum: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 rank_sum tf_rank_sum: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD rank_sum tf_rank_sum: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit vision_kernel<80, 8> (RADIO): 143 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit vision_kernel<80, 8> (RADIO): 143 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit vision_kernel<80, 8> (RADIO): 143 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit vision_kernel<80, 8> (RADIO): 122 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit vision_kernel<80, 8> (RADIO): 122 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit vision_kernel<80, 8> (RADIO): .usage = .{ .{ .regs = 143, .static = 0, .bound = 256 }, .{ .regs = 143, .static = 0, .bound = 256 }, .{ .regs = 143, .static = 0, .bound = 256 }, .{ .regs = 122, .static = 0, .bound = 256 }, .{ .regs = 122, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit full_kernel<72> (SigLIP2): 158 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit full_kernel<72> (SigLIP2): 158 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit full_kernel<72> (SigLIP2): 158 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit full_kernel<72> (SigLIP2): 169 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit full_kernel<72> (SigLIP2): 169 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit full_kernel<72> (SigLIP2): .usage = .{ .{ .regs = 158, .static = 0, .bound = 256 }, .{ .regs = 158, .static = 0, .bound = 256 }, .{ .regs = 158, .static = 0, .bound = 256 }, .{ .regs = 169, .static = 0, .bound = 256 }, .{ .regs = 169, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit tf_vit_layer_norm: 38 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit tf_vit_layer_norm: 38 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit tf_vit_layer_norm: 38 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit tf_vit_layer_norm: 40 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit tf_vit_layer_norm: 40 registers, 32 static shared bytes, bound 256 (record current) +RECORD vit tf_vit_layer_norm: .usage = .{ .{ .regs = 38, .static = 32, .bound = 256 }, .{ .regs = 38, .static = 32, .bound = 256 }, .{ .regs = 38, .static = 32, .bound = 256 }, .{ .regs = 40, .static = 32, .bound = 256 }, .{ .regs = 40, .static = 32, .bound = 256 }, } +PASS limits sm_80 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit tf_vit_pointwise: .usage = .{ .{ .regs = 14, .static = 0, .bound = 256 }, .{ .regs = 14, .static = 0, .bound = 256 }, .{ .regs = 14, .static = 0, .bound = 256 }, .{ .regs = 14, .static = 0, .bound = 256 }, .{ .regs = 14, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit tf_vit_bias: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit tf_vit_unshuffle: 29 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit tf_vit_unshuffle: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit tf_vit_unshuffle: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit tf_vit_unshuffle: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit tf_vit_unshuffle: 34 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit tf_vit_unshuffle: .usage = .{ .{ .regs = 29, .static = 0, .bound = 256 }, .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_embed: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_embed: 33 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_embed: 33 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_embed: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_embed: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_embed: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_layernorm: 23 registers, 132 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_layernorm: 26 registers, 132 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_layernorm: 26 registers, 132 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_layernorm: 23 registers, 132 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_layernorm: 23 registers, 132 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_layernorm: .usage = .{ .{ .regs = 23, .static = 132, .bound = 256 }, .{ .regs = 26, .static = 132, .bound = 256 }, .{ .regs = 26, .static = 132, .bound = 256 }, .{ .regs = 23, .static = 132, .bound = 256 }, .{ .regs = 23, .static = 132, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_qkv_rope: 32 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qwen_vision tf_qv_qkv_rope: 36 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qwen_vision tf_qv_qkv_rope: 36 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qwen_vision tf_qv_qkv_rope: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qwen_vision tf_qv_qkv_rope: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD qwen_vision tf_qv_qkv_rope: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 36, .static = 0, .bound = 128 }, .{ .regs = 36, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen_vision tf_qv_attention72: 180 registers, 36864 static shared bytes, bound 64 (record current) +PASS limits sm_86 qwen_vision tf_qv_attention72: 180 registers, 36864 static shared bytes, bound 64 (record current) +PASS limits sm_89 qwen_vision tf_qv_attention72: 180 registers, 36864 static shared bytes, bound 64 (record current) +PASS limits sm_120 qwen_vision tf_qv_attention72: 186 registers, 36864 static shared bytes, bound 64 (record current) +PASS limits sm_121 qwen_vision tf_qv_attention72: 186 registers, 36864 static shared bytes, bound 64 (record current) +RECORD qwen_vision tf_qv_attention72: .usage = .{ .{ .regs = 180, .static = 36864, .bound = 64 }, .{ .regs = 180, .static = 36864, .bound = 64 }, .{ .regs = 180, .static = 36864, .bound = 64 }, .{ .regs = 186, .static = 36864, .bound = 64 }, .{ .regs = 186, .static = 36864, .bound = 64 }, } +PASS limits sm_80 qwen_vision tf_qv_attention64: 164 registers, 32768 static shared bytes, bound 64 (record current) +PASS limits sm_86 qwen_vision tf_qv_attention64: 164 registers, 32768 static shared bytes, bound 64 (record current) +PASS limits sm_89 qwen_vision tf_qv_attention64: 164 registers, 32768 static shared bytes, bound 64 (record current) +PASS limits sm_120 qwen_vision tf_qv_attention64: 167 registers, 32768 static shared bytes, bound 64 (record current) +PASS limits sm_121 qwen_vision tf_qv_attention64: 167 registers, 32768 static shared bytes, bound 64 (record current) +RECORD qwen_vision tf_qv_attention64: .usage = .{ .{ .regs = 164, .static = 32768, .bound = 64 }, .{ .regs = 164, .static = 32768, .bound = 64 }, .{ .regs = 164, .static = 32768, .bound = 64 }, .{ .regs = 167, .static = 32768, .bound = 64 }, .{ .regs = 167, .static = 32768, .bound = 64 }, } +PASS limits sm_80 qwen_vision tf_qv_bias_gelu: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_bias_gelu: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_bias_gelu: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_bias_gelu: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_bias_gelu: 20 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_bias_gelu: .usage = .{ .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_bias_residual: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_bias_residual: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_bias_residual: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_bias_residual: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_bias_residual: 20 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_bias_residual: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_bias_cast: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_bias_cast: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_bias_cast: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_bias_cast: 18 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_bias_cast: 18 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_bias_cast: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 18, .static = 0, .bound = 256 }, .{ .regs = 18, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_splice: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qwen_vision tf_qv_splice: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qwen_vision tf_qv_splice: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qwen_vision tf_qv_splice: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qwen_vision tf_qv_splice: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD qwen_vision tf_qv_splice: .usage = .{ .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_attn fn_attn_prep_media: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_attn_prep_media: 34 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_attn_prep_media: 34 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_attn_prep_media: 30 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn fn_attn_prep_media: 30 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_attn fn_attn_prep_media: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 34, .static = 128, .bound = 256 }, .{ .regs = 34, .static = 128, .bound = 256 }, .{ .regs = 30, .static = 128, .bound = 256 }, .{ .regs = 30, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_attn fn_qsa_pool_media: 32 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_qsa_pool_media: 40 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_qsa_pool_media: 40 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_qsa_pool_media: 39 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_qsa_pool_media: 39 registers, 1152 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_qsa_pool_media: .usage = .{ .{ .regs = 32, .static = 1152, .bound = 128 }, .{ .regs = 40, .static = 1152, .bound = 128 }, .{ .regs = 40, .static = 1152, .bound = 128 }, .{ .regs = 39, .static = 1152, .bound = 128 }, .{ .regs = 39, .static = 1152, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_rot_in: 19 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_rot_in: 22 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_rot_in: 22 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_rot_in: 22 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_rot_in: 22 registers, 0 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_rot_in: .usage = .{ .{ .regs = 19, .static = 0, .bound = 128 }, .{ .regs = 22, .static = 0, .bound = 128 }, .{ .regs = 22, .static = 0, .bound = 128 }, .{ .regs = 22, .static = 0, .bound = 128 }, .{ .regs = 22, .static = 0, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_2_4: 152 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_2_4: 152 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_2_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 152, .static = 16, .bound = 128 }, .{ .regs = 152, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_2_4: 152 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_2_4: 152 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_4_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 152, .static = 16, .bound = 128 }, .{ .regs = 152, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_2_4: 155 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_2_4: 155 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_6_2_4: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 155, .static = 16, .bound = 128 }, .{ .regs = 155, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_2_4: 154 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_2_4: 154 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_8_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 154, .static = 16, .bound = 128 }, .{ .regs = 154, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_10_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_12_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_2_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_2_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_14_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_2_4: 125 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_2_4: 125 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_16_2_4: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 125, .static = 16, .bound = 128 }, .{ .regs = 125, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_3_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_3_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_3_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_3_2_4: 153 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_3_2_4: 153 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_3_2_4: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 153, .static = 16, .bound = 128 }, .{ .regs = 153, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_5_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_5_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_5_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_5_2_4: 155 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_5_2_4: 155 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_5_2_4: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 155, .static = 16, .bound = 128 }, .{ .regs = 155, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_7_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_9_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_11_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_13_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_13_2_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_13_2_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_13_2_4: 126 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_13_2_4: 126 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_13_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 126, .static = 16, .bound = 128 }, .{ .regs = 126, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_1_4: 153 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_1_4: 153 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_2_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 153, .static = 16, .bound = 128 }, .{ .regs = 153, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_1_4: 153 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_1_4: 153 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_4_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 153, .static = 16, .bound = 128 }, .{ .regs = 153, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_1_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_1_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_1_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_1_4: 154 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_1_4: 154 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_6_1_4: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 154, .static = 16, .bound = 128 }, .{ .regs = 154, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_1_4: 150 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_1_4: 150 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_8_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 150, .static = 16, .bound = 128 }, .{ .regs = 150, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_10_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_12_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_1_4: 166 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_1_4: 166 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_14_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 166, .static = 16, .bound = 128 }, .{ .regs = 166, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_1_4: 123 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_1_4: 123 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_16_1_4: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 123, .static = 16, .bound = 128 }, .{ .regs = 123, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_0_4: 151 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_0_4: 151 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_2_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 151, .static = 16, .bound = 128 }, .{ .regs = 151, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_0_4: 151 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_0_4: 151 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_4_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 151, .static = 16, .bound = 128 }, .{ .regs = 151, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_0_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_0_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_0_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_0_4: 158 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_0_4: 158 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_6_0_4: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 158, .static = 16, .bound = 128 }, .{ .regs = 158, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_0_4: 159 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_0_4: 159 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_8_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 159, .static = 16, .bound = 128 }, .{ .regs = 159, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_10_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_12_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_0_4: 126 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_0_4: 126 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_14_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 126, .static = 16, .bound = 128 }, .{ .regs = 126, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_0_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_0_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_0_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_0_4: 126 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_0_4: 126 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_16_0_4: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 126, .static = 16, .bound = 128 }, .{ .regs = 126, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_2_8: 196 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_2_8: 196 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_2_8: 196 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_2_8: 126 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_2_8: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_2_2_8: .usage = .{ .{ .regs = 196, .static = 16, .bound = 256 }, .{ .regs = 196, .static = 16, .bound = 256 }, .{ .regs = 196, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_2_8: 196 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_2_8: 196 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_2_8: 196 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_2_8: 126 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_2_8: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_4_2_8: .usage = .{ .{ .regs = 196, .static = 16, .bound = 256 }, .{ .regs = 196, .static = 16, .bound = 256 }, .{ .regs = 196, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_2_8: 159 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_6_2_8: .usage = .{ .{ .regs = 159, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_2_8: 202 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_2_8: 202 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_2_8: 202 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_2_8: 126 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_2_8: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_8_2_8: .usage = .{ .{ .regs = 202, .static = 16, .bound = 256 }, .{ .regs = 202, .static = 16, .bound = 256 }, .{ .regs = 202, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_2_8: 178 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_2_8: 178 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_2_8: 178 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_2_8: 223 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_2_8: 223 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_10_2_8: .usage = .{ .{ .regs = 178, .static = 16, .bound = 256 }, .{ .regs = 178, .static = 16, .bound = 256 }, .{ .regs = 178, .static = 16, .bound = 256 }, .{ .regs = 223, .static = 16, .bound = 256 }, .{ .regs = 223, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_2_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_2_8: 222 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_12_2_8: .usage = .{ .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_2_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_2_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_2_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_2_8: 127 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_2_8: 127 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_14_2_8: .usage = .{ .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 127, .static = 16, .bound = 256 }, .{ .regs = 127, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_2_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_2_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_2_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_2_8: 124 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_2_8: 124 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_16_2_8: .usage = .{ .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 124, .static = 16, .bound = 256 }, .{ .regs = 124, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_3_2_8: 159 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_3_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_3_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_3_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_3_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_3_2_8: .usage = .{ .{ .regs = 159, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_5_2_8: 160 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_5_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_5_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_5_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_5_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_5_2_8: .usage = .{ .{ .regs = 160, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_7_2_8: 179 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_7_2_8: 179 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_7_2_8: 179 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_7_2_8: 232 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_7_2_8: 232 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_7_2_8: .usage = .{ .{ .regs = 179, .static = 16, .bound = 256 }, .{ .regs = 179, .static = 16, .bound = 256 }, .{ .regs = 179, .static = 16, .bound = 256 }, .{ .regs = 232, .static = 16, .bound = 256 }, .{ .regs = 232, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_9_2_8: 176 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_9_2_8: 177 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_9_2_8: 177 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_9_2_8: 220 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_9_2_8: 220 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_9_2_8: .usage = .{ .{ .regs = 176, .static = 16, .bound = 256 }, .{ .regs = 177, .static = 16, .bound = 256 }, .{ .regs = 177, .static = 16, .bound = 256 }, .{ .regs = 220, .static = 16, .bound = 256 }, .{ .regs = 220, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_11_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_11_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_11_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_11_2_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_11_2_8: 222 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_11_2_8: .usage = .{ .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_13_2_8: 216 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_13_2_8: 216 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_13_2_8: 216 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_13_2_8: 127 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_13_2_8: 127 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_13_2_8: .usage = .{ .{ .regs = 216, .static = 16, .bound = 256 }, .{ .regs = 216, .static = 16, .bound = 256 }, .{ .regs = 216, .static = 16, .bound = 256 }, .{ .regs = 127, .static = 16, .bound = 256 }, .{ .regs = 127, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_1_8: 224 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_1_8: 224 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_1_8: 224 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_1_8: 126 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_1_8: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_2_1_8: .usage = .{ .{ .regs = 224, .static = 16, .bound = 256 }, .{ .regs = 224, .static = 16, .bound = 256 }, .{ .regs = 224, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_1_8: 224 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_1_8: 224 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_1_8: 224 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_1_8: 126 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_1_8: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_4_1_8: .usage = .{ .{ .regs = 224, .static = 16, .bound = 256 }, .{ .regs = 224, .static = 16, .bound = 256 }, .{ .regs = 224, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_1_8: 160 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_1_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_1_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_6_1_8: .usage = .{ .{ .regs = 160, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_1_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_1_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_1_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_1_8: 126 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_1_8: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_8_1_8: .usage = .{ .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_1_8: 203 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_1_8: 203 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_1_8: 203 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_1_8: 224 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_1_8: 224 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_10_1_8: .usage = .{ .{ .regs = 203, .static = 16, .bound = 256 }, .{ .regs = 203, .static = 16, .bound = 256 }, .{ .regs = 203, .static = 16, .bound = 256 }, .{ .regs = 224, .static = 16, .bound = 256 }, .{ .regs = 224, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_1_8: 200 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_1_8: 200 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_1_8: 200 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_1_8: 220 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_1_8: 220 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_12_1_8: .usage = .{ .{ .regs = 200, .static = 16, .bound = 256 }, .{ .regs = 200, .static = 16, .bound = 256 }, .{ .regs = 200, .static = 16, .bound = 256 }, .{ .regs = 220, .static = 16, .bound = 256 }, .{ .regs = 220, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_1_8: 227 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_1_8: 227 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_1_8: 227 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_14_1_8: .usage = .{ .{ .regs = 227, .static = 16, .bound = 256 }, .{ .regs = 227, .static = 16, .bound = 256 }, .{ .regs = 227, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_1_8: 218 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_1_8: 218 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_1_8: 218 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_1_8: 124 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_1_8: 124 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_16_1_8: .usage = .{ .{ .regs = 218, .static = 16, .bound = 256 }, .{ .regs = 218, .static = 16, .bound = 256 }, .{ .regs = 218, .static = 16, .bound = 256 }, .{ .regs = 124, .static = 16, .bound = 256 }, .{ .regs = 124, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_0_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_0_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_0_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_0_8: 126 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_0_8: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_2_0_8: .usage = .{ .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_0_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_0_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_0_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_0_8: 126 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_0_8: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_4_0_8: .usage = .{ .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_0_8: 159 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_0_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_0_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_6_0_8: .usage = .{ .{ .regs = 159, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_0_8: 228 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_0_8: 228 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_0_8: 228 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_0_8: 126 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_0_8: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_8_0_8: .usage = .{ .{ .regs = 228, .static = 16, .bound = 256 }, .{ .regs = 228, .static = 16, .bound = 256 }, .{ .regs = 228, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, .{ .regs = 126, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_0_8: 196 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_0_8: 196 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_0_8: 196 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_0_8: 223 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_0_8: 223 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_10_0_8: .usage = .{ .{ .regs = 196, .static = 16, .bound = 256 }, .{ .regs = 196, .static = 16, .bound = 256 }, .{ .regs = 196, .static = 16, .bound = 256 }, .{ .regs = 223, .static = 16, .bound = 256 }, .{ .regs = 223, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_0_8: 198 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_0_8: 198 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_0_8: 198 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_0_8: 226 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_0_8: 226 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_12_0_8: .usage = .{ .{ .regs = 198, .static = 16, .bound = 256 }, .{ .regs = 198, .static = 16, .bound = 256 }, .{ .regs = 198, .static = 16, .bound = 256 }, .{ .regs = 226, .static = 16, .bound = 256 }, .{ .regs = 226, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_0_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_0_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_0_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_0_8: 127 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_0_8: 127 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_14_0_8: .usage = .{ .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 127, .static = 16, .bound = 256 }, .{ .regs = 127, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_0_8: 220 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_0_8: 220 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_0_8: 220 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_0_8: 125 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_0_8: 125 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_16_0_8: .usage = .{ .{ .regs = 220, .static = 16, .bound = 256 }, .{ .regs = 220, .static = 16, .bound = 256 }, .{ .regs = 220, .static = 16, .bound = 256 }, .{ .regs = 125, .static = 16, .bound = 256 }, .{ .regs = 125, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_experts tf_exl3_group: 32 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_86 exl3_experts tf_exl3_group: 42 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_89 exl3_experts tf_exl3_group: 42 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_120 exl3_experts tf_exl3_group: 42 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_121 exl3_experts tf_exl3_group: 42 registers, 128 static shared bytes, bound 1024 (record current) +RECORD exl3_experts tf_exl3_group: .usage = .{ .{ .regs = 32, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, } +PASS limits sm_80 exl3_experts tf_exl3_experts_rot_in: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_experts_rot_in: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_experts_rot_in: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_experts_rot_in: 19 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_experts_rot_in: 19 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_experts_rot_in: .usage = .{ .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 19, .static = 0, .bound = 0 }, .{ .regs = 19, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_grouped_0: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_grouped_0: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_grouped_0: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_grouped_0: 127 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_grouped_0: 127 registers, 32832 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_grouped_0: .usage = .{ .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 127, .static = 32832, .bound = 0 }, .{ .regs = 127, .static = 32832, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_grouped_1: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_grouped_1: 127 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_grouped_1: 127 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_grouped_1: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_grouped_1: 128 registers, 32832 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_grouped_1: .usage = .{ .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 127, .static = 32832, .bound = 0 }, .{ .regs = 127, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_grouped_2: .usage = .{ .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_gateup_epilogue: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_gateup_epilogue: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_gateup_epilogue: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_gateup_epilogue: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_gateup_epilogue: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_gateup_epilogue: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_down_combine: 32 registers, 16384 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_down_combine: 40 registers, 16384 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_down_combine: 40 registers, 16384 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_down_combine: 44 registers, 16384 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_down_combine: 44 registers, 16384 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_down_combine: .usage = .{ .{ .regs = 32, .static = 16384, .bound = 0 }, .{ .regs = 40, .static = 16384, .bound = 0 }, .{ .regs = 40, .static = 16384, .bound = 0 }, .{ .regs = 44, .static = 16384, .bound = 0 }, .{ .regs = 44, .static = 16384, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_plan_rotate: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_plan_rotate: 33 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_plan_rotate: 33 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_plan_rotate: 31 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_plan_rotate: 31 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_plan_rotate: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 33, .static = 0, .bound = 0 }, .{ .regs = 33, .static = 0, .bound = 0 }, .{ .regs = 31, .static = 0, .bound = 0 }, .{ .regs = 31, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_plan_finish: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_prompt_rotate: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_prompt_rotate: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_prompt_rotate: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_prompt_rotate: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_prompt_rotate: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_prompt_rotate: .usage = .{ .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_prompt_gu: 128 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_experts tf_exl3_prompt_gu: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_experts tf_exl3_prompt_gu: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_experts tf_exl3_prompt_gu: 124 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_experts tf_exl3_prompt_gu: 124 registers, 43264 static shared bytes, bound 256 (record current) +RECORD exl3_experts tf_exl3_prompt_gu: .usage = .{ .{ .regs = 128, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 124, .static = 43264, .bound = 256 }, .{ .regs = 124, .static = 43264, .bound = 256 }, } +PASS limits sm_80 exl3_experts tf_exl3_prompt_down: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_experts tf_exl3_prompt_down: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_experts tf_exl3_prompt_down: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_experts tf_exl3_prompt_down: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_experts tf_exl3_prompt_down: 126 registers, 43264 static shared bytes, bound 256 (record current) +RECORD exl3_experts tf_exl3_prompt_down: .usage = .{ .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_tc_0: .usage = .{ .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_tc_1: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_tc_1: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_tc_1: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_tc_1: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_tc_1: 128 registers, 8196 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_tc_1: .usage = .{ .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_tc_2: .usage = .{ .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_prompt_0: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_prompt_0: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_prompt_0: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_prompt_0: 75 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_prompt_0: 75 registers, 32768 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_prompt_0: .usage = .{ .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 75, .static = 32768, .bound = 256 }, .{ .regs = 75, .static = 32768, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_prompt_1: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_prompt_1: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_prompt_1: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_prompt_1: 76 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_prompt_1: 76 registers, 32768 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_prompt_1: .usage = .{ .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 76, .static = 32768, .bound = 256 }, .{ .regs = 76, .static = 32768, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_prompt_2: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_prompt_2: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_prompt_2: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_prompt_2: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_prompt_2: 80 registers, 32768 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_prompt_2: .usage = .{ .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_rotate_f: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_rotate_f: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_rotate_f: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_rotate_f: 24 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_rotate_f: 24 registers, 0 static shared bytes, bound 32 (record current) +RECORD exl3_candidate tf_exl3_rotate_f: .usage = .{ .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 24, .static = 0, .bound = 32 }, .{ .regs = 24, .static = 0, .bound = 32 }, } +PASS limits sm_80 exl3_candidate tf_exl3_rotate_h: 26 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_rotate_h: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_rotate_h: 26 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_rotate_h: 20 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_rotate_h: 20 registers, 0 static shared bytes, bound 32 (record current) +RECORD exl3_candidate tf_exl3_rotate_h: .usage = .{ .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 20, .static = 0, .bound = 32 }, .{ .regs = 20, .static = 0, .bound = 32 }, } +PASS limits sm_80 exl3_candidate tf_exl3_codebook: 6 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_codebook: 6 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_codebook: 6 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_codebook: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_codebook: 8 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_candidate tf_exl3_codebook: .usage = .{ .{ .regs = 6, .static = 0, .bound = 0 }, .{ .regs = 6, .static = 0, .bound = 0 }, .{ .regs = 6, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_candidate tf_exl3_unpack: 31 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_unpack: 31 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_unpack: 31 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_unpack: 21 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_unpack: 21 registers, 0 static shared bytes, bound 32 (record current) +RECORD exl3_candidate tf_exl3_unpack: .usage = .{ .{ .regs = 31, .static = 0, .bound = 32 }, .{ .regs = 31, .static = 0, .bound = 32 }, .{ .regs = 31, .static = 0, .bound = 32 }, .{ .regs = 21, .static = 0, .bound = 32 }, .{ .regs = 21, .static = 0, .bound = 32 }, } +PASS limits sm_80 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +RECORD experts3 decode pair: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, } +PASS limits sm_80 experts3 decode down: 196 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 experts3 decode down: 197 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 experts3 decode down: 197 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 experts3 decode down: 190 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 experts3 decode down: 190 registers, 0 static shared bytes, bound 128 (record current) +RECORD experts3 decode down: .usage = .{ .{ .regs = 196, .static = 0, .bound = 128 }, .{ .regs = 197, .static = 0, .bound = 128 }, .{ .regs = 197, .static = 0, .bound = 128 }, .{ .regs = 190, .static = 0, .bound = 128 }, .{ .regs = 190, .static = 0, .bound = 128 }, } +PASS limits sm_80 experts3_prefill pair: 179 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 experts3_prefill pair: 179 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 experts3_prefill pair: 179 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 experts3_prefill pair: 146 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 experts3_prefill pair: 146 registers, 0 static shared bytes, bound 256 (record current) +RECORD experts3_prefill pair: .usage = .{ .{ .regs = 179, .static = 0, .bound = 256 }, .{ .regs = 179, .static = 0, .bound = 256 }, .{ .regs = 179, .static = 0, .bound = 256 }, .{ .regs = 146, .static = 0, .bound = 256 }, .{ .regs = 146, .static = 0, .bound = 256 }, } +PASS limits sm_80 experts3_prefill down: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 experts3_prefill down: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 experts3_prefill down: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 experts3_prefill down: 98 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 experts3_prefill down: 98 registers, 0 static shared bytes, bound 256 (record current) +RECORD experts3_prefill down: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 98, .static = 0, .bound = 256 }, .{ .regs = 98, .static = 0, .bound = 256 }, } +PASS limits sm_80 experts3 tf_experts3_widen: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 experts3 tf_experts3_widen: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 experts3 tf_experts3_widen: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 experts3 tf_experts3_widen: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 experts3 tf_experts3_widen: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD experts3 tf_experts3_widen: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 experts3 tf_experts3_planes: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 experts3 tf_experts3_planes: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 experts3 tf_experts3_planes: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 experts3 tf_experts3_planes: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 experts3 tf_experts3_planes: 36 registers, 0 static shared bytes, bound 0 (record current) +RECORD experts3 tf_experts3_planes: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, } +PASS limits sm_80 h3 h3_norm: 25 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_norm: 26 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_norm: 26 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_norm: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_norm: 32 registers, 128 static shared bytes, bound 256 (record current) +RECORD h3 h3_norm: .usage = .{ .{ .regs = 25, .static = 128, .bound = 256 }, .{ .regs = 26, .static = 128, .bound = 256 }, .{ .regs = 26, .static = 128, .bound = 256 }, .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 32, .static = 128, .bound = 256 }, } +PASS limits sm_80 h3 h3_gate_add: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_gate_add: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_gate_add: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_gate_add: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_gate_add: 22 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_gate_add: .usage = .{ .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_gate_mix: 18 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_gate_mix: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_gate_mix: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_gate_mix: 24 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_gate_mix: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_gate_mix: .usage = .{ .{ .regs = 18, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_swiglu: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_swiglu: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_swiglu: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_swiglu: 24 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_swiglu: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_swiglu: .usage = .{ .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_heads: 234 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_heads: 234 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_heads: 234 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_heads: 255 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_heads: 255 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_heads: .usage = .{ .{ .regs = 234, .static = 0, .bound = 256 }, .{ .regs = 234, .static = 0, .bound = 256 }, .{ .regs = 234, .static = 0, .bound = 256 }, .{ .regs = 255, .static = 0, .bound = 256 }, .{ .regs = 255, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_pool: 31 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 h3 h3_pool: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 h3 h3_pool: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 h3 h3_pool: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 h3 h3_pool: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD h3 h3_pool: .usage = .{ .{ .regs = 31, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 h3 h3_tile_scores: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_tile_scores: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_tile_scores: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_tile_scores: 21 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_tile_scores: 21 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_tile_scores: .usage = .{ .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 21, .static = 0, .bound = 256 }, .{ .regs = 21, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_coarse: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 h3 h3_coarse: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 h3 h3_coarse: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 h3 h3_coarse: 39 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 h3 h3_coarse: 39 registers, 0 static shared bytes, bound 0 (record current) +RECORD h3 h3_coarse: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, } +PASS limits sm_80 h3 h3_attention: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 h3 h3_attention: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 h3 h3_attention: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 h3 h3_attention: 217 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 h3 h3_attention: 217 registers, 0 static shared bytes, bound 128 (record current) +RECORD h3 h3_attention: .usage = .{ .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 217, .static = 0, .bound = 128 }, .{ .regs = 217, .static = 0, .bound = 128 }, } +PASS limits sm_80 h3 h3_topk: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 h3 h3_topk: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 h3 h3_topk: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 h3 h3_topk: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 h3 h3_topk: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD h3 h3_topk: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_gemv: 54 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 drama3 d3_gemv: 54 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 drama3 d3_gemv: 54 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 drama3 d3_gemv: 62 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 drama3 d3_gemv: 62 registers, 0 static shared bytes, bound 256 (record current) +RECORD drama3 d3_gemv: .usage = .{ .{ .regs = 54, .static = 0, .bound = 256 }, .{ .regs = 54, .static = 0, .bound = 256 }, .{ .regs = 54, .static = 0, .bound = 256 }, .{ .regs = 62, .static = 0, .bound = 256 }, .{ .regs = 62, .static = 0, .bound = 256 }, } +PASS limits sm_80 drama3 d3_rmsnorm: 19 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_rmsnorm: 20 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_rmsnorm: 20 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_rmsnorm: 18 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_rmsnorm: 18 registers, 144 static shared bytes, bound 0 (record current) +RECORD drama3 d3_rmsnorm: .usage = .{ .{ .regs = 19, .static = 144, .bound = 0 }, .{ .regs = 20, .static = 144, .bound = 0 }, .{ .regs = 20, .static = 144, .bound = 0 }, .{ .regs = 18, .static = 144, .bound = 0 }, .{ .regs = 18, .static = 144, .bound = 0 }, } +PASS limits sm_80 drama3 d3_add_rmsnorm: 19 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_add_rmsnorm: 21 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_add_rmsnorm: 21 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_add_rmsnorm: 24 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_add_rmsnorm: 24 registers, 144 static shared bytes, bound 0 (record current) +RECORD drama3 d3_add_rmsnorm: .usage = .{ .{ .regs = 19, .static = 144, .bound = 0 }, .{ .regs = 21, .static = 144, .bound = 0 }, .{ .regs = 21, .static = 144, .bound = 0 }, .{ .regs = 24, .static = 144, .bound = 0 }, .{ .regs = 24, .static = 144, .bound = 0 }, } +PASS limits sm_80 drama3 d3_embed_backbone: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_embed_backbone: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_embed_backbone: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_embed_backbone: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_embed_backbone: 24 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_embed_backbone: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_conv_silu: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_conv_silu: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_conv_silu: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_conv_silu: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_conv_silu: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_conv_silu: .usage = .{ .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +PASS limits sm_86 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +PASS limits sm_89 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +PASS limits sm_120 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +PASS limits sm_121 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +RECORD drama3 d3_gdn: .usage = .{ .{ .regs = 255, .static = 1024, .bound = 32 }, .{ .regs = 255, .static = 1024, .bound = 32 }, .{ .regs = 255, .static = 1024, .bound = 32 }, .{ .regs = 255, .static = 1024, .bound = 32 }, .{ .regs = 255, .static = 1024, .bound = 32 }, } +PASS limits sm_80 drama3 d3_gated_norm: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_gated_norm: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_gated_norm: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_gated_norm: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_gated_norm: 26 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_gated_norm: .usage = .{ .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_qk_rope: 28 registers, 1168 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_qk_rope: 28 registers, 1168 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_qk_rope: 28 registers, 1168 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_qk_rope: 30 registers, 1168 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_qk_rope: 30 registers, 1168 static shared bytes, bound 0 (record current) +RECORD drama3 d3_qk_rope: .usage = .{ .{ .regs = 28, .static = 1168, .bound = 0 }, .{ .regs = 28, .static = 1168, .bound = 0 }, .{ .regs = 28, .static = 1168, .bound = 0 }, .{ .regs = 30, .static = 1168, .bound = 0 }, .{ .regs = 30, .static = 1168, .bound = 0 }, } +PASS limits sm_80 drama3 d3_attn_decode: 38 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_86 drama3 d3_attn_decode: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_89 drama3 d3_attn_decode: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_120 drama3 d3_attn_decode: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_121 drama3 d3_attn_decode: 40 registers, 8256 static shared bytes, bound 256 (record current) +RECORD drama3 d3_attn_decode: .usage = .{ .{ .regs = 38, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, } +PASS limits sm_80 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_86 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_89 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_120 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_121 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +RECORD drama3 d3_attn_partial: .usage = .{ .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, } +PASS limits sm_80 drama3 d3_attn_merge: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_attn_merge: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_q_group: .usage = .{ .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_causal_softmax: 17 registers, 272 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_causal_softmax: 19 registers, 272 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_causal_softmax: 19 registers, 272 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_causal_softmax: 20 registers, 272 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_causal_softmax: 20 registers, 272 static shared bytes, bound 0 (record current) +RECORD drama3 d3_causal_softmax: .usage = .{ .{ .regs = 17, .static = 272, .bound = 0 }, .{ .regs = 19, .static = 272, .bound = 0 }, .{ .regs = 19, .static = 272, .bound = 0 }, .{ .regs = 20, .static = 272, .bound = 0 }, .{ .regs = 20, .static = 272, .bound = 0 }, } +PASS limits sm_80 drama3 d3_attn_gate: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_attn_gate: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_attn_gate: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_attn_gate: 19 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_attn_gate: 19 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_attn_gate: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 19, .static = 0, .bound = 0 }, .{ .regs = 19, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_bf16_f32: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_bf16_f32: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_bf16_f32: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_bf16_f32: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_bf16_f32: 16 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_bf16_f32: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_router: 17 registers, 592 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_router: 20 registers, 592 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_router: 20 registers, 592 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_router: 26 registers, 592 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_router: 26 registers, 592 static shared bytes, bound 0 (record current) +RECORD drama3 d3_router: .usage = .{ .{ .regs = 17, .static = 592, .bound = 0 }, .{ .regs = 20, .static = 592, .bound = 0 }, .{ .regs = 20, .static = 592, .bound = 0 }, .{ .regs = 26, .static = 592, .bound = 0 }, .{ .regs = 26, .static = 592, .bound = 0 }, } +PASS limits sm_80 drama3 d3_swiglu: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_swiglu: .usage = .{ .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_moe_combine: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_moe_combine: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_moe_combine: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_moe_combine: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_moe_combine: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_moe_combine: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_act_quant: 23 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_act_quant: 25 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_act_quant: 23 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_act_quant: 21 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_act_quant: 21 registers, 144 static shared bytes, bound 0 (record current) +RECORD drama3 d3_act_quant: .usage = .{ .{ .regs = 23, .static = 144, .bound = 0 }, .{ .regs = 25, .static = 144, .bound = 0 }, .{ .regs = 23, .static = 144, .bound = 0 }, .{ .regs = 21, .static = 144, .bound = 0 }, .{ .regs = 21, .static = 144, .bound = 0 }, } +PASS limits sm_80 drama3 d3_tower_mix: 40 registers, 8336 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_tower_mix: 40 registers, 8336 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_tower_mix: 40 registers, 8336 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_tower_mix: 36 registers, 8336 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_tower_mix: 36 registers, 8336 static shared bytes, bound 0 (record current) +RECORD drama3 d3_tower_mix: .usage = .{ .{ .regs = 40, .static = 8336, .bound = 0 }, .{ .regs = 40, .static = 8336, .bound = 0 }, .{ .regs = 40, .static = 8336, .bound = 0 }, .{ .regs = 36, .static = 8336, .bound = 0 }, .{ .regs = 36, .static = 8336, .bound = 0 }, } +PASS limits sm_80 drama3 d3_sample_main: 31 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_86 drama3 d3_sample_main: 37 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_89 drama3 d3_sample_main: 37 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_120 drama3 d3_sample_main: 42 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_121 drama3 d3_sample_main: 49 registers, 400 static shared bytes, bound 1024 (record current) +RECORD drama3 d3_sample_main: .usage = .{ .{ .regs = 31, .static = 400, .bound = 1024 }, .{ .regs = 37, .static = 400, .bound = 1024 }, .{ .regs = 37, .static = 400, .bound = 1024 }, .{ .regs = 42, .static = 400, .bound = 1024 }, .{ .regs = 49, .static = 400, .bound = 1024 }, } +PASS limits sm_80 drama3 d3_sample_main_fast: 31 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_86 drama3 d3_sample_main_fast: 35 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_89 drama3 d3_sample_main_fast: 35 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_120 drama3 d3_sample_main_fast: 37 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_121 drama3 d3_sample_main_fast: 37 registers, 3616 static shared bytes, bound 1024 (record current) +RECORD drama3 d3_sample_main_fast: .usage = .{ .{ .regs = 31, .static = 3616, .bound = 1024 }, .{ .regs = 35, .static = 3616, .bound = 1024 }, .{ .regs = 35, .static = 3616, .bound = 1024 }, .{ .regs = 37, .static = 3616, .bound = 1024 }, .{ .regs = 37, .static = 3616, .bound = 1024 }, } +PASS limits sm_80 drama3 d3_sample_code: 30 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_86 drama3 d3_sample_code: 31 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_89 drama3 d3_sample_code: 31 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_120 drama3 d3_sample_code: 39 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_121 drama3 d3_sample_code: 39 registers, 400 static shared bytes, bound 1024 (record current) +RECORD drama3 d3_sample_code: .usage = .{ .{ .regs = 30, .static = 400, .bound = 1024 }, .{ .regs = 31, .static = 400, .bound = 1024 }, .{ .regs = 31, .static = 400, .bound = 1024 }, .{ .regs = 39, .static = 400, .bound = 1024 }, .{ .regs = 39, .static = 400, .bound = 1024 }, } +PASS limits sm_80 drama3 d3_sample_code_fast: 25 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_86 drama3 d3_sample_code_fast: 25 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_89 drama3 d3_sample_code_fast: 25 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_120 drama3 d3_sample_code_fast: 27 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_121 drama3 d3_sample_code_fast: 29 registers, 3616 static shared bytes, bound 1024 (record current) +RECORD drama3 d3_sample_code_fast: .usage = .{ .{ .regs = 25, .static = 3616, .bound = 1024 }, .{ .regs = 25, .static = 3616, .bound = 1024 }, .{ .regs = 25, .static = 3616, .bound = 1024 }, .{ .regs = 27, .static = 3616, .bound = 1024 }, .{ .regs = 29, .static = 3616, .bound = 1024 }, } +PASS limits sm_80 drama3 d3_argmax_main: 12 registers, 256 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_argmax_main: 14 registers, 256 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_argmax_main: 14 registers, 256 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_argmax_main: 13 registers, 256 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_argmax_main: 13 registers, 256 static shared bytes, bound 0 (record current) +RECORD drama3 d3_argmax_main: .usage = .{ .{ .regs = 12, .static = 256, .bound = 0 }, .{ .regs = 14, .static = 256, .bound = 0 }, .{ .regs = 14, .static = 256, .bound = 0 }, .{ .regs = 13, .static = 256, .bound = 0 }, .{ .regs = 13, .static = 256, .bound = 0 }, } +PASS limits sm_80 drama3 d3_finish_frame: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_finish_frame: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_finish_frame: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_finish_frame: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_finish_frame: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_finish_frame: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_next_input: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_copy_i32: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_copy_i32: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_copy_i32: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_copy_i32: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_copy_i32: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_copy_i32: .usage = .{ .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_vq_gather: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_vq_gather: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_vq_gather: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_vq_gather: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_vq_gather: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_vq_gather: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_vq_encode: 31 registers, 424 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_vq_encode: 39 registers, 424 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_vq_encode: 39 registers, 424 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_vq_encode: 37 registers, 424 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_vq_encode: 39 registers, 424 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_vq_encode: .usage = .{ .{ .regs = 31, .static = 424, .bound = 0 }, .{ .regs = 39, .static = 424, .bound = 0 }, .{ .regs = 39, .static = 424, .bound = 0 }, .{ .regs = 37, .static = 424, .bound = 0 }, .{ .regs = 39, .static = 424, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_rmsnorm: 15 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_rmsnorm: 16 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_rmsnorm: 16 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_rmsnorm: 18 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_rmsnorm: 18 registers, 132 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_rmsnorm: .usage = .{ .{ .regs = 15, .static = 132, .bound = 0 }, .{ .regs = 16, .static = 132, .bound = 0 }, .{ .regs = 16, .static = 132, .bound = 0 }, .{ .regs = 18, .static = 132, .bound = 0 }, .{ .regs = 18, .static = 132, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_layernorm: 21 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_layernorm: 23 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_layernorm: 23 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_layernorm: 24 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_layernorm: 24 registers, 132 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_layernorm: .usage = .{ .{ .regs = 21, .static = 132, .bound = 0 }, .{ .regs = 23, .static = 132, .bound = 0 }, .{ .regs = 23, .static = 132, .bound = 0 }, .{ .regs = 24, .static = 132, .bound = 0 }, .{ .regs = 24, .static = 132, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_rope_kv: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_rope_kv: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_rope_kv: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_rope_kv: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_rope_kv: 24 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_rope_kv: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_attn: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_attn: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_attn: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_attn: 34 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_attn: 34 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_attn: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_add_gamma: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_add_gamma: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_add_gamma: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_add_gamma: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_add_gamma: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_add_gamma: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_silu_mul: 23 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_silu_mul: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_silu_mul: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_silu_mul: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_silu_mul: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_silu_mul: .usage = .{ .{ .regs = 23, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_gelu: 15 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_gelu: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_gelu: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_gelu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_gelu: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_gelu: .usage = .{ .{ .regs = 15, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_bias: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_bias: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_bias: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_bias: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_bias: 26 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_bias: .usage = .{ .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_snake: 21 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_snake: 21 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_snake: 21 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_snake: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_snake: 26 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_snake: .usage = .{ .{ .regs = 21, .static = 0, .bound = 0 }, .{ .regs = 21, .static = 0, .bound = 0 }, .{ .regs = 21, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_dwconv7: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_dwconv7: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_dwconv7: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_dwconv7: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_dwconv7: 36 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_dwconv7: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_copy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_copy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_copy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_copy: 12 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_copy: 12 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_copy: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_fp8 decode gate-up: 157 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 drama3_fp8 decode gate-up: 157 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 drama3_fp8 decode gate-up: 157 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 drama3_fp8 decode gate-up: 164 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 drama3_fp8 decode gate-up: 164 registers, 0 static shared bytes, bound 128 (record current) +RECORD drama3_fp8 decode gate-up: .usage = .{ .{ .regs = 157, .static = 0, .bound = 128 }, .{ .regs = 157, .static = 0, .bound = 128 }, .{ .regs = 157, .static = 0, .bound = 128 }, .{ .regs = 164, .static = 0, .bound = 128 }, .{ .regs = 164, .static = 0, .bound = 128 }, } +PASS limits sm_80 drama3_fp8 decode down: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 drama3_fp8 decode down: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 drama3_fp8 decode down: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 drama3_fp8 decode down: 118 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 drama3_fp8 decode down: 118 registers, 0 static shared bytes, bound 128 (record current) +RECORD drama3_fp8 decode down: .usage = .{ .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 118, .static = 0, .bound = 128 }, .{ .regs = 118, .static = 0, .bound = 128 }, } +PASS limits sm_80 drama3_fp8 prompt gate-up: 213 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_86 drama3_fp8 prompt gate-up: 212 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_89 drama3_fp8 prompt gate-up: 212 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_120 drama3_fp8 prompt gate-up: 213 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_121 drama3_fp8 prompt gate-up: 213 registers, 32768 static shared bytes, bound 128 (record current) +RECORD drama3_fp8 prompt gate-up: .usage = .{ .{ .regs = 213, .static = 32768, .bound = 128 }, .{ .regs = 212, .static = 32768, .bound = 128 }, .{ .regs = 212, .static = 32768, .bound = 128 }, .{ .regs = 213, .static = 32768, .bound = 128 }, .{ .regs = 213, .static = 32768, .bound = 128 }, } +PASS limits sm_80 drama3_fp8 prompt down: 213 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_86 drama3_fp8 prompt down: 211 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_89 drama3_fp8 prompt down: 211 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_120 drama3_fp8 prompt down: 213 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_121 drama3_fp8 prompt down: 213 registers, 32768 static shared bytes, bound 128 (record current) +RECORD drama3_fp8 prompt down: .usage = .{ .{ .regs = 213, .static = 32768, .bound = 128 }, .{ .regs = 211, .static = 32768, .bound = 128 }, .{ .regs = 211, .static = 32768, .bound = 128 }, .{ .regs = 213, .static = 32768, .bound = 128 }, .{ .regs = 213, .static = 32768, .bound = 128 }, } +PASS limits sm_80 router tf_nemo_router_tiles: 181 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 router tf_nemo_router_tiles: 184 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 router tf_nemo_router_tiles: 184 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 router tf_nemo_router_tiles: 177 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 router tf_nemo_router_tiles: 177 registers, 0 static shared bytes, bound 256 (record current) +RECORD router tf_nemo_router_tiles: .usage = .{ .{ .regs = 181, .static = 0, .bound = 256 }, .{ .regs = 184, .static = 0, .bound = 256 }, .{ .regs = 184, .static = 0, .bound = 256 }, .{ .regs = 177, .static = 0, .bound = 256 }, .{ .regs = 177, .static = 0, .bound = 256 }, } +PASS limits sm_80 latent tf_super_combine_sums: 32 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 latent tf_super_combine_sums: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 latent tf_super_combine_sums: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 latent tf_super_combine_sums: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 latent tf_super_combine_sums: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD latent tf_super_combine_sums: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 latent tf_super_moe_out_sums: 26 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 latent tf_super_moe_out_sums: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 latent tf_super_moe_out_sums: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 latent tf_super_moe_out_sums: 24 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 latent tf_super_moe_out_sums: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD latent tf_super_moe_out_sums: .usage = .{ .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits ck tf_ck4_up_nvfp4_prompt sm_80: below_minimum +N/A limits ck tf_ck4_up_nvfp4_prompt sm_86: below_minimum +N/A limits ck tf_ck4_up_nvfp4_prompt sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_up_nvfp4_prompt: 114 registers, 46080 static shared bytes, bound 256 (record current) +PASS limits sm_121 ck tf_ck4_up_nvfp4_prompt: 114 registers, 46080 static shared bytes, bound 256 (record current) +RECORD ck tf_ck4_up_nvfp4_prompt: .usage = .{ .{ .regs = 114, .static = 46080, .bound = 256 }, .{ .regs = 114, .static = 46080, .bound = 256 }, } +N/A limits ck tf_ck4_down_f32_prompt sm_80: below_minimum +N/A limits ck tf_ck4_down_f32_prompt sm_86: below_minimum +N/A limits ck tf_ck4_down_f32_prompt sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_down_f32_prompt: 128 registers, 36864 static shared bytes, bound 256 (record current) +PASS limits sm_121 ck tf_ck4_down_f32_prompt: 128 registers, 36864 static shared bytes, bound 256 (record current) +RECORD ck tf_ck4_down_f32_prompt: .usage = .{ .{ .regs = 128, .static = 36864, .bound = 256 }, .{ .regs = 128, .static = 36864, .bound = 256 }, } +N/A limits ck tf_ck4_down_sum_prompt sm_80: below_minimum +N/A limits ck tf_ck4_down_sum_prompt sm_86: below_minimum +N/A limits ck tf_ck4_down_sum_prompt sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_down_sum_prompt: 222 registers, 36864 static shared bytes, bound 256 (record current) +PASS limits sm_121 ck tf_ck4_down_sum_prompt: 222 registers, 36864 static shared bytes, bound 256 (record current) +RECORD ck tf_ck4_down_sum_prompt: .usage = .{ .{ .regs = 222, .static = 36864, .bound = 256 }, .{ .regs = 222, .static = 36864, .bound = 256 }, } +N/A limits ck tf_ck8_128 sm_80: below_minimum +N/A limits ck tf_ck8_128 sm_86: below_minimum +PASS limits sm_89 ck tf_ck8_128: 122 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 ck tf_ck8_128: 76 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 ck tf_ck8_128: 76 registers, 0 static shared bytes, bound 256 (record current) +RECORD ck tf_ck8_128: .usage = .{ .{ .regs = 122, .static = 0, .bound = 256 }, .{ .regs = 76, .static = 0, .bound = 256 }, .{ .regs = 76, .static = 0, .bound = 256 }, } +N/A limits ck tf_ck8_128c cluster of 8 sm_80: below_minimum +N/A limits ck tf_ck8_128c cluster of 8 sm_86: below_minimum +N/A limits ck tf_ck8_128c cluster of 8 sm_89: below_minimum +PASS limits sm_120 ck tf_ck8_128c cluster of 8: 104 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 ck tf_ck8_128c cluster of 8: 104 registers, 0 static shared bytes, bound 256 (record current) +RECORD ck tf_ck8_128c cluster of 8: .usage = .{ .{ .regs = 104, .static = 0, .bound = 256 }, .{ .regs = 104, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm16_16: 95 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm16_16: 95 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm16_16: 95 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm16_16: 52 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm16_16: 52 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm16_16: .usage = .{ .{ .regs = 95, .static = 0, .bound = 128 }, .{ .regs = 95, .static = 0, .bound = 128 }, .{ .regs = 95, .static = 0, .bound = 128 }, .{ .regs = 52, .static = 0, .bound = 128 }, .{ .regs = 52, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm16_32: 95 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm16_32: 95 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm16_32: 95 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm16_32: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm16_32: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm16_32: .usage = .{ .{ .regs = 95, .static = 0, .bound = 128 }, .{ .regs = 95, .static = 0, .bound = 128 }, .{ .regs = 95, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm16_64: 95 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm16_64: 95 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm16_64: 95 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm16_64: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm16_64: 64 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm16_64: .usage = .{ .{ .regs = 95, .static = 0, .bound = 256 }, .{ .regs = 95, .static = 0, .bound = 256 }, .{ .regs = 95, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm16_128: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm16_128: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm16_128: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm16_128: 100 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm16_128: 100 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm16_128: .usage = .{ .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 100, .static = 0, .bound = 256 }, .{ .regs = 100, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_tile16: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_tile16: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_tile16: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_tile16: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_tile16: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_tile16: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_mm4_16: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm4_16: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm4_16: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm4_16: 50 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm4_16: 50 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm4_16: .usage = .{ .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 50, .static = 0, .bound = 128 }, .{ .regs = 50, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm4_32: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm4_32: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm4_32: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm4_32: 65 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm4_32: 65 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm4_32: .usage = .{ .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 65, .static = 0, .bound = 128 }, .{ .regs = 65, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm4_64: 80 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm4_64: 80 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm4_64: 80 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm4_64: 61 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm4_64: 61 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm4_64: .usage = .{ .{ .regs = 80, .static = 0, .bound = 256 }, .{ .regs = 80, .static = 0, .bound = 256 }, .{ .regs = 80, .static = 0, .bound = 256 }, .{ .regs = 61, .static = 0, .bound = 256 }, .{ .regs = 61, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm4_128: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm4_128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm4_128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm4_128: 106 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm4_128: 106 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm4_128: .usage = .{ .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_tile4: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_tile4: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_tile4: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_tile4: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_tile4: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_tile4: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_mm6_16: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm6_16: 91 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm6_16: 91 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm6_16: 54 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm6_16: 54 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm6_16: .usage = .{ .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 91, .static = 0, .bound = 128 }, .{ .regs = 91, .static = 0, .bound = 128 }, .{ .regs = 54, .static = 0, .bound = 128 }, .{ .regs = 54, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm6_32: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm6_32: 127 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm6_32: 127 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm6_32: 70 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm6_32: 70 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm6_32: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 127, .static = 0, .bound = 128 }, .{ .regs = 127, .static = 0, .bound = 128 }, .{ .regs = 70, .static = 0, .bound = 128 }, .{ .regs = 70, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm6_64: 97 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm6_64: 91 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm6_64: 91 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm6_64: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm6_64: 64 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm6_64: .usage = .{ .{ .regs = 97, .static = 0, .bound = 256 }, .{ .regs = 91, .static = 0, .bound = 256 }, .{ .regs = 91, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm6_128: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm6_128: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm6_128: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm6_128: 106 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm6_128: 106 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm6_128: .usage = .{ .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_tile6: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_tile6: 39 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_tile6: 39 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_tile6: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_tile6: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_tile6: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_mm8_16: 93 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm8_16: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm8_16: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm8_16: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm8_16: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm8_16: .usage = .{ .{ .regs = 93, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm8_32: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm8_32: 126 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm8_32: 126 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm8_32: 68 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm8_32: 68 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm8_32: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 126, .static = 0, .bound = 128 }, .{ .regs = 126, .static = 0, .bound = 128 }, .{ .regs = 68, .static = 0, .bound = 128 }, .{ .regs = 68, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm8_64: 92 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm8_64: 92 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm8_64: 92 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm8_64: 63 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm8_64: 63 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm8_64: .usage = .{ .{ .regs = 92, .static = 0, .bound = 256 }, .{ .regs = 92, .static = 0, .bound = 256 }, .{ .regs = 92, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm8_128: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm8_128: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm8_128: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm8_128: 106 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm8_128: 106 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm8_128: .usage = .{ .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_tile8: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_tile8: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_tile8: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_tile8: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_tile8: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_tile8: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_mm16_rows: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm16_rows: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm16_rows: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm16_rows: 99 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm16_rows: 99 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm16_rows: .usage = .{ .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 99, .static = 0, .bound = 256 }, .{ .regs = 99, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_reduce: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_reduce: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_reduce: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_reduce: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_reduce: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_reduce: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_tap: 15 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_tap: 15 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_tap: 15 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_tap: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_tap: 36 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_tap: .usage = .{ .{ .regs = 15, .static = 0, .bound = 0 }, .{ .regs = 15, .static = 0, .bound = 0 }, .{ .regs = 15, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_block_qkv: 32 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_block_qkv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_block_qkv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_block_qkv: 27 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_block_qkv: 27 registers, 8 static shared bytes, bound 0 (record current) +RECORD df_block_qkv: .usage = .{ .{ .regs = 32, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 27, .static = 8, .bound = 0 }, .{ .regs = 27, .static = 8, .bound = 0 }, } +PASS limits sm_80 df_ctx_kv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_ctx_kv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_ctx_kv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_ctx_kv: 32 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_ctx_kv: 32 registers, 8 static shared bytes, bound 0 (record current) +RECORD df_ctx_kv: .usage = .{ .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 32, .static = 8, .bound = 0 }, .{ .regs = 32, .static = 8, .bound = 0 }, } +PASS limits sm_80 df_conv: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_conv: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_conv: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_conv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_conv: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_conv: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_swiglu: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_swiglu: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_swiglu: .usage = .{ .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_attn_part: 74 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 df_attn_part: 77 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 df_attn_part: 77 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 df_attn_part: 75 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 df_attn_part: 75 registers, 0 static shared bytes, bound 256 (record current) +RECORD df_attn_part: .usage = .{ .{ .regs = 74, .static = 0, .bound = 256 }, .{ .regs = 77, .static = 0, .bound = 256 }, .{ .regs = 77, .static = 0, .bound = 256 }, .{ .regs = 75, .static = 0, .bound = 256 }, .{ .regs = 75, .static = 0, .bound = 256 }, } +PASS limits sm_80 df_attn_merge: 39 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_attn_merge: .usage = .{ .{ .regs = 39, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_sum2: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_sum2: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_sum2: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_sum2: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_sum2: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_sum2: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_top16_part: 32 registers, 32864 static shared bytes, bound 256 (record current) +PASS limits sm_86 df_top16_part: 32 registers, 32864 static shared bytes, bound 256 (record current) +PASS limits sm_89 df_top16_part: 32 registers, 32864 static shared bytes, bound 256 (record current) +PASS limits sm_120 df_top16_part: 30 registers, 32864 static shared bytes, bound 256 (record current) +PASS limits sm_121 df_top16_part: 30 registers, 32864 static shared bytes, bound 256 (record current) +RECORD df_top16_part: .usage = .{ .{ .regs = 32, .static = 32864, .bound = 256 }, .{ .regs = 32, .static = 32864, .bound = 256 }, .{ .regs = 32, .static = 32864, .bound = 256 }, .{ .regs = 30, .static = 32864, .bound = 256 }, .{ .regs = 30, .static = 32864, .bound = 256 }, } +PASS limits sm_80 df_top16_merge: 23 registers, 8416 static shared bytes, bound 256 (record current) +PASS limits sm_86 df_top16_merge: 23 registers, 8416 static shared bytes, bound 256 (record current) +PASS limits sm_89 df_top16_merge: 23 registers, 8416 static shared bytes, bound 256 (record current) +PASS limits sm_120 df_top16_merge: 23 registers, 8416 static shared bytes, bound 256 (record current) +PASS limits sm_121 df_top16_merge: 23 registers, 8416 static shared bytes, bound 256 (record current) +RECORD df_top16_merge: .usage = .{ .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, } +PASS limits sm_80 df_gsum: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_gsum: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_gsum: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_gsum: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_gsum: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_gsum: .usage = .{ .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_glue tf_glm_lat_write: 13 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 glm_glue tf_glm_lat_write: 13 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 glm_glue tf_glm_lat_write: 13 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 glm_glue tf_glm_lat_write: 36 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 glm_glue tf_glm_lat_write: 36 registers, 0 static shared bytes, bound 0 (record stale) +RECORD glm_glue tf_glm_lat_write: .usage = .{ .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_glue tf_glm_pool_keys: 25 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_86 glm_glue tf_glm_pool_keys: 26 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_89 glm_glue tf_glm_pool_keys: 26 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_120 glm_glue tf_glm_pool_keys: 24 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 glm_glue tf_glm_pool_keys: 24 registers, 0 static shared bytes, bound 32 (record stale) +RECORD glm_glue tf_glm_pool_keys: .usage = .{ .{ .regs = 25, .static = 0, .bound = 32 }, .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 24, .static = 0, .bound = 32 }, .{ .regs = 24, .static = 0, .bound = 32 }, } +PASS limits sm_80 glm_glue tf_glm_scores: 126 registers, 32768 static shared bytes, bound 128 (record stale) +PASS limits sm_86 glm_glue tf_glm_scores: 131 registers, 32768 static shared bytes, bound 128 (record stale) +PASS limits sm_89 glm_glue tf_glm_scores: 131 registers, 32768 static shared bytes, bound 128 (record stale) +PASS limits sm_120 glm_glue tf_glm_scores: 155 registers, 32768 static shared bytes, bound 128 (record stale) +PASS limits sm_121 glm_glue tf_glm_scores: 155 registers, 32768 static shared bytes, bound 128 (record stale) +RECORD glm_glue tf_glm_scores: .usage = .{ .{ .regs = 126, .static = 32768, .bound = 128 }, .{ .regs = 131, .static = 32768, .bound = 128 }, .{ .regs = 131, .static = 32768, .bound = 128 }, .{ .regs = 155, .static = 32768, .bound = 128 }, .{ .regs = 155, .static = 32768, .bound = 128 }, } +PASS limits sm_80 glm_glue tf_glm_chunks_16: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_glue tf_glm_chunks_16: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_glue tf_glm_chunks_16: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_glue tf_glm_chunks_16: 123 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 glm_glue tf_glm_chunks_16: 123 registers, 0 static shared bytes, bound 256 (record stale) +RECORD glm_glue tf_glm_chunks_16: .usage = .{ .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 123, .static = 0, .bound = 256 }, .{ .regs = 123, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_glue tf_glm_chunks_32: 150 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_glue tf_glm_chunks_32: 150 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_glue tf_glm_chunks_32: 150 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_glue tf_glm_chunks_32: 130 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 glm_glue tf_glm_chunks_32: 130 registers, 0 static shared bytes, bound 256 (record stale) +RECORD glm_glue tf_glm_chunks_32: .usage = .{ .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 130, .static = 0, .bound = 256 }, .{ .regs = 130, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_ops tf_glm_dsa_tokens: 25 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 glm_ops tf_glm_dsa_tokens: 25 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 glm_ops tf_glm_dsa_tokens: 25 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 glm_ops tf_glm_dsa_tokens: 24 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 glm_ops tf_glm_dsa_tokens: 24 registers, 0 static shared bytes, bound 0 (record stale) +RECORD glm_ops tf_glm_dsa_tokens: .usage = .{ .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree attention tf_tree_paths: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 tree attention tf_tree_paths: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 tree attention tf_tree_paths: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 tree attention tf_tree_paths: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 tree attention tf_tree_paths: 30 registers, 0 static shared bytes, bound 0 (record current) +RECORD tree attention tf_tree_paths: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree attention tf_tree_shared: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 tree attention tf_tree_shared: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 tree attention tf_tree_shared: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 tree attention tf_tree_shared: 212 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 tree attention tf_tree_shared: 212 registers, 0 static shared bytes, bound 128 (record current) +RECORD tree attention tf_tree_shared: .usage = .{ .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, } +PASS limits sm_80 tree attention tf_tree_shared_w1: 193 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_shared_w1: 193 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_shared_w1: 193 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_shared_w1: 195 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_shared_w1: 195 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_shared_w1: .usage = .{ .{ .regs = 193, .static = 0, .bound = 32 }, .{ .regs = 193, .static = 0, .bound = 32 }, .{ .regs = 193, .static = 0, .bound = 32 }, .{ .regs = 195, .static = 0, .bound = 32 }, .{ .regs = 195, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_shared_w1s3: 190 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_shared_w1s3: 190 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_shared_w1s3: 190 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_shared_w1s3: 198 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_shared_w1s3: 198 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_shared_w1s3: .usage = .{ .{ .regs = 190, .static = 0, .bound = 32 }, .{ .regs = 190, .static = 0, .bound = 32 }, .{ .regs = 190, .static = 0, .bound = 32 }, .{ .regs = 198, .static = 0, .bound = 32 }, .{ .regs = 198, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_part: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 tree attention tf_tree_part: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 tree attention tf_tree_part: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 tree attention tf_tree_part: 228 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 tree attention tf_tree_part: 228 registers, 0 static shared bytes, bound 128 (record current) +RECORD tree attention tf_tree_part: .usage = .{ .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 228, .static = 0, .bound = 128 }, .{ .regs = 228, .static = 0, .bound = 128 }, } +PASS limits sm_80 tree attention tf_tree_part_w1: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_part_w1: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_part_w1: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_part_w1: 168 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_part_w1: 168 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_part_w1: .usage = .{ .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_part_w1s3: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_part_w1s3: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_part_w1s3: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_part_w1s3: 202 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_part_w1s3: 202 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_part_w1s3: .usage = .{ .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 202, .static = 0, .bound = 32 }, .{ .regs = 202, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +RECORD tree attention tf_tree_shared_256: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, } +PASS limits sm_80 tree attention tf_tree_shared_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_shared_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_shared_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_shared_256_w1: 249 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_shared_256_w1: 249 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_shared_256_w1: .usage = .{ .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 249, .static = 0, .bound = 32 }, .{ .regs = 249, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_shared_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_shared_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_shared_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_shared_256_w1s3: 250 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_shared_256_w1s3: 250 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_shared_256_w1s3: .usage = .{ .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 250, .static = 0, .bound = 32 }, .{ .regs = 250, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +RECORD tree attention tf_tree_part_256: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, } +PASS limits sm_80 tree attention tf_tree_part_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_part_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_part_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_part_256_w1: 253 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_part_256_w1: 253 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_part_256_w1: .usage = .{ .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 253, .static = 0, .bound = 32 }, .{ .regs = 253, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_part_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_part_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_part_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_part_256_w1s3: 251 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_part_256_w1s3: 251 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_part_256_w1s3: .usage = .{ .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 251, .static = 0, .bound = 32 }, .{ .regs = 251, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_merge: 114 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 tree attention tf_tree_merge: 114 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 tree attention tf_tree_merge: 114 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 tree attention tf_tree_merge: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 tree attention tf_tree_merge: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD tree attention tf_tree_merge: .usage = .{ .{ .regs = 114, .static = 0, .bound = 256 }, .{ .regs = 114, .static = 0, .bound = 256 }, .{ .regs = 114, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, } +PASS limits sm_80 block attention tf_block_attention (up to 8 query heads a key head): 230 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 block attention tf_block_attention (up to 8 query heads a key head): 230 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 block attention tf_block_attention (up to 8 query heads a key head): 230 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 block attention tf_block_attention (up to 8 query heads a key head): 228 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 block attention tf_block_attention (up to 8 query heads a key head): 228 registers, 0 static shared bytes, bound 256 (record current) +RECORD block attention tf_block_attention (up to 8 query heads a key head): .usage = .{ .{ .regs = 230, .static = 0, .bound = 256 }, .{ .regs = 230, .static = 0, .bound = 256 }, .{ .regs = 230, .static = 0, .bound = 256 }, .{ .regs = 228, .static = 0, .bound = 256 }, .{ .regs = 228, .static = 0, .bound = 256 }, } +PASS limits sm_80 window attention tf_window_part: 55 registers, 45824 static shared bytes, bound 256 (record current) +PASS limits sm_86 window attention tf_window_part: 50 registers, 45824 static shared bytes, bound 256 (record current) +PASS limits sm_89 window attention tf_window_part: 50 registers, 45824 static shared bytes, bound 256 (record current) +PASS limits sm_120 window attention tf_window_part: 56 registers, 45824 static shared bytes, bound 256 (record current) +PASS limits sm_121 window attention tf_window_part: 56 registers, 45824 static shared bytes, bound 256 (record current) +RECORD window attention tf_window_part: .usage = .{ .{ .regs = 55, .static = 45824, .bound = 256 }, .{ .regs = 50, .static = 45824, .bound = 256 }, .{ .regs = 50, .static = 45824, .bound = 256 }, .{ .regs = 56, .static = 45824, .bound = 256 }, .{ .regs = 56, .static = 45824, .bound = 256 }, } +PASS limits sm_80 window attention tf_window_merge: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 window attention tf_window_merge: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 window attention tf_window_merge: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 window attention tf_window_merge: 48 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 window attention tf_window_merge: 48 registers, 0 static shared bytes, bound 256 (record current) +RECORD window attention tf_window_merge: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 48, .static = 0, .bound = 256 }, .{ .regs = 48, .static = 0, .bound = 256 }, } +PASS limits sm_80 window attention tf_window_decode_tc: 176 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 window attention tf_window_decode_tc: 176 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 window attention tf_window_decode_tc: 176 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 window attention tf_window_decode_tc: 178 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 window attention tf_window_decode_tc: 178 registers, 0 static shared bytes, bound 128 (record current) +RECORD window attention tf_window_decode_tc: .usage = .{ .{ .regs = 176, .static = 0, .bound = 128 }, .{ .regs = 176, .static = 0, .bound = 128 }, .{ .regs = 176, .static = 0, .bound = 128 }, .{ .regs = 178, .static = 0, .bound = 128 }, .{ .regs = 178, .static = 0, .bound = 128 }, } +PASS limits sm_80 window attention wattn_kernel<128, 8, 4, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 window attention wattn_kernel<128, 8, 4, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 window attention wattn_kernel<128, 8, 4, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 window attention wattn_kernel<128, 8, 4, 8>: 222 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 window attention wattn_kernel<128, 8, 4, 8>: 222 registers, 0 static shared bytes, bound 256 (record current) +RECORD window attention wattn_kernel<128, 8, 4, 8>: .usage = .{ .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 222, .static = 0, .bound = 256 }, .{ .regs = 222, .static = 0, .bound = 256 }, } +PASS limits sm_80 window attention wattn_kernel<128, 8, 8, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 window attention wattn_kernel<128, 8, 8, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 window attention wattn_kernel<128, 8, 8, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 window attention wattn_kernel<128, 8, 8, 8>: 222 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 window attention wattn_kernel<128, 8, 8, 8>: 222 registers, 0 static shared bytes, bound 256 (record current) +RECORD window attention wattn_kernel<128, 8, 8, 8>: .usage = .{ .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 222, .static = 0, .bound = 256 }, .{ .regs = 222, .static = 0, .bound = 256 }, } +PASS limits sm_80 moe tf_moe_topk_sigmoid: 32 registers, 16704 static shared bytes, bound 256 (record current) +PASS limits sm_86 moe tf_moe_topk_sigmoid: 48 registers, 16704 static shared bytes, bound 256 (record current) +PASS limits sm_89 moe tf_moe_topk_sigmoid: 48 registers, 16704 static shared bytes, bound 256 (record current) +PASS limits sm_120 moe tf_moe_topk_sigmoid: 48 registers, 16704 static shared bytes, bound 256 (record current) +PASS limits sm_121 moe tf_moe_topk_sigmoid: 48 registers, 16704 static shared bytes, bound 256 (record current) +RECORD moe tf_moe_topk_sigmoid: .usage = .{ .{ .regs = 32, .static = 16704, .bound = 256 }, .{ .regs = 48, .static = 16704, .bound = 256 }, .{ .regs = 48, .static = 16704, .bound = 256 }, .{ .regs = 48, .static = 16704, .bound = 256 }, .{ .regs = 48, .static = 16704, .bound = 256 }, } +PASS limits sm_80 moe tf_moe_topk_sigmoid_warp: 63 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 moe tf_moe_topk_sigmoid_warp: 63 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 moe tf_moe_topk_sigmoid_warp: 63 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 moe tf_moe_topk_sigmoid_warp: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 moe tf_moe_topk_sigmoid_warp: 64 registers, 0 static shared bytes, bound 256 (record current) +RECORD moe tf_moe_topk_sigmoid_warp: .usage = .{ .{ .regs = 63, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 moe tf_moe_combine: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 moe tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 moe tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 moe tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 moe tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD moe tf_moe_combine: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 dense rows tf_dense_rows_1: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 dense rows tf_dense_rows_1: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 dense rows tf_dense_rows_1: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 dense rows tf_dense_rows_1: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 dense rows tf_dense_rows_1: 38 registers, 0 static shared bytes, bound 256 (record current) +RECORD dense rows tf_dense_rows_1: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, } +PASS limits sm_80 dense rows tf_dense_rows_4: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 dense rows tf_dense_rows_4: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 dense rows tf_dense_rows_4: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 dense rows tf_dense_rows_4: 45 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 dense rows tf_dense_rows_4: 45 registers, 0 static shared bytes, bound 256 (record current) +RECORD dense rows tf_dense_rows_4: .usage = .{ .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 45, .static = 0, .bound = 256 }, .{ .regs = 45, .static = 0, .bound = 256 }, } +PASS limits sm_80 dense rows tf_dense_rows_16: 79 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 dense rows tf_dense_rows_16: 79 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 dense rows tf_dense_rows_16: 79 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 dense rows tf_dense_rows_16: 98 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 dense rows tf_dense_rows_16: 98 registers, 0 static shared bytes, bound 256 (record current) +RECORD dense rows tf_dense_rows_16: .usage = .{ .{ .regs = 79, .static = 0, .bound = 256 }, .{ .regs = 79, .static = 0, .bound = 256 }, .{ .regs = 79, .static = 0, .bound = 256 }, .{ .regs = 98, .static = 0, .bound = 256 }, .{ .regs = 98, .static = 0, .bound = 256 }, } +PASS limits sm_80 dense rows tf_dense_rows_32: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 dense rows tf_dense_rows_32: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 dense rows tf_dense_rows_32: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 dense rows tf_dense_rows_32: 180 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 dense rows tf_dense_rows_32: 180 registers, 0 static shared bytes, bound 256 (record current) +RECORD dense rows tf_dense_rows_32: .usage = .{ .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 180, .static = 0, .bound = 256 }, .{ .regs = 180, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_geglu_rows 256 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 act_rows tf_geglu_rows 256 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 act_rows tf_geglu_rows 256 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 act_rows tf_geglu_rows 256 threads: 33 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 act_rows tf_geglu_rows 256 threads: 33 registers, 0 static shared bytes, bound 256 (record current) +RECORD act_rows tf_geglu_rows 256 threads: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_geglu_rows 128 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 act_rows tf_geglu_rows 128 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 act_rows tf_geglu_rows 128 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 act_rows tf_geglu_rows 128 threads: 33 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 act_rows tf_geglu_rows 128 threads: 33 registers, 0 static shared bytes, bound 256 (record current) +RECORD act_rows tf_geglu_rows 128 threads: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +RECORD act_rows tf_swiglu_rows_round 256 threads: .usage = .{ .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +RECORD act_rows tf_swiglu_rows_round 128 threads: .usage = .{ .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 embed_rows tf_embed_rows_affine: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 embed_rows tf_embed_rows_affine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 embed_rows tf_embed_rows_affine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 embed_rows tf_embed_rows_affine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 embed_rows tf_embed_rows_affine: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD embed_rows tf_embed_rows_affine: .usage = .{ .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 qk_rows tf_qk_rows_once 512 threads: 30 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_86 qk_rows tf_qk_rows_once 512 threads: 36 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_89 qk_rows tf_qk_rows_once 512 threads: 36 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_120 qk_rows tf_qk_rows_once 512 threads: 34 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_121 qk_rows tf_qk_rows_once 512 threads: 34 registers, 64 static shared bytes, bound 512 (record current) +RECORD qk_rows tf_qk_rows_once 512 threads: .usage = .{ .{ .regs = 30, .static = 64, .bound = 512 }, .{ .regs = 36, .static = 64, .bound = 512 }, .{ .regs = 36, .static = 64, .bound = 512 }, .{ .regs = 34, .static = 64, .bound = 512 }, .{ .regs = 34, .static = 64, .bound = 512 }, } +PASS limits sm_80 qk_rows tf_qk_rows_round 512 threads: 30 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_86 qk_rows tf_qk_rows_round 512 threads: 36 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_89 qk_rows tf_qk_rows_round 512 threads: 36 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_120 qk_rows tf_qk_rows_round 512 threads: 34 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_121 qk_rows tf_qk_rows_round 512 threads: 34 registers, 64 static shared bytes, bound 512 (record current) +RECORD qk_rows tf_qk_rows_round 512 threads: .usage = .{ .{ .regs = 30, .static = 64, .bound = 512 }, .{ .regs = 36, .static = 64, .bound = 512 }, .{ .regs = 36, .static = 64, .bound = 512 }, .{ .regs = 34, .static = 64, .bound = 512 }, .{ .regs = 34, .static = 64, .bound = 512 }, } +PASS limits sm_80 norms tf_add_rms_rows_bf16h_tail: 94 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_86 norms tf_add_rms_rows_bf16h_tail: 95 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_89 norms tf_add_rms_rows_bf16h_tail: 95 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_120 norms tf_add_rms_rows_bf16h_tail: 147 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_121 norms tf_add_rms_rows_bf16h_tail: 147 registers, 64 static shared bytes, bound 256 (record current) +RECORD norms tf_add_rms_rows_bf16h_tail: .usage = .{ .{ .regs = 94, .static = 64, .bound = 256 }, .{ .regs = 95, .static = 64, .bound = 256 }, .{ .regs = 95, .static = 64, .bound = 256 }, .{ .regs = 147, .static = 64, .bound = 256 }, .{ .regs = 147, .static = 64, .bound = 256 }, } +PASS limits sm_80 norms tf_add_rms_rows_bf16h_tail_round: 94 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_86 norms tf_add_rms_rows_bf16h_tail_round: 95 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_89 norms tf_add_rms_rows_bf16h_tail_round: 95 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_120 norms tf_add_rms_rows_bf16h_tail_round: 147 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_121 norms tf_add_rms_rows_bf16h_tail_round: 147 registers, 64 static shared bytes, bound 256 (record current) +RECORD norms tf_add_rms_rows_bf16h_tail_round: .usage = .{ .{ .regs = 94, .static = 64, .bound = 256 }, .{ .regs = 95, .static = 64, .bound = 256 }, .{ .regs = 95, .static = 64, .bound = 256 }, .{ .regs = 147, .static = 64, .bound = 256 }, .{ .regs = 147, .static = 64, .bound = 256 }, } +PASS limits sm_80 flashnext_fact factored prompt matmul 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_fact factored prompt matmul 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_fact factored prompt matmul 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_fact factored prompt matmul 64x128: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_fact factored prompt matmul 64x128: 127 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_fact factored prompt matmul 64x128: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_fact factored prompt matmul 64x128 folded: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_hcf hyper-connection down 32x192 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hcf hyper-connection down 32x192 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hcf hyper-connection down 32x192 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hcf hyper-connection down 32x192 folded: 126 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hcf hyper-connection down 32x192 folded: 126 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_hcf hyper-connection down 32x192 folded: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 126, .static = 0, .bound = 256 }, .{ .regs = 126, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_hcf hyper-connection up 64x128: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_fexp factored experts gate and up: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_fexp factored experts gate and up: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_fexp factored experts gate and up: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_fexp factored experts gate and up: 116 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_fexp factored experts gate and up: 116 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_fexp factored experts gate and up: .usage = .{ .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 116, .static = 0, .bound = 256 }, .{ .regs = 116, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_fexp factored experts down: 136 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_fexp factored experts down: 136 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_fexp factored experts down: 136 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_fexp factored experts down: 118 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_fexp factored experts down: 118 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_fexp factored experts down: .usage = .{ .{ .regs = 136, .static = 0, .bound = 256 }, .{ .regs = 136, .static = 0, .bound = 256 }, .{ .regs = 136, .static = 0, .bound = 256 }, .{ .regs = 118, .static = 0, .bound = 256 }, .{ .regs = 118, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_gdn_chain_tpr: 129 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_chain_tpr: 129 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_chain_tpr: 129 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_chain_tpr: 154 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_chain_tpr: 154 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_batch fn_gdn_chain_tpr: .usage = .{ .{ .regs = 129, .static = 0, .bound = 256 }, .{ .regs = 129, .static = 0, .bound = 256 }, .{ .regs = 129, .static = 0, .bound = 256 }, .{ .regs = 154, .static = 0, .bound = 256 }, .{ .regs = 154, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_attn qsa_tile scores: 215 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn qsa_tile scores: 216 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn qsa_tile scores: 216 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn qsa_tile scores: 224 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn qsa_tile scores: 224 registers, 16384 static shared bytes, bound 256 (record current) +RECORD flashnext_attn qsa_tile scores: .usage = .{ .{ .regs = 215, .static = 16384, .bound = 256 }, .{ .regs = 216, .static = 16384, .bound = 256 }, .{ .regs = 216, .static = 16384, .bound = 256 }, .{ .regs = 224, .static = 16384, .bound = 256 }, .{ .regs = 224, .static = 16384, .bound = 256 }, } +PASS limits sm_80 flashnext_attn fn_attn_fused: 118 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_attn_fused: 118 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_attn_fused: 118 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_attn_fused: 121 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_attn_fused: 121 registers, 640 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_attn_fused: .usage = .{ .{ .regs = 118, .static = 640, .bound = 128 }, .{ .regs = 118, .static = 640, .bound = 128 }, .{ .regs = 118, .static = 640, .bound = 128 }, .{ .regs = 121, .static = 640, .bound = 128 }, .{ .regs = 121, .static = 640, .bound = 128 }, } +PASS limits sm_80 flashnext_attn fn_attn_chunks_one: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_attn_chunks_one: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_attn_chunks_one: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_attn_chunks_one: 96 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_attn_chunks_one: 96 registers, 640 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_attn_chunks_one: .usage = .{ .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 96, .static = 640, .bound = 128 }, .{ .regs = 96, .static = 640, .bound = 128 }, } +PASS limits sm_80 flashnext_hc fn_hc_rinv: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_hc_rinv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_hc_rinv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_hc_rinv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_hc_rinv: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_hc_rinv: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_hc_mix_h: 27 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_mix_h: 27 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_mix_h: 27 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_mix_h: 40 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_mix_h: 40 registers, 1024 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_mix_h: .usage = .{ .{ .regs = 27, .static = 1024, .bound = 256 }, .{ .regs = 27, .static = 1024, .bound = 256 }, .{ .regs = 27, .static = 1024, .bound = 256 }, .{ .regs = 40, .static = 1024, .bound = 256 }, .{ .regs = 40, .static = 1024, .bound = 256 }, } +PASS limits sm_80 route_rows tf_route_logits_wide: 47 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 route_rows tf_route_logits_wide: 47 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 route_rows tf_route_logits_wide: 47 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 route_rows tf_route_logits_wide: 61 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 route_rows tf_route_logits_wide: 61 registers, 0 static shared bytes, bound 256 (record current) +RECORD route_rows tf_route_logits_wide: .usage = .{ .{ .regs = 47, .static = 0, .bound = 256 }, .{ .regs = 47, .static = 0, .bound = 256 }, .{ .regs = 47, .static = 0, .bound = 256 }, .{ .regs = 61, .static = 0, .bound = 256 }, .{ .regs = 61, .static = 0, .bound = 256 }, } +PASS limits sm_80 gdn_train tf_gdn_front threads 128: 32 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_86 gdn_train tf_gdn_front threads 128: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_89 gdn_train tf_gdn_front threads 128: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_120 gdn_train tf_gdn_front threads 128: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_121 gdn_train tf_gdn_front threads 128: 40 registers, 1024 static shared bytes, bound 128 (record current) +RECORD gdn_train tf_gdn_front threads 128: .usage = .{ .{ .regs = 32, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, } +PASS limits sm_80 gdn_train tf_gdn_scan_fwd threads 256: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 gdn_train tf_gdn_scan_fwd threads 256: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 gdn_train tf_gdn_scan_fwd threads 256: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 gdn_train tf_gdn_scan_fwd threads 256: 60 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 gdn_train tf_gdn_scan_fwd threads 256: 60 registers, 1152 static shared bytes, bound 256 (record current) +RECORD gdn_train tf_gdn_scan_fwd threads 256: .usage = .{ .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 60, .static = 1152, .bound = 256 }, .{ .regs = 60, .static = 1152, .bound = 256 }, } +PASS limits sm_80 gdn_train tf_gdn_norm_back threads 128: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_86 gdn_train tf_gdn_norm_back threads 128: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_89 gdn_train tf_gdn_norm_back threads 128: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_120 gdn_train tf_gdn_norm_back threads 128: 25 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_121 gdn_train tf_gdn_norm_back threads 128: 25 registers, 644 static shared bytes, bound 128 (record current) +RECORD gdn_train tf_gdn_norm_back threads 128: .usage = .{ .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 25, .static = 644, .bound = 128 }, .{ .regs = 25, .static = 644, .bound = 128 }, } +PASS limits sm_80 gdn_train tf_gdn_scan_back threads 256: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_86 gdn_train tf_gdn_scan_back threads 256: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_89 gdn_train tf_gdn_scan_back threads 256: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_120 gdn_train tf_gdn_scan_back threads 256: 100 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_121 gdn_train tf_gdn_scan_back threads 256: 100 registers, 9536 static shared bytes, bound 256 (record current) +RECORD gdn_train tf_gdn_scan_back threads 256: .usage = .{ .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 100, .static = 9536, .bound = 256 }, .{ .regs = 100, .static = 9536, .bound = 256 }, } +PASS limits sm_80 gdn_train tf_gdn_qk_back threads 128: 32 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_86 gdn_train tf_gdn_qk_back threads 128: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_89 gdn_train tf_gdn_qk_back threads 128: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_120 gdn_train tf_gdn_qk_back threads 128: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_121 gdn_train tf_gdn_qk_back threads 128: 40 registers, 128 static shared bytes, bound 128 (record current) +RECORD gdn_train tf_gdn_qk_back threads 128: .usage = .{ .{ .regs = 32, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, } +PASS limits sm_80 gdn_train tf_gdn_conv_back threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_conv_back threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_conv_back threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_conv_back threads 256: 34 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_conv_back threads 256: 35 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_conv_back threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_dequant threads 256: .usage = .{ .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 32: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 32: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 32: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 32: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 32: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 32: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 64: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 64: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 64: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 64: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 64: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 64: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 96: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 96: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 96: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 96: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 96: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 96: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 128: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 128: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 128: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 128: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 128: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 128: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 160: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 160: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 160: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 160: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 160: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 160: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 192: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 192: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 192: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 192: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 192: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 192: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 224: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 224: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 224: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 224: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 224: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 224: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 256: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 256: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 256: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 256: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 288: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 288: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 288: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 288: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 288: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 288: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 320: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 320: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 320: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 320: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 320: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 320: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 352: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 352: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 352: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 352: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 352: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 352: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 384: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 384: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 384: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 384: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 384: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 384: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 416: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 416: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 416: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 416: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 416: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 416: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 448: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 448: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 448: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 448: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 448: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 448: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 480: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 480: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 480: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 480: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 480: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 480: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 512: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 512: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 512: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 512: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 512: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 512: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 544: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 544: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 544: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 544: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 544: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 544: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 576: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 576: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 576: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 576: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 576: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 576: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 608: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 608: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 608: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 608: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 608: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 608: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 640: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 640: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 640: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 640: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 640: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 640: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 672: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 672: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 672: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 672: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 672: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 672: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 704: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 704: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 704: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 704: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 704: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 704: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 736: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 736: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 736: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 736: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 736: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 736: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 768: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 768: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 768: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 768: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 768: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 768: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 800: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 800: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 800: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 800: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 800: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 800: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 832: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 832: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 832: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 832: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 832: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 832: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 864: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 864: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 864: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 864: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 864: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 864: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 896: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 896: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 896: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 896: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 896: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 896: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 928: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 928: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 928: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 928: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 928: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 928: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 960: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 960: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 960: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 960: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 960: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 960: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 992: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 992: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 992: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 992: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 992: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 992: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 1024: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 1024: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 1024: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 1024: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 1024: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 1024: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_q4 tf_q4_back threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 train_q4 tf_q4_back threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 train_q4 tf_q4_back threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 train_q4 tf_q4_back threads 128: 60 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 train_q4 tf_q4_back threads 128: 60 registers, 10240 static shared bytes, bound 128 (record current) +RECORD train_q4 tf_q4_back threads 128: .usage = .{ .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 60, .static = 10240, .bound = 128 }, .{ .regs = 60, .static = 10240, .bound = 128 }, } +PASS limits sm_80 train_q4 tf_q4_back_bf threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 train_q4 tf_q4_back_bf threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 train_q4 tf_q4_back_bf threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 train_q4 tf_q4_back_bf threads 128: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 train_q4 tf_q4_back_bf threads 128: 56 registers, 10240 static shared bytes, bound 128 (record current) +RECORD train_q4 tf_q4_back_bf threads 128: .usage = .{ .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, } +PASS limits sm_80 train_q4 tf_q4_sum_parts threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train_q4 tf_q4_sum_parts threads 256: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train_q4 tf_q4_sum_parts threads 256: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train_q4 tf_q4_sum_parts threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train_q4 tf_q4_sum_parts threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train_q4 tf_q4_sum_parts threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_q4 tf_q4_experts_mm threads 64: 72 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_86 train_q4 tf_q4_experts_mm threads 64: 90 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_89 train_q4 tf_q4_experts_mm threads 64: 90 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_120 train_q4 tf_q4_experts_mm threads 64: 79 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_121 train_q4 tf_q4_experts_mm threads 64: 79 registers, 10304 static shared bytes, bound 64 (record current) +RECORD train_q4 tf_q4_experts_mm threads 64: .usage = .{ .{ .regs = 72, .static = 10304, .bound = 64 }, .{ .regs = 90, .static = 10304, .bound = 64 }, .{ .regs = 90, .static = 10304, .bound = 64 }, .{ .regs = 79, .static = 10304, .bound = 64 }, .{ .regs = 79, .static = 10304, .bound = 64 }, } +PASS limits sm_80 train_q4 tf_q4_experts_nt threads 64: 72 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_86 train_q4 tf_q4_experts_nt threads 64: 91 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_89 train_q4 tf_q4_experts_nt threads 64: 91 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_120 train_q4 tf_q4_experts_nt threads 64: 76 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_121 train_q4 tf_q4_experts_nt threads 64: 76 registers, 10816 static shared bytes, bound 64 (record current) +RECORD train_q4 tf_q4_experts_nt threads 64: .usage = .{ .{ .regs = 72, .static = 10816, .bound = 64 }, .{ .regs = 91, .static = 10816, .bound = 64 }, .{ .regs = 91, .static = 10816, .bound = 64 }, .{ .regs = 76, .static = 10816, .bound = 64 }, .{ .regs = 76, .static = 10816, .bound = 64 }, } +PASS limits sm_80 train_q4 tf_q4_swiglu_back threads 128: 40 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_86 train_q4 tf_q4_swiglu_back threads 128: 40 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_89 train_q4 tf_q4_swiglu_back threads 128: 40 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_120 train_q4 tf_q4_swiglu_back threads 128: 44 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_121 train_q4 tf_q4_swiglu_back threads 128: 44 registers, 128 static shared bytes, bound 128 (record current) +RECORD train_q4 tf_q4_swiglu_back threads 128: .usage = .{ .{ .regs = 40, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, .{ .regs = 44, .static = 128, .bound = 128 }, .{ .regs = 44, .static = 128, .bound = 128 }, } +PASS limits sm_80 train_q4 tf_q4_experts_nt_packed threads 64: 72 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_86 train_q4 tf_q4_experts_nt_packed threads 64: 83 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_89 train_q4 tf_q4_experts_nt_packed threads 64: 83 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_120 train_q4 tf_q4_experts_nt_packed threads 64: 68 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_121 train_q4 tf_q4_experts_nt_packed threads 64: 68 registers, 10816 static shared bytes, bound 64 (record current) +RECORD train_q4 tf_q4_experts_nt_packed threads 64: .usage = .{ .{ .regs = 72, .static = 10816, .bound = 64 }, .{ .regs = 83, .static = 10816, .bound = 64 }, .{ .regs = 83, .static = 10816, .bound = 64 }, .{ .regs = 68, .static = 10816, .bound = 64 }, .{ .regs = 68, .static = 10816, .bound = 64 }, } +PASS limits sm_80 train_q4 tf_q4_experts_mm_packed threads 64: 72 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_86 train_q4 tf_q4_experts_mm_packed threads 64: 81 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_89 train_q4 tf_q4_experts_mm_packed threads 64: 81 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_120 train_q4 tf_q4_experts_mm_packed threads 64: 76 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_121 train_q4 tf_q4_experts_mm_packed threads 64: 76 registers, 10304 static shared bytes, bound 64 (record current) +RECORD train_q4 tf_q4_experts_mm_packed threads 64: .usage = .{ .{ .regs = 72, .static = 10304, .bound = 64 }, .{ .regs = 81, .static = 10304, .bound = 64 }, .{ .regs = 81, .static = 10304, .bound = 64 }, .{ .regs = 76, .static = 10304, .bound = 64 }, .{ .regs = 76, .static = 10304, .bound = 64 }, } +PASS limits sm_80 train_q4 tf_q4_route_back threads 256: 32 registers, 136 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_q4 tf_q4_route_back threads 256: 40 registers, 136 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_q4 tf_q4_route_back threads 256: 40 registers, 136 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_q4 tf_q4_route_back threads 256: 40 registers, 136 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_q4 tf_q4_route_back threads 256: 40 registers, 136 static shared bytes, bound 256 (record current) +RECORD train_q4 tf_q4_route_back threads 256: .usage = .{ .{ .regs = 32, .static = 136, .bound = 256 }, .{ .regs = 40, .static = 136, .bound = 256 }, .{ .regs = 40, .static = 136, .bound = 256 }, .{ .regs = 40, .static = 136, .bound = 256 }, .{ .regs = 40, .static = 136, .bound = 256 }, } +PASS limits sm_80 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD train_q4 tf_q4_axpy threads 256: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_q threads 256: 32 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_q threads 256: 47 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_q threads 256: 47 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_q threads 256: 48 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_q threads 256: 48 registers, 18560 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_q threads 256: .usage = .{ .{ .regs = 32, .static = 18560, .bound = 256 }, .{ .regs = 47, .static = 18560, .bound = 256 }, .{ .regs = 47, .static = 18560, .bound = 256 }, .{ .regs = 48, .static = 18560, .bound = 256 }, .{ .regs = 48, .static = 18560, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +RECORD train_attn tf_gattn_gate_back threads 256: .usage = .{ .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 32: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 32: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 32: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 32: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 32: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 32: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 32: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 32: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 32: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 32: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 32: 30 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 32: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 32: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 32: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 32: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 32: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 32: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 32: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 64: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 64: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 64: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 64: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 64: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 64: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 64: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 64: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 64: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 64: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 64: 30 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 64: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 64: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 64: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 64: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 64: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 64: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 64: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 96: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 96: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 96: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 96: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 96: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 96: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 96: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 96: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 96: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 96: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 96: 30 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 96: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 96: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 96: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 96: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 96: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 96: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 96: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 128: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 128: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 128: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 128: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 128: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 128: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 128: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 128: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 128: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 128: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 128: 30 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 128: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 128: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 128: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 128: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 128: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 128: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 128: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 160: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 160: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 160: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 160: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 160: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 160: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 160: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 160: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 160: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 160: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 160: 30 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 160: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 160: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 160: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 160: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 160: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 160: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 160: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 192: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 192: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 192: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 192: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 192: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 192: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 192: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 192: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 192: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 192: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 192: 30 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 192: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 192: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 192: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 192: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 192: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 192: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 192: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 224: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 224: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 224: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 224: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 224: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 224: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 224: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 224: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 224: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 224: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 224: 30 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 224: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 224: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 224: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 224: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 224: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 224: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 224: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 256: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 256: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 256: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 256: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 256: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 256: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 256: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 256: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 256: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 256: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 256: 30 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 256: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_softmax threads 1024: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_86 train tf_train_softmax threads 1024: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_89 train tf_train_softmax threads 1024: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_120 train tf_train_softmax threads 1024: 42 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_121 train tf_train_softmax threads 1024: 42 registers, 128 static shared bytes, bound 1024 (record current) +RECORD train tf_train_softmax threads 1024: .usage = .{ .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, } +PASS limits sm_80 train tf_train_logits threads 256: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 train tf_train_logits threads 256: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 train tf_train_logits threads 256: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 train tf_train_logits threads 256: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 train tf_train_logits threads 256: 34 registers, 0 static shared bytes, bound 256 (record current) +RECORD train tf_train_logits threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 train tf_train_sort_pairs threads 256: 32 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_86 train tf_train_sort_pairs threads 256: 40 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_89 train tf_train_sort_pairs threads 256: 40 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_120 train tf_train_sort_pairs threads 256: 39 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_121 train tf_train_sort_pairs threads 256: 39 registers, 4096 static shared bytes, bound 256 (record current) +RECORD train tf_train_sort_pairs threads 256: .usage = .{ .{ .regs = 32, .static = 4096, .bound = 256 }, .{ .regs = 40, .static = 4096, .bound = 256 }, .{ .regs = 40, .static = 4096, .bound = 256 }, .{ .regs = 39, .static = 4096, .bound = 256 }, .{ .regs = 39, .static = 4096, .bound = 256 }, } +PASS limits sm_80 train tf_train_experts_tiles threads 1: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_experts_tiles threads 1: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_experts_tiles threads 1: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_experts_tiles threads 1: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_experts_tiles threads 1: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_experts_tiles threads 1: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_pairs_out threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_pairs_out threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_pairs_out threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_pairs_out threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_pairs_out threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_pairs_out threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_zero threads 256: .usage = .{ .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_narrow threads 256: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_gemm threads 128: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 train tf_train_gemm threads 128: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 train tf_train_gemm threads 128: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 train tf_train_gemm threads 128: 64 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 train tf_train_gemm threads 128: 64 registers, 10240 static shared bytes, bound 128 (record current) +RECORD train tf_train_gemm threads 128: .usage = .{ .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 64, .static = 10240, .bound = 128 }, .{ .regs = 64, .static = 10240, .bound = 128 }, } +PASS limits sm_80 flashnext_train tf_fn_hc_branch_back threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_hc_branch_back threads 256: 48 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_hc_branch_back threads 256: 48 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_hc_branch_back threads 256: 48 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_hc_branch_back threads 256: 48 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_hc_branch_back threads 256: .usage = .{ .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 48, .static = 128, .bound = 256 }, .{ .regs = 48, .static = 128, .bound = 256 }, .{ .regs = 48, .static = 128, .bound = 256 }, .{ .regs = 48, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_hc_mix_back threads 256: 29 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_train tf_fn_hc_mix_back threads 256: 37 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_train tf_fn_hc_mix_back threads 256: 37 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_train tf_fn_hc_mix_back threads 256: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_train tf_fn_hc_mix_back threads 256: 30 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_train tf_fn_hc_mix_back threads 256: .usage = .{ .{ .regs = 29, .static = 0, .bound = 0 }, .{ .regs = 37, .static = 0, .bound = 0 }, .{ .regs = 37, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_train tf_fn_hc_act_back threads 256: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_hc_act_back threads 256: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_hc_act_back threads 256: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_hc_act_back threads 256: 29 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_hc_act_back threads 256: 29 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_hc_act_back threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 29, .static = 0, .bound = 256 }, .{ .regs = 29, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_hc_norm_back threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_hc_norm_back threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_hc_norm_back threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_hc_norm_back threads 256: 37 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_hc_norm_back threads 256: 37 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_hc_norm_back threads 256: .usage = .{ .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 37, .static = 128, .bound = 256 }, .{ .regs = 37, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_in threads 128: 32 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_in threads 128: 33 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_in threads 128: 33 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_in threads 128: 31 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_in threads 128: 31 registers, 0 static shared bytes, bound 128 (record current) +RECORD flashnext_train tf_fn_pocket_in threads 128: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 33, .static = 0, .bound = 128 }, .{ .regs = 33, .static = 0, .bound = 128 }, .{ .regs = 31, .static = 0, .bound = 128 }, .{ .regs = 31, .static = 0, .bound = 128 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_pocket_in_rows threads 256: .usage = .{ .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_out threads 128: 48 registers, 512 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_out threads 128: 48 registers, 512 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_out threads 128: 48 registers, 512 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_out threads 128: 54 registers, 512 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_out threads 128: 54 registers, 512 static shared bytes, bound 128 (record current) +RECORD flashnext_train tf_fn_pocket_out threads 128: .usage = .{ .{ .regs = 48, .static = 512, .bound = 128 }, .{ .regs = 48, .static = 512, .bound = 128 }, .{ .regs = 48, .static = 512, .bound = 128 }, .{ .regs = 54, .static = 512, .bound = 128 }, .{ .regs = 54, .static = 512, .bound = 128 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_out_rows threads 128: 72 registers, 2048 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_out_rows threads 128: 72 registers, 2048 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_out_rows threads 128: 72 registers, 2048 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_out_rows threads 128: 64 registers, 2048 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_out_rows threads 128: 64 registers, 2048 static shared bytes, bound 128 (record current) +RECORD flashnext_train tf_fn_pocket_out_rows threads 128: .usage = .{ .{ .regs = 72, .static = 2048, .bound = 128 }, .{ .regs = 72, .static = 2048, .bound = 128 }, .{ .regs = 72, .static = 2048, .bound = 128 }, .{ .regs = 64, .static = 2048, .bound = 128 }, .{ .regs = 64, .static = 2048, .bound = 128 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_back threads 256: 42 registers, 1536 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_back threads 256: 40 registers, 1536 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_back threads 256: 40 registers, 1536 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_back threads 256: 40 registers, 1536 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_back threads 256: 40 registers, 1536 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_pocket_back threads 256: .usage = .{ .{ .regs = 42, .static = 1536, .bound = 256 }, .{ .regs = 40, .static = 1536, .bound = 256 }, .{ .regs = 40, .static = 1536, .bound = 256 }, .{ .regs = 40, .static = 1536, .bound = 256 }, .{ .regs = 40, .static = 1536, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_grad threads 256: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_grad threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_grad threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_grad threads 256: 39 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_grad threads 256: 39 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_pocket_grad threads 256: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 39, .static = 128, .bound = 256 }, .{ .regs = 39, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_moe_sum threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_batch fn_moe_sum threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_batch fn_moe_sum threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_batch fn_moe_sum threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_batch fn_moe_sum threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_batch fn_moe_sum threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_hc_upmix_up threads 256: 69 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_upmix_up threads 256: 64 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_upmix_up threads 256: 64 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_upmix_up threads 256: 78 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_upmix_up threads 256: 78 registers, 2048 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_upmix_up threads 256: .usage = .{ .{ .regs = 69, .static = 2048, .bound = 256 }, .{ .regs = 64, .static = 2048, .bound = 256 }, .{ .regs = 64, .static = 2048, .bound = 256 }, .{ .regs = 78, .static = 2048, .bound = 256 }, .{ .regs = 78, .static = 2048, .bound = 256 }, } +PASS limits sm_80 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_train_zero: .usage = .{ .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_gemm_q4w_bf: .usage = .{ .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_gemm_q4w: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_gemm_q4w: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_gemm_q4w: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_gemm_q4w: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_gemm_q4w: 128 registers, 37888 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_gemm_q4w: .usage = .{ .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_gemm_q4_bf: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_q27_gemm_q4_bf: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_q27_gemm_q4_bf: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_q27_gemm_q4_bf: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_q27_gemm_q4_bf: 56 registers, 10240 static shared bytes, bound 128 (record current) +RECORD 27B tf_q27_gemm_q4_bf: .usage = .{ .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, } +PASS limits sm_80 27B tf_q27_gemm_q4: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_q27_gemm_q4: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_q27_gemm_q4: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_q27_gemm_q4: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_q27_gemm_q4: 56 registers, 10240 static shared bytes, bound 128 (record current) +RECORD 27B tf_q27_gemm_q4: .usage = .{ .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, } +PASS limits sm_80 27B tf_sum_parts: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_sum_parts: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_sum_parts: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_sum_parts: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_sum_parts: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_sum_parts: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_fit_step: 40 registers, 5120 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_fit_step: 40 registers, 5120 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_fit_step: 40 registers, 5120 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_fit_step: 39 registers, 5120 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_fit_step: 39 registers, 5120 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_fit_step: .usage = .{ .{ .regs = 40, .static = 5120, .bound = 256 }, .{ .regs = 40, .static = 5120, .bound = 256 }, .{ .regs = 40, .static = 5120, .bound = 256 }, .{ .regs = 39, .static = 5120, .bound = 256 }, .{ .regs = 39, .static = 5120, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_zero_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_q27_zero_rows: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_q27_zero_rows: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_q27_zero_rows: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_q27_zero_rows: 13 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_q27_zero_rows: .usage = .{ .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_train_softmax: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_86 27B tf_train_softmax: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_89 27B tf_train_softmax: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_120 27B tf_train_softmax: 42 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_121 27B tf_train_softmax: 42 registers, 128 static shared bytes, bound 1024 (record current) +RECORD 27B tf_train_softmax: .usage = .{ .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, } +PASS limits sm_80 27B tf_train_rms_back: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_train_rms_back: 39 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_train_rms_back: 39 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_train_rms_back: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_train_rms_back: 40 registers, 128 static shared bytes, bound 256 (record current) +RECORD 27B tf_train_rms_back: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 39, .static = 128, .bound = 256 }, .{ .regs = 39, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, } +PASS limits sm_80 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_axpy: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_add_bf16: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_q27_add_bf16: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_q27_add_bf16: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_q27_add_bf16: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_q27_add_bf16: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_q27_add_bf16: .usage = .{ .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_q27_cols: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_pocket_rows: 34 registers, 512 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_pocket_rows: 40 registers, 512 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_pocket_rows: 40 registers, 512 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_pocket_rows: 40 registers, 512 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_pocket_rows: 40 registers, 512 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_pocket_rows: .usage = .{ .{ .regs = 34, .static = 512, .bound = 256 }, .{ .regs = 40, .static = 512, .bound = 256 }, .{ .regs = 40, .static = 512, .bound = 256 }, .{ .regs = 40, .static = 512, .bound = 256 }, .{ .regs = 40, .static = 512, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_pocket_back: 44 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_pocket_back: 42 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_pocket_back: 42 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_pocket_back: 40 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_pocket_back: 40 registers, 1024 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_pocket_back: .usage = .{ .{ .regs = 44, .static = 1024, .bound = 256 }, .{ .regs = 42, .static = 1024, .bound = 256 }, .{ .regs = 42, .static = 1024, .bound = 256 }, .{ .regs = 40, .static = 1024, .bound = 256 }, .{ .regs = 40, .static = 1024, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_pocket_grad: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_pocket_grad: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_pocket_grad: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_pocket_grad: 38 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_pocket_grad: 38 registers, 128 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_pocket_grad: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 38, .static = 128, .bound = 256 }, .{ .regs = 38, .static = 128, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_swiglu_back: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_q27_swiglu_back: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_q27_swiglu_back: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_q27_swiglu_back: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_q27_swiglu_back: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_q27_swiglu_back: .usage = .{ .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_attn_prep: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_attn_prep: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_attn_prep: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_attn_prep: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_attn_prep: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD 27B tf_attn_prep: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_attn_gate_back: .usage = .{ .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_attn_q: 32 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_attn_q: 47 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_attn_q: 47 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_attn_q: 48 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_attn_q: 48 registers, 18560 static shared bytes, bound 256 (record current) +RECORD 27B tf_attn_q: .usage = .{ .{ .regs = 32, .static = 18560, .bound = 256 }, .{ .regs = 47, .static = 18560, .bound = 256 }, .{ .regs = 47, .static = 18560, .bound = 256 }, .{ .regs = 48, .static = 18560, .bound = 256 }, .{ .regs = 48, .static = 18560, .bound = 256 }, } +PASS limits sm_80 27B tf_train_attn_kv: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_train_attn_kv: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_train_attn_kv: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_train_attn_kv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_train_attn_kv: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_train_attn_kv: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_attn_prep_back: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_attn_prep_back: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_attn_prep_back: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_attn_prep_back: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_attn_prep_back: 30 registers, 1152 static shared bytes, bound 256 (record current) +RECORD 27B tf_attn_prep_back: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 30, .static = 1152, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_fit_dots: 31 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_fit_dots: 39 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_fit_dots: 39 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_fit_dots: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_fit_dots: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_fit_dots: .usage = .{ .{ .regs = 31, .static = 0, .bound = 256 }, .{ .regs = 39, .static = 0, .bound = 256 }, .{ .regs = 39, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_fit_dots_tree: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_fit_dots_tree: 39 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_fit_dots_tree: 39 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_fit_dots_tree: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_fit_dots_tree: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_fit_dots_tree: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 39, .static = 0, .bound = 256 }, .{ .regs = 39, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 27B tf_gdn_front: 32 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_gdn_front: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_gdn_front: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_gdn_front: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_gdn_front: 40 registers, 1024 static shared bytes, bound 128 (record current) +RECORD 27B tf_gdn_front: .usage = .{ .{ .regs = 32, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, } +PASS limits sm_80 27B tf_gdn_scan_fwd: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_gdn_scan_fwd: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_gdn_scan_fwd: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_gdn_scan_fwd: 60 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_gdn_scan_fwd: 60 registers, 1152 static shared bytes, bound 256 (record current) +RECORD 27B tf_gdn_scan_fwd: .usage = .{ .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 60, .static = 1152, .bound = 256 }, .{ .regs = 60, .static = 1152, .bound = 256 }, } +PASS limits sm_80 27B tf_gdn_norm_back: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_gdn_norm_back: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_gdn_norm_back: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_gdn_norm_back: 25 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_gdn_norm_back: 25 registers, 644 static shared bytes, bound 128 (record current) +RECORD 27B tf_gdn_norm_back: .usage = .{ .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 25, .static = 644, .bound = 128 }, .{ .regs = 25, .static = 644, .bound = 128 }, } +PASS limits sm_80 27B tf_gdn_scan_back: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_gdn_scan_back: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_gdn_scan_back: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_gdn_scan_back: 100 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_gdn_scan_back: 100 registers, 9536 static shared bytes, bound 256 (record current) +RECORD 27B tf_gdn_scan_back: .usage = .{ .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 100, .static = 9536, .bound = 256 }, .{ .regs = 100, .static = 9536, .bound = 256 }, } +PASS limits sm_80 27B tf_gdn_qk_back: 32 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_gdn_qk_back: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_gdn_qk_back: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_gdn_qk_back: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_gdn_qk_back: 40 registers, 128 static shared bytes, bound 128 (record current) +RECORD 27B tf_gdn_qk_back: .usage = .{ .{ .regs = 32, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, } +PASS limits sm_80 27B tf_gdn_gates_back: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_gdn_gates_back: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_gdn_gates_back: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_gdn_gates_back: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_gdn_gates_back: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_gdn_gates_back: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_gdn_conv_back: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_gdn_conv_back: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_gdn_conv_back: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_gdn_conv_back: 34 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_gdn_conv_back: 35 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_gdn_conv_back: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_gdn_dequant: .usage = .{ .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_train_narrow: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_train_gemm: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_train_gemm: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_train_gemm: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_train_gemm: 64 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_train_gemm: 64 registers, 10240 static shared bytes, bound 128 (record current) +RECORD 27B tf_train_gemm: .usage = .{ .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 64, .static = 10240, .bound = 128 }, .{ .regs = 64, .static = 10240, .bound = 128 }, } +PASS limits sm_80 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD paging tf_nemo_kv_write: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 paging tf_nemo_kv_write_segs: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 paging tf_nemo_kv_write_segs: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 paging tf_nemo_kv_write_segs: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 paging tf_nemo_kv_write_segs: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 paging tf_nemo_kv_write_segs: 24 registers, 0 static shared bytes, bound 0 (record current) +RECORD paging tf_nemo_kv_write_segs: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, } +PASS limits sm_80 paging tf_nemo_attn_chunk: 76 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_86 paging tf_nemo_attn_chunk: 91 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_89 paging tf_nemo_attn_chunk: 91 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_120 paging tf_nemo_attn_chunk: 118 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_121 paging tf_nemo_attn_chunk: 118 registers, 45568 static shared bytes, bound 256 (record current) +RECORD paging tf_nemo_attn_chunk: .usage = .{ .{ .regs = 76, .static = 45568, .bound = 256 }, .{ .regs = 91, .static = 45568, .bound = 256 }, .{ .regs = 91, .static = 45568, .bound = 256 }, .{ .regs = 118, .static = 45568, .bound = 256 }, .{ .regs = 118, .static = 45568, .bound = 256 }, } +PASS limits sm_80 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_86 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_89 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_120 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_121 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +RECORD paging tf_nemo_attn_chunk_pf: .usage = .{ .{ .regs = 128, .static = 45568, .bound = 256 }, .{ .regs = 128, .static = 45568, .bound = 256 }, .{ .regs = 128, .static = 45568, .bound = 256 }, .{ .regs = 128, .static = 45568, .bound = 256 }, .{ .regs = 128, .static = 45568, .bound = 256 }, } +PASS limits sm_80 paging tf_nemo_attn_chunk_segs: 78 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_86 paging tf_nemo_attn_chunk_segs: 97 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_89 paging tf_nemo_attn_chunk_segs: 97 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_120 paging tf_nemo_attn_chunk_segs: 120 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_121 paging tf_nemo_attn_chunk_segs: 120 registers, 45568 static shared bytes, bound 256 (record current) +RECORD paging tf_nemo_attn_chunk_segs: .usage = .{ .{ .regs = 78, .static = 45568, .bound = 256 }, .{ .regs = 97, .static = 45568, .bound = 256 }, .{ .regs = 97, .static = 45568, .bound = 256 }, .{ .regs = 120, .static = 45568, .bound = 256 }, .{ .regs = 120, .static = 45568, .bound = 256 }, } +PASS limits sm_80 paging pattn128: 206 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 paging pattn128: 206 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 paging pattn128: 206 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 paging pattn128: 231 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 paging pattn128: 231 registers, 0 static shared bytes, bound 256 (record current) +RECORD paging pattn128: .usage = .{ .{ .regs = 206, .static = 0, .bound = 256 }, .{ .regs = 206, .static = 0, .bound = 256 }, .{ .regs = 206, .static = 0, .bound = 256 }, .{ .regs = 231, .static = 0, .bound = 256 }, .{ .regs = 231, .static = 0, .bound = 256 }, } +PASS limits sm_80 tf_moe_router: 32 registers, 16512 static shared bytes, bound 256 (record current) +PASS limits sm_86 tf_moe_router: 48 registers, 16512 static shared bytes, bound 256 (record current) +PASS limits sm_89 tf_moe_router: 48 registers, 16512 static shared bytes, bound 256 (record current) +PASS limits sm_120 tf_moe_router: 48 registers, 16512 static shared bytes, bound 256 (record current) +PASS limits sm_121 tf_moe_router: 48 registers, 16512 static shared bytes, bound 256 (record current) +RECORD tf_moe_router: .usage = .{ .{ .regs = 32, .static = 16512, .bound = 256 }, .{ .regs = 48, .static = 16512, .bound = 256 }, .{ .regs = 48, .static = 16512, .bound = 256 }, .{ .regs = 48, .static = 16512, .bound = 256 }, .{ .regs = 48, .static = 16512, .bound = 256 }, } +PASS limits sm_80 tf_moe_topk: 35 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tf_moe_topk: 35 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tf_moe_topk: 35 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tf_moe_topk: 34 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tf_moe_topk: 34 registers, 0 static shared bytes, bound 32 (record current) +RECORD tf_moe_topk: .usage = .{ .{ .regs = 35, .static = 0, .bound = 32 }, .{ .regs = 35, .static = 0, .bound = 32 }, .{ .regs = 35, .static = 0, .bound = 32 }, .{ .regs = 34, .static = 0, .bound = 32 }, .{ .regs = 34, .static = 0, .bound = 32 }, } +PASS limits sm_80 tf_moe_combine: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 tf_moe_combine: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 tf_moe_combine: 38 registers, 0 static shared bytes, bound 256 (record current) +RECORD tf_moe_combine: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, } +PASS limits sm_80 experts plan_rows: 32 registers, 12672 static shared bytes, bound 1024 (record current) +PASS limits sm_86 experts plan_rows: 38 registers, 12672 static shared bytes, bound 1024 (record current) +PASS limits sm_89 experts plan_rows: 38 registers, 12672 static shared bytes, bound 1024 (record current) +PASS limits sm_120 experts plan_rows: 32 registers, 12672 static shared bytes, bound 1024 (record current) +PASS limits sm_121 experts plan_rows: 32 registers, 12672 static shared bytes, bound 1024 (record current) +RECORD experts plan_rows: .usage = .{ .{ .regs = 32, .static = 12672, .bound = 1024 }, .{ .regs = 38, .static = 12672, .bound = 1024 }, .{ .regs = 38, .static = 12672, .bound = 1024 }, .{ .regs = 32, .static = 12672, .bound = 1024 }, .{ .regs = 32, .static = 12672, .bound = 1024 }, } +PASS limits sm_80 experts plan_rank_rows: 32 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_86 experts plan_rank_rows: 40 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_89 experts plan_rank_rows: 40 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_120 experts plan_rank_rows: 42 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_121 experts plan_rank_rows: 42 registers, 4096 static shared bytes, bound 1024 (record current) +RECORD experts plan_rank_rows: .usage = .{ .{ .regs = 32, .static = 4096, .bound = 1024 }, .{ .regs = 40, .static = 4096, .bound = 1024 }, .{ .regs = 40, .static = 4096, .bound = 1024 }, .{ .regs = 42, .static = 4096, .bound = 1024 }, .{ .regs = 42, .static = 4096, .bound = 1024 }, } +PASS limits sm_80 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD experts plan_scatter_rows: .usage = .{ .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 tf_q36m_norm_rope: 36 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tf_q36m_norm_rope: 37 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tf_q36m_norm_rope: 39 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tf_q36m_norm_rope: 42 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tf_q36m_norm_rope: 42 registers, 0 static shared bytes, bound 32 (record current) +RECORD tf_q36m_norm_rope: .usage = .{ .{ .regs = 36, .static = 0, .bound = 32 }, .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 39, .static = 0, .bound = 32 }, .{ .regs = 42, .static = 0, .bound = 32 }, .{ .regs = 42, .static = 0, .bound = 32 }, } +PASS limits sm_80 tf_q36m_attn: 50 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tf_q36m_attn: 52 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tf_q36m_attn: 51 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tf_q36m_attn: 54 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tf_q36m_attn: 54 registers, 0 static shared bytes, bound 32 (record current) +RECORD tf_q36m_attn: .usage = .{ .{ .regs = 50, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 51, .static = 0, .bound = 32 }, .{ .regs = 54, .static = 0, .bound = 32 }, .{ .regs = 54, .static = 0, .bound = 32 }, } +PASS limits sm_80 tf_q36m_attn_tree: 53 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tf_q36m_attn_tree: 54 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tf_q36m_attn_tree: 52 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tf_q36m_attn_tree: 57 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tf_q36m_attn_tree: 57 registers, 0 static shared bytes, bound 32 (record current) +RECORD tf_q36m_attn_tree: .usage = .{ .{ .regs = 53, .static = 0, .bound = 32 }, .{ .regs = 54, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 57, .static = 0, .bound = 32 }, .{ .regs = 57, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_1: 56 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_1: 55 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_1: 55 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_1: 48 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_1: 48 registers, 0 static shared bytes, bound 128 (record stale) +RECORD qwen35 tf_q35_gdn_slots_1: .usage = .{ .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 55, .static = 0, .bound = 128 }, .{ .regs = 55, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_2: 64 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_2: 64 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_2: 64 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_2: 48 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_2: 48 registers, 0 static shared bytes, bound 128 (record stale) +RECORD qwen35 tf_q35_gdn_slots_2: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_4: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_4: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_4: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_4: 56 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_4: 56 registers, 0 static shared bytes, bound 128 (record stale) +RECORD qwen35 tf_q35_gdn_slots_4: .usage = .{ .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_8: 96 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_8: 96 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_8: 96 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_8: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_8: 72 registers, 0 static shared bytes, bound 128 (record stale) +RECORD qwen35 tf_q35_gdn_slots_8: .usage = .{ .{ .regs = 96, .static = 0, .bound = 128 }, .{ .regs = 96, .static = 0, .bound = 128 }, .{ .regs = 96, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_16: 150 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_16: 149 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_16: 149 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_16: 108 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_16: 108 registers, 0 static shared bytes, bound 128 (record stale) +RECORD qwen35 tf_q35_gdn_slots_16: .usage = .{ .{ .regs = 150, .static = 0, .bound = 128 }, .{ .regs = 149, .static = 0, .bound = 128 }, .{ .regs = 149, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_chain: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 qwen35 tf_q35_gdn_chain: 64 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 qwen35 tf_q35_gdn_chain: 64 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 qwen35 tf_q35_gdn_chain: 56 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 qwen35 tf_q35_gdn_chain: 56 registers, 0 static shared bytes, bound 128 (record stale) +RECORD qwen35 tf_q35_gdn_chain: .usage = .{ .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_rope: 32 registers, 16384 static shared bytes, bound 256 (record stale) +PASS limits sm_86 qwen35 tf_q35_rope: 47 registers, 16384 static shared bytes, bound 256 (record stale) +PASS limits sm_89 qwen35 tf_q35_rope: 47 registers, 16384 static shared bytes, bound 256 (record stale) +PASS limits sm_120 qwen35 tf_q35_rope: 34 registers, 16384 static shared bytes, bound 256 (record stale) +PASS limits sm_121 qwen35 tf_q35_rope: 34 registers, 16384 static shared bytes, bound 256 (record stale) +RECORD qwen35 tf_q35_rope: .usage = .{ .{ .regs = 32, .static = 16384, .bound = 256 }, .{ .regs = 47, .static = 16384, .bound = 256 }, .{ .regs = 47, .static = 16384, .bound = 256 }, .{ .regs = 34, .static = 16384, .bound = 256 }, .{ .regs = 34, .static = 16384, .bound = 256 }, } +PASS limits sm_80 qwen35 tf_q35_attn_partial: 48 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 qwen35 tf_q35_attn_partial: 48 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 qwen35 tf_q35_attn_partial: 48 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 qwen35 tf_q35_attn_partial: 40 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 qwen35 tf_q35_attn_partial: 40 registers, 0 static shared bytes, bound 0 (record stale) +RECORD qwen35 tf_q35_attn_partial: .usage = .{ .{ .regs = 48, .static = 0, .bound = 0 }, .{ .regs = 48, .static = 0, .bound = 0 }, .{ .regs = 48, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35 tf_q35_attn_merge: 40 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_86 qwen35 tf_q35_attn_merge: 38 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_89 qwen35 tf_q35_attn_merge: 37 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_120 qwen35 tf_q35_attn_merge: 42 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 qwen35 tf_q35_attn_merge: 42 registers, 0 static shared bytes, bound 32 (record stale) +RECORD qwen35 tf_q35_attn_merge: .usage = .{ .{ .regs = 40, .static = 0, .bound = 32 }, .{ .regs = 38, .static = 0, .bound = 32 }, .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 42, .static = 0, .bound = 32 }, .{ .regs = 42, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35_kv8 tf_q35_rope8: 40 registers, 16416 static shared bytes, bound 256 (record stale) +PASS limits sm_86 qwen35_kv8 tf_q35_rope8: 41 registers, 16416 static shared bytes, bound 256 (record stale) +PASS limits sm_89 qwen35_kv8 tf_q35_rope8: 42 registers, 16416 static shared bytes, bound 256 (record stale) +PASS limits sm_120 qwen35_kv8 tf_q35_rope8: 40 registers, 16416 static shared bytes, bound 256 (record stale) +PASS limits sm_121 qwen35_kv8 tf_q35_rope8: 40 registers, 16416 static shared bytes, bound 256 (record stale) +RECORD qwen35_kv8 tf_q35_rope8: .usage = .{ .{ .regs = 40, .static = 16416, .bound = 256 }, .{ .regs = 41, .static = 16416, .bound = 256 }, .{ .regs = 42, .static = 16416, .bound = 256 }, .{ .regs = 40, .static = 16416, .bound = 256 }, .{ .regs = 40, .static = 16416, .bound = 256 }, } +PASS limits sm_80 qwen35_kv8 tf_q35_attn_partial8: 40 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 qwen35_kv8 tf_q35_attn_partial8: 40 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 qwen35_kv8 tf_q35_attn_partial8: 56 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 qwen35_kv8 tf_q35_attn_partial8: 34 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 qwen35_kv8 tf_q35_attn_partial8: 34 registers, 0 static shared bytes, bound 0 (record stale) +RECORD qwen35_kv8 tf_q35_attn_partial8: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 56, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35_keep tf_q35k_apply: 32 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 qwen35_keep tf_q35k_apply: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 qwen35_keep tf_q35k_apply: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 qwen35_keep tf_q35k_apply: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 qwen35_keep tf_q35k_apply: 40 registers, 0 static shared bytes, bound 128 (record stale) +RECORD qwen35_keep tf_q35k_apply: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35_keep tf_q35k_compact: 18 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 qwen35_keep tf_q35k_compact: 17 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 qwen35_keep tf_q35k_compact: 17 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 qwen35_keep tf_q35k_compact: 24 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 qwen35_keep tf_q35k_compact: 36 registers, 0 static shared bytes, bound 256 (record stale) +RECORD qwen35_keep tf_q35k_compact: .usage = .{ .{ .regs = 18, .static = 0, .bound = 256 }, .{ .regs = 17, .static = 0, .bound = 256 }, .{ .regs = 17, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 36, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen35_kv8 tf_q35k_compact8: 18 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 qwen35_kv8 tf_q35k_compact8: 38 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 qwen35_kv8 tf_q35k_compact8: 38 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 qwen35_kv8 tf_q35k_compact8: 28 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 qwen35_kv8 tf_q35k_compact8: 28 registers, 0 static shared bytes, bound 256 (record stale) +RECORD qwen35_kv8 tf_q35k_compact8: .usage = .{ .{ .regs = 18, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 28, .static = 0, .bound = 256 }, .{ .regs = 28, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD qwen35_draft tf_q35d_gather_rows: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_pre: 50 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_86 qwen35 tf_q35_gdn_pre: 52 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_89 qwen35 tf_q35_gdn_pre: 52 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_120 qwen35 tf_q35_gdn_pre: 40 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 qwen35 tf_q35_gdn_pre: 40 registers, 0 static shared bytes, bound 32 (record stale) +RECORD qwen35 tf_q35_gdn_pre: .usage = .{ .{ .regs = 50, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 40, .static = 0, .bound = 32 }, .{ .regs = 40, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35 tf_q35_conv_commit: 32 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 qwen35 tf_q35_conv_commit: 40 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 qwen35 tf_q35_conv_commit: 40 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 qwen35 tf_q35_conv_commit: 40 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 qwen35 tf_q35_conv_commit: 40 registers, 0 static shared bytes, bound 0 (record stale) +RECORD qwen35 tf_q35_conv_commit: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35 tf_q35_tap: 24 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 qwen35 tf_q35_tap: 24 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 qwen35 tf_q35_tap: 24 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 qwen35 tf_q35_tap: 26 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 qwen35 tf_q35_tap: 26 registers, 0 static shared bytes, bound 256 (record stale) +RECORD qwen35 tf_q35_tap: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 26, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen35_keep tf_q35k_conv: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 qwen35_keep tf_q35k_conv: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 qwen35_keep tf_q35k_conv: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 qwen35_keep tf_q35k_conv: 24 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 qwen35_keep tf_q35k_conv: 24 registers, 0 static shared bytes, bound 0 (record stale) +RECORD qwen35_keep tf_q35k_conv: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35_draft tf_q35d_dyn_conv_batch: 29 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 qwen35_draft tf_q35d_dyn_conv_batch: 28 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 qwen35_draft tf_q35d_dyn_conv_batch: 28 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 qwen35_draft tf_q35d_dyn_conv_batch: 36 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 qwen35_draft tf_q35d_dyn_conv_batch: 38 registers, 0 static shared bytes, bound 0 (record stale) +RECORD qwen35_draft tf_q35d_dyn_conv_batch: .usage = .{ .{ .regs = 29, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35_draft tf_q35d_norm_rope_batch: 30 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_86 qwen35_draft tf_q35d_norm_rope_batch: 32 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_89 qwen35_draft tf_q35d_norm_rope_batch: 32 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_120 qwen35_draft tf_q35d_norm_rope_batch: 36 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 qwen35_draft tf_q35d_norm_rope_batch: 36 registers, 0 static shared bytes, bound 32 (record stale) +RECORD qwen35_draft tf_q35d_norm_rope_batch: .usage = .{ .{ .regs = 30, .static = 0, .bound = 32 }, .{ .regs = 32, .static = 0, .bound = 32 }, .{ .regs = 32, .static = 0, .bound = 32 }, .{ .regs = 36, .static = 0, .bound = 32 }, .{ .regs = 36, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35_draft tf_q35d_mask_rows: 14 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 qwen35_draft tf_q35d_mask_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 qwen35_draft tf_q35d_mask_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 qwen35_draft tf_q35d_mask_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 qwen35_draft tf_q35d_mask_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +RECORD qwen35_draft tf_q35d_mask_rows: .usage = .{ .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35_draft tf_q35d_attn_partial: 52 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_86 qwen35_draft tf_q35d_attn_partial: 52 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_89 qwen35_draft tf_q35d_attn_partial: 50 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_120 qwen35_draft tf_q35d_attn_partial: 50 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 qwen35_draft tf_q35d_attn_partial: 50 registers, 0 static shared bytes, bound 32 (record stale) +RECORD qwen35_draft tf_q35d_attn_partial: .usage = .{ .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 50, .static = 0, .bound = 32 }, .{ .regs = 50, .static = 0, .bound = 32 }, .{ .regs = 50, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35_draft tf_q35d_attn_merge: 37 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_86 qwen35_draft tf_q35d_attn_merge: 37 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_89 qwen35_draft tf_q35d_attn_merge: 37 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_120 qwen35_draft tf_q35d_attn_merge: 34 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 qwen35_draft tf_q35d_attn_merge: 34 registers, 0 static shared bytes, bound 32 (record stale) +RECORD qwen35_draft tf_q35d_attn_merge: .usage = .{ .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 34, .static = 0, .bound = 32 }, .{ .regs = 34, .static = 0, .bound = 32 }, } +PASS limits sm_80 tf_topk_slices: 24 registers, 16448 static shared bytes, bound 256 (record stale) +PASS limits sm_86 tf_topk_slices: 20 registers, 16448 static shared bytes, bound 256 (record stale) +PASS limits sm_89 tf_topk_slices: 20 registers, 16448 static shared bytes, bound 256 (record stale) +PASS limits sm_120 tf_topk_slices: 28 registers, 16448 static shared bytes, bound 256 (record stale) +PASS limits sm_121 tf_topk_slices: 28 registers, 16448 static shared bytes, bound 256 (record stale) +RECORD tf_topk_slices: .usage = .{ .{ .regs = 24, .static = 16448, .bound = 256 }, .{ .regs = 20, .static = 16448, .bound = 256 }, .{ .regs = 20, .static = 16448, .bound = 256 }, .{ .regs = 28, .static = 16448, .bound = 256 }, .{ .regs = 28, .static = 16448, .bound = 256 }, } +PASS limits sm_80 tf_topk_merge: 24 registers, 64 static shared bytes, bound 256 (record stale) +PASS limits sm_86 tf_topk_merge: 22 registers, 64 static shared bytes, bound 256 (record stale) +PASS limits sm_89 tf_topk_merge: 22 registers, 64 static shared bytes, bound 256 (record stale) +PASS limits sm_120 tf_topk_merge: 34 registers, 64 static shared bytes, bound 256 (record stale) +PASS limits sm_121 tf_topk_merge: 36 registers, 64 static shared bytes, bound 256 (record stale) +RECORD tf_topk_merge: .usage = .{ .{ .regs = 24, .static = 64, .bound = 256 }, .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 34, .static = 64, .bound = 256 }, .{ .regs = 36, .static = 64, .bound = 256 }, } +PASS limits sm_80 tf_q35_norm: 57 registers, 16416 static shared bytes, bound 256 (record stale) +PASS limits sm_86 tf_q35_norm: 48 registers, 16416 static shared bytes, bound 256 (record stale) +PASS limits sm_89 tf_q35_norm: 48 registers, 16416 static shared bytes, bound 256 (record stale) +PASS limits sm_120 tf_q35_norm: 48 registers, 16416 static shared bytes, bound 256 (record stale) +PASS limits sm_121 tf_q35_norm: 48 registers, 16416 static shared bytes, bound 256 (record stale) +RECORD tf_q35_norm: .usage = .{ .{ .regs = 57, .static = 16416, .bound = 256 }, .{ .regs = 48, .static = 16416, .bound = 256 }, .{ .regs = 48, .static = 16416, .bound = 256 }, .{ .regs = 48, .static = 16416, .bound = 256 }, .{ .regs = 48, .static = 16416, .bound = 256 }, } +PASS limits sm_80 tf_q35_sums: 32 registers, 0 static shared bytes, bound 64 (record stale) +PASS limits sm_86 tf_q35_sums: 42 registers, 0 static shared bytes, bound 64 (record stale) +PASS limits sm_89 tf_q35_sums: 40 registers, 0 static shared bytes, bound 64 (record stale) +PASS limits sm_120 tf_q35_sums: 40 registers, 0 static shared bytes, bound 64 (record stale) +PASS limits sm_121 tf_q35_sums: 40 registers, 0 static shared bytes, bound 64 (record stale) +RECORD tf_q35_sums: .usage = .{ .{ .regs = 32, .static = 0, .bound = 64 }, .{ .regs = 42, .static = 0, .bound = 64 }, .{ .regs = 40, .static = 0, .bound = 64 }, .{ .regs = 40, .static = 0, .bound = 64 }, .{ .regs = 40, .static = 0, .bound = 64 }, } +PASS limits sm_80 tf_q35_argmax_slices: 22 registers, 64 static shared bytes, bound 256 (record stale) +PASS limits sm_86 tf_q35_argmax_slices: 22 registers, 64 static shared bytes, bound 256 (record stale) +PASS limits sm_89 tf_q35_argmax_slices: 22 registers, 64 static shared bytes, bound 256 (record stale) +PASS limits sm_120 tf_q35_argmax_slices: 26 registers, 64 static shared bytes, bound 256 (record stale) +PASS limits sm_121 tf_q35_argmax_slices: 26 registers, 64 static shared bytes, bound 256 (record stale) +RECORD tf_q35_argmax_slices: .usage = .{ .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 26, .static = 64, .bound = 256 }, .{ .regs = 26, .static = 64, .bound = 256 }, } +PASS limits sm_80 tf_q35_argmax_merge: 28 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_86 tf_q35_argmax_merge: 28 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_89 tf_q35_argmax_merge: 28 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_120 tf_q35_argmax_merge: 30 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 tf_q35_argmax_merge: 30 registers, 0 static shared bytes, bound 32 (record stale) +RECORD tf_q35_argmax_merge: .usage = .{ .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 30, .static = 0, .bound = 32 }, .{ .regs = 30, .static = 0, .bound = 32 }, } +PASS limits sm_80 sample tf_draw: 52 registers, 9752 static shared bytes, bound 1024 (record stale) +PASS limits sm_86 sample tf_draw: 64 registers, 9752 static shared bytes, bound 1024 (record stale) +PASS limits sm_89 sample tf_draw: 64 registers, 9752 static shared bytes, bound 1024 (record stale) +PASS limits sm_120 sample tf_draw: 64 registers, 9752 static shared bytes, bound 1024 (record stale) +PASS limits sm_121 sample tf_draw: 64 registers, 9752 static shared bytes, bound 1024 (record stale) +RECORD sample tf_draw: .usage = .{ .{ .regs = 52, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, } +PASS limits sm_80 sample tf_logprob_rows: 27 registers, 520 static shared bytes, bound 1024 (record stale) +PASS limits sm_86 sample tf_logprob_rows: 28 registers, 520 static shared bytes, bound 1024 (record stale) +PASS limits sm_89 sample tf_logprob_rows: 28 registers, 520 static shared bytes, bound 1024 (record stale) +PASS limits sm_120 sample tf_logprob_rows: 40 registers, 520 static shared bytes, bound 1024 (record stale) +PASS limits sm_121 sample tf_logprob_rows: 40 registers, 520 static shared bytes, bound 1024 (record stale) +RECORD sample tf_logprob_rows: .usage = .{ .{ .regs = 27, .static = 520, .bound = 1024 }, .{ .regs = 28, .static = 520, .bound = 1024 }, .{ .regs = 28, .static = 520, .bound = 1024 }, .{ .regs = 40, .static = 520, .bound = 1024 }, .{ .regs = 40, .static = 520, .bound = 1024 }, } +PASS limits sm_80 sample tf_grammar_mask: 28 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 sample tf_grammar_mask: 26 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 sample tf_grammar_mask: 26 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 sample tf_grammar_mask: 16 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 sample tf_grammar_mask: 16 registers, 0 static shared bytes, bound 256 (record stale) +RECORD sample tf_grammar_mask: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_glue tf_glm_top1: 29 registers, 8192 static shared bytes, bound 1024 (record stale) +PASS limits sm_86 glm_glue tf_glm_top1: 30 registers, 8192 static shared bytes, bound 1024 (record stale) +PASS limits sm_89 glm_glue tf_glm_top1: 30 registers, 8192 static shared bytes, bound 1024 (record stale) +PASS limits sm_120 glm_glue tf_glm_top1: 36 registers, 8192 static shared bytes, bound 1024 (record stale) +PASS limits sm_121 glm_glue tf_glm_top1: 36 registers, 8192 static shared bytes, bound 1024 (record stale) +RECORD glm_glue tf_glm_top1: .usage = .{ .{ .regs = 29, .static = 8192, .bound = 1024 }, .{ .regs = 30, .static = 8192, .bound = 1024 }, .{ .regs = 30, .static = 8192, .bound = 1024 }, .{ .regs = 36, .static = 8192, .bound = 1024 }, .{ .regs = 36, .static = 8192, .bound = 1024 }, } +PASS limits sm_80 glm_glue tf_glm_lat_write_fp8: 26 registers, 64 static shared bytes, bound 512 (record stale) +PASS limits sm_86 glm_glue tf_glm_lat_write_fp8: 28 registers, 64 static shared bytes, bound 512 (record stale) +PASS limits sm_89 glm_glue tf_glm_lat_write_fp8: 28 registers, 64 static shared bytes, bound 512 (record stale) +PASS limits sm_120 glm_glue tf_glm_lat_write_fp8: 25 registers, 64 static shared bytes, bound 512 (record stale) +PASS limits sm_121 glm_glue tf_glm_lat_write_fp8: 25 registers, 64 static shared bytes, bound 512 (record stale) +RECORD glm_glue tf_glm_lat_write_fp8: .usage = .{ .{ .regs = 26, .static = 64, .bound = 512 }, .{ .regs = 28, .static = 64, .bound = 512 }, .{ .regs = 28, .static = 64, .bound = 512 }, .{ .regs = 25, .static = 64, .bound = 512 }, .{ .regs = 25, .static = 64, .bound = 512 }, } +PASS limits sm_80 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +RECORD glm_glue tf_glm_chunks_16_fp8: .usage = .{ .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_glue tf_glm_chunks_32_fp8: 128 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_glue tf_glm_chunks_32_fp8: 128 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_glue tf_glm_chunks_32_fp8: 127 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_glue tf_glm_chunks_32_fp8: 130 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 glm_glue tf_glm_chunks_32_fp8: 130 registers, 0 static shared bytes, bound 256 (record stale) +RECORD glm_glue tf_glm_chunks_32_fp8: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 130, .static = 0, .bound = 256 }, .{ .regs = 130, .static = 0, .bound = 256 }, } +PASS limits sm_80 sample tf_draw_pitched: 57 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_86 sample tf_draw_pitched: 64 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_89 sample tf_draw_pitched: 64 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_120 sample tf_draw_pitched: 64 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_121 sample tf_draw_pitched: 64 registers, 9752 static shared bytes, bound 1024 (record current) +RECORD sample tf_draw_pitched: .usage = .{ .{ .regs = 57, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 120 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 120, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 64 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 64, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 166 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 166, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 72 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 72, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 167 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 167, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 72 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 72, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 145 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 145, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 79 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 79, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 168 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 168, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 102 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 102, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 167 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 167, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 102 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 102, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 96 registers, 16448 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 96, .static = 16448, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 registers, 16448 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 128, .static = 16448, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 registers, 16448 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 128, .static = 16448, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 205 registers, 16640 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 205, .static = 16640, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 229 registers, 16640 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 229, .static = 16640, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 229 registers, 16640 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 229, .static = 16640, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 189 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 189, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 248 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 248, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 250 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 250, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8: 125 registers, 32848 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8: .usage = .{ .{ .regs = 125, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10: 168 registers, 32848 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10: .usage = .{ .{ .regs = 168, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12: 168 registers, 32848 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12: .usage = .{ .{ .regs = 168, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8: 146 registers, 32848 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8: .usage = .{ .{ .regs = 146, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10: 168 registers, 32848 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10: .usage = .{ .{ .regs = 168, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12: 168 registers, 32848 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12: .usage = .{ .{ .regs = 168, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8: 117 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8: .usage = .{ .{ .regs = 117, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10: 167 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10: .usage = .{ .{ .regs = 167, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12: 167 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12: .usage = .{ .{ .regs = 167, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8: 138 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8: .usage = .{ .{ .regs = 138, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8: 117 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8: .usage = .{ .{ .regs = 117, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10: 167 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10: .usage = .{ .{ .regs = 167, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12: 167 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12: .usage = .{ .{ .regs = 167, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8: 138 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8: .usage = .{ .{ .regs = 138, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 114 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 114, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 167 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 167, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 168 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 168, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 135 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 135, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 167 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 167, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 168 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 168, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 88 registers, 16448 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 88, .static = 16448, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 96 registers, 16448 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 96, .static = 16448, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 registers, 16448 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 128, .static = 16448, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii: 23 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii: .usage = .{ .{ .regs = 23, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii: 23 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii: .usage = .{ .{ .regs = 23, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii: 30 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii: .usage = .{ .{ .regs = 30, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii: 30 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii: .usage = .{ .{ .regs = 30, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii: 28 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii: .usage = .{ .{ .regs = 28, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii: 28 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii: .usage = .{ .{ .regs = 28, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii: 40 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii: 40 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii: 40 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii: 40 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii: 42 registers, 128 static shared bytes, bound 1024 (record stale) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii: .usage = .{ .{ .regs = 42, .static = 128, .bound = 1024 }, } +N/A limits dsv41_router_glue router_group_rot sm_80: below_minimum +N/A limits dsv41_router_glue router_group_rot sm_86: below_minimum +N/A limits dsv41_router_glue router_group_rot sm_89: below_minimum +N/A limits dsv41_router_glue router_group_rot sm_120: below_minimum +PASS limits sm_121 dsv41_router_glue router_group_rot: 25 registers, 6528 static shared bytes, bound 1024 (record stale) +RECORD dsv41_router_glue router_group_rot: .usage = .{ .{ .regs = 25, .static = 6528, .bound = 1024 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi: 40 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii: 48 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii: .usage = .{ .{ .regs = 48, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii: 38 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii: .usage = .{ .{ .regs = 38, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii: 40 registers, 16384 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii: .usage = .{ .{ .regs = 40, .static = 16384, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii: 40 registers, 16384 static shared bytes, bound 0 (record stale) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii: .usage = .{ .{ .regs = 40, .static = 16384, .bound = 0 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 125 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 159 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 159, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 239 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 239, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 167 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 167, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 157 registers, 272 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 157, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 123 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 123, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 162 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 162, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 161 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 161, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 244 registers, 528 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 244, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi: 26 registers, 48 static shared bytes, bound 32 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi: .usage = .{ .{ .regs = 26, .static = 48, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 239 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 239, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 159 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 159, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 238 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 238, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 166 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 166, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 155 registers, 272 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 155, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 123 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 123, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 160 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 160, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 159 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 159, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi: 20 registers, 64 static shared bytes, bound 32 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi: .usage = .{ .{ .regs = 20, .static = 64, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 238 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 238, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 125 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 151 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 151, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 168 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 168, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 143 registers, 272 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 143, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 124 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 124, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 164 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 164, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 160 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 160, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi: 26 registers, 80 static shared bytes, bound 32 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi: .usage = .{ .{ .regs = 26, .static = 80, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 238 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 238, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 125 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 151 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 151, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 168 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 168, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 143 registers, 272 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 143, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 124 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 124, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 164 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 164, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 162 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 162, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi: 26 registers, 96 static shared bytes, bound 32 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi: .usage = .{ .{ .regs = 26, .static = 96, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 126 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 156 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 156, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 124 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 124, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 173 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 173, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 149 registers, 272 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 149, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 123 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 123, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 166 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 166, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 156 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 156, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi: 22 registers, 112 static shared bytes, bound 32 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi: .usage = .{ .{ .regs = 22, .static = 112, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 161 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 161, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 125 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 125 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 238 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 238, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 166 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 166, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 147 registers, 272 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 147, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 122 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 122, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 160 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 160, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 162 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 162, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi: 20 registers, 128 static shared bytes, bound 32 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi: .usage = .{ .{ .regs = 20, .static = 128, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 158 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 158, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 125 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 155 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 155, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 156 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 156, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 155 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 155, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 238 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 238, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 155 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 155, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 149 registers, 272 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 149, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 119 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 119, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 152 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 152, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 156 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 156, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi: 21 registers, 160 static shared bytes, bound 32 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi: .usage = .{ .{ .regs = 21, .static = 160, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 241 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 241, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 227 registers, 272 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 227, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 512 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 512 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 512 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: 242 registers, 544 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 544, .bound = 128 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: 242 registers, 544 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 544, .bound = 128 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE: 127 registers, 544 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ sm_80: below_minimum +N/A limits dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ sm_86: below_minimum +N/A limits dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ sm_89: below_minimum +N/A limits dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_: 53 registers, 38980 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_: .usage = .{ .{ .regs = 53, .static = 38980, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 24 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 29 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: .usage = .{ .{ .regs = 29, .static = 0, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 24 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 29 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: .usage = .{ .{ .regs = 29, .static = 0, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i: 22 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i: .usage = .{ .{ .regs = 22, .static = 0, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 114 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 114, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 117 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 117, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 125 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 125, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 161 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 161, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 162 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 162, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 117 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 117, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 121 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 121, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 122 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 122, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 160 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 160, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 163 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 163, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 160 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 160, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 164 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 230 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 230, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 150 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 150, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 155 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 155, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 221 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 221, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill: 24 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 24, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 217 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 217, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill: 23 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 23, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 127 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 127, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 127 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 127, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 127 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 127, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill: 21 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 21, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 123 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 123, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 122 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 122, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 124 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 124, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE: 39 registers, 32 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE: .usage = .{ .{ .regs = 39, .static = 32, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii: 189 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 189, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii: 179 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 179, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii: 242 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 242, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii: 236 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 236, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii: 208 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 208, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii: 200 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 200, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii: 254 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 254, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii: 255 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 255, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii: 254 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 254, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii: 255 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 255, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii: 174 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 174, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 167 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 167, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 193 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 193, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill: 32 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 32, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 165 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 165, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 199 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 199, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill: 31 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 31, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 166 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 166, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 206 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 206, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill: 29 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 29, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 214 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 214, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill: 37 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 37, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 222 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 222, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill: 40 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 40, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii: 36 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii: .usage = .{ .{ .regs = 36, .static = 0, .bound = 0 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii: 30 registers, 0 static shared bytes, bound 0 (record stale) +RECORD dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii: .usage = .{ .{ .regs = 30, .static = 0, .bound = 0 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 244 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 244, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 246 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 246, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 246 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 246, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 246 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 246, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 244 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 244, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 244 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 244, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 248 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 248, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 248 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 248, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 244 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 244, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 240 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 240, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 209 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 209, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 194 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 194, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 228 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 228, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 228 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 228, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 230 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 230, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 213 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 213, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 197 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 197, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 196 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 196, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 231 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 231, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 236 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 236, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 229 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 229, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 232 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 232, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 231 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 231, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 231 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 231, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 255 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 255, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 255 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 255, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 254 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 254, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 213 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 213, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 211 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 211, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 211 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 211, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 248 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 248, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 248 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 248, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 253 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 253, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 255 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 255, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 255 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 255, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 254 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 254, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 209 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 209, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 177 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 177, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 206 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 206, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 138 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 138, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 138 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 138, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 244 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 244, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 240 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 240, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 148 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 148, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 148 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 148, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 243 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 243, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 244 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 244, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 148 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 148, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 146 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 146, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 245 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 245, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 246 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 246, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 162 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 162, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 167 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 167, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 246 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 246, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 250 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 250, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 153 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 153, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 159 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 159, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 248 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 248, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 248 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 248, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 175 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 175, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 175 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 175, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 175 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 175, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 208 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 208, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 179 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 179, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 192 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 192, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 164 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 208 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 208, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 179 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 179, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 210 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 210, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 179 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 179, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 209 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 209, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 177 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 177, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 206 registers, 16 static shared bytes, bound 64 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 206, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record stale) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE: 112 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 112, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE: 112 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 112, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE: 112 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 112, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE: 112 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 112, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE: 107 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 107, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE: 107 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 107, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE: 107 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 107, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE: 107 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 107, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE: 228 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 228, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE: 228 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 228, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE: 230 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 230, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE: 230 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 230, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE: 228 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 228, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE: 228 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 228, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE: 230 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 230, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE: 230 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 230, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE: 90 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 90, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE: 90 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 90, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE: 90 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 90, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE: 90 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 90, .static = 0, .bound = 256 }, } +N/A limits dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE sm_80: below_minimum +N/A limits dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE sm_86: below_minimum +N/A limits dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE sm_89: below_minimum +N/A limits dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE: 64 registers, 3232 static shared bytes, bound 512 (record stale) +RECORD dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE: .usage = .{ .{ .regs = 64, .static = 3232, .bound = 512 }, } +N/A limits dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE sm_80: below_minimum +N/A limits dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE sm_86: below_minimum +N/A limits dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE sm_89: below_minimum +N/A limits dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE: 72 registers, 3232 static shared bytes, bound 512 (record stale) +RECORD dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE: .usage = .{ .{ .regs = 72, .static = 3232, .bound = 512 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 113 registers, 20496 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 113, .static = 20496, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 125 registers, 21392 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 21392, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 80 registers, 20496 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 20496, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 62 registers, 21392 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 62, .static = 21392, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 80 registers, 20496 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 20496, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 89 registers, 21392 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 89, .static = 21392, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE: 109 registers, 20496 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 109, .static = 20496, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 113 registers, 20656 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 113, .static = 20656, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 128 registers, 21552 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 21552, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 80 registers, 20656 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 20656, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 64 registers, 21552 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 64, .static = 21552, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 80 registers, 20656 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 20656, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 91 registers, 21552 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 91, .static = 21552, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE: 96 registers, 20656 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 96, .static = 20656, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE: 22 registers, 18048 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE: .usage = .{ .{ .regs = 22, .static = 18048, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE: 141 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 141, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE: 144 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 144, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE: 141 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 141, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE: 144 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 144, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE: 150 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 150, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE: 148 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 148, .static = 0, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE: 103 registers, 2052 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 103, .static = 2052, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE: 87 registers, 2052 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 87, .static = 2052, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE: 84 registers, 2052 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 84, .static = 2052, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE: 84 registers, 2052 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 84, .static = 2052, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE: 92 registers, 10244 static shared bytes, bound 32 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 92, .static = 10244, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE: 106 registers, 20484 static shared bytes, bound 64 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 106, .static = 20484, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE: 106 registers, 40964 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 106, .static = 40964, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE: 103 registers, 40964 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 103, .static = 40964, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE: 80 registers, 4100 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 4100, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE: 80 registers, 4100 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 4100, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE: 103 registers, 4100 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 103, .static = 4100, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE: 89 registers, 4100 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 89, .static = 4100, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE: 116 registers, 10244 static shared bytes, bound 32 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 116, .static = 10244, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE: 112 registers, 20484 static shared bytes, bound 64 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 112, .static = 20484, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE: 112 registers, 40964 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 112, .static = 40964, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE: 118 registers, 40964 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 118, .static = 40964, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE: 80 registers, 8196 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 8196, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE: 88 registers, 8196 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 8196, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE: 80 registers, 8196 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 8196, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE: 80 registers, 8196 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 8196, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE: 191 registers, 10244 static shared bytes, bound 32 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 191, .static = 10244, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE: 189 registers, 20484 static shared bytes, bound 64 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 189, .static = 20484, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE: 189 registers, 40964 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 189, .static = 40964, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE: 128 registers, 40964 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 40964, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE: 101 registers, 16388 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 101, .static = 16388, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE: 105 registers, 16388 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 105, .static = 16388, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE: 106 registers, 16388 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 106, .static = 16388, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE: 118 registers, 16388 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 118, .static = 16388, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE: 142 registers, 5636 static shared bytes, bound 32 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 142, .static = 5636, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE: 128 registers, 10244 static shared bytes, bound 64 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 10244, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE: 128 registers, 20484 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 20484, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE: 142 registers, 40964 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 142, .static = 40964, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE: 120 registers, 32772 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 120, .static = 32772, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE: 204 registers, 32772 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 204, .static = 32772, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE: 180 registers, 32772 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 180, .static = 32772, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE: 216 registers, 32772 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 216, .static = 32772, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE: 202 registers, 5636 static shared bytes, bound 32 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 202, .static = 5636, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE: 196 registers, 10244 static shared bytes, bound 64 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 196, .static = 10244, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE: 202 registers, 20484 static shared bytes, bound 128 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 202, .static = 20484, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE: 204 registers, 40964 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 204, .static = 40964, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 240 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi: 26 registers, 10240 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 26, .static = 10240, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 127 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 134 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 134, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 240 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi: 30 registers, 10240 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 30, .static = 10240, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 134 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 134, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 87 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 87, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 242 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi: 31 registers, 10752 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 31, .static = 10752, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 136 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 136, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 133 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 133, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 92 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 242 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi: 31 registers, 10752 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 31, .static = 10752, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 134 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 134, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 87 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 87, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 242 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi: 31 registers, 11264 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 31, .static = 11264, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 134 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 134, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 134 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 134, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 91 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 91, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 242 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi: 32 registers, 11264 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 32, .static = 11264, .bound = 128 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 86 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 240 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi: 27 registers, 12288 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 27, .static = 12288, .bound = 128 }, } +N/A limits dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi sm_80: below_minimum +N/A limits dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi sm_86: below_minimum +N/A limits dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi sm_89: below_minimum +N/A limits dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi sm_120: below_minimum +PASS limits sm_121 dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi: 48 registers, 8 static shared bytes, bound 0 (record stale) +RECORD dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi: .usage = .{ .{ .regs = 48, .static = 8, .bound = 0 }, } +N/A limits dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm sm_80: below_minimum +N/A limits dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm sm_86: below_minimum +N/A limits dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm sm_89: below_minimum +N/A limits dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm sm_120: below_minimum +PASS limits sm_121 dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm: 20 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm: .usage = .{ .{ .regs = 20, .static = 0, .bound = 32 }, } +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij sm_80: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij sm_86: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij sm_89: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij sm_120: below_minimum +PASS limits sm_121 dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij: 26 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij: .usage = .{ .{ .regs = 26, .static = 0, .bound = 256 }, } +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij sm_80: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij sm_86: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij sm_89: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij sm_120: below_minimum +PASS limits sm_121 dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij: 28 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx sm_80: below_minimum +N/A limits dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx sm_86: below_minimum +N/A limits dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx sm_89: below_minimum +N/A limits dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx sm_120: below_minimum +PASS limits sm_121 dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx: 64 registers, 9232 static shared bytes, bound 1024 (record stale) +RECORD dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx: .usage = .{ .{ .regs = 64, .static = 9232, .bound = 1024 }, } +N/A limits dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif sm_80: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif sm_86: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif sm_89: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif sm_120: below_minimum +PASS limits sm_121 dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif: 28 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx sm_80: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx sm_86: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx sm_89: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx sm_120: below_minimum +PASS limits sm_121 dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx: 28 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x sm_80: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x sm_86: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x sm_89: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x sm_120: below_minimum +PASS limits sm_121 dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x: 29 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x: .usage = .{ .{ .regs = 29, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE: 34 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE: .usage = .{ .{ .regs = 34, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh: 30 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE: 36 registers, 2336 static shared bytes, bound 256 (record stale) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE: .usage = .{ .{ .regs = 36, .static = 2336, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE: 36 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE: .usage = .{ .{ .regs = 36, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE: 24 registers, 32 static shared bytes, bound 256 (record stale) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE: .usage = .{ .{ .regs = 24, .static = 32, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE: 38 registers, 2336 static shared bytes, bound 256 (record stale) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE: .usage = .{ .{ .regs = 38, .static = 2336, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE: 35 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE: .usage = .{ .{ .regs = 35, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE: 38 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE: .usage = .{ .{ .regs = 38, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE: 23 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE: .usage = .{ .{ .regs = 23, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt: 32 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt: 64 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt: .usage = .{ .{ .regs = 64, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx: 24 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx: 24 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx: 22 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx: .usage = .{ .{ .regs = 22, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_: 12 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt: 40 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf: 16 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf: .usage = .{ .{ .regs = 16, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_: 12 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx: 30 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf: 44 registers, 2080 static shared bytes, bound 1024 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf: .usage = .{ .{ .regs = 44, .static = 2080, .bound = 1024 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd: 14 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd: .usage = .{ .{ .regs = 14, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx: 23 registers, 0 static shared bytes, bound 1024 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx: .usage = .{ .{ .regs = 23, .static = 0, .bound = 1024 }, } +N/A limits dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii: 40 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx: 20 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx: .usage = .{ .{ .regs = 20, .static = 0, .bound = 32 }, } +N/A limits dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii: 40 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx: 60 registers, 24576 static shared bytes, bound 256 (record stale) +RECORD dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx: .usage = .{ .{ .regs = 60, .static = 24576, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi: 46 registers, 9244 static shared bytes, bound 1024 (record stale) +RECORD dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi: .usage = .{ .{ .regs = 46, .static = 9244, .bound = 1024 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi: 40 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi: 40 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf: 32 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi: 28 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx: 26 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx: .usage = .{ .{ .regs = 26, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf: 40 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx: 36 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx: .usage = .{ .{ .regs = 36, .static = 0, .bound = 256 }, } +N/A limits dsv41_kv_glue kv_norm_store sm_80: below_minimum +N/A limits dsv41_kv_glue kv_norm_store sm_86: below_minimum +N/A limits dsv41_kv_glue kv_norm_store sm_89: below_minimum +N/A limits dsv41_kv_glue kv_norm_store sm_120: below_minimum +PASS limits sm_121 dsv41_kv_glue kv_norm_store: 26 registers, 1056 static shared bytes, bound 128 (record stale) +RECORD dsv41_kv_glue kv_norm_store: .usage = .{ .{ .regs = 26, .static = 1056, .bound = 128 }, } +N/A limits dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd sm_80: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd sm_86: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd sm_89: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd sm_120: below_minimum +PASS limits sm_121 dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd: 43 registers, 264 static shared bytes, bound 1024 (record stale) +RECORD dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd: .usage = .{ .{ .regs = 43, .static = 264, .bound = 1024 }, } +N/A limits dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf sm_80: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf sm_86: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf sm_89: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf sm_120: below_minimum +PASS limits sm_121 dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf: 18 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf: .usage = .{ .{ .regs = 18, .static = 0, .bound = 256 }, } +N/A limits dsv41_sampling dsv41_sampling_stats_max sm_80: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_max sm_86: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_max sm_89: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_max sm_120: below_minimum +PASS limits sm_121 dsv41_sampling dsv41_sampling_stats_max: 40 registers, 32 static shared bytes, bound 256 (record stale) +RECORD dsv41_sampling dsv41_sampling_stats_max: .usage = .{ .{ .regs = 40, .static = 32, .bound = 256 }, } +N/A limits dsv41_sampling dsv41_sampling_stats_sum sm_80: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_sum sm_86: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_sum sm_89: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_sum sm_120: below_minimum +PASS limits sm_121 dsv41_sampling dsv41_sampling_stats_sum: 38 registers, 64 static shared bytes, bound 256 (record stale) +RECORD dsv41_sampling dsv41_sampling_stats_sum: .usage = .{ .{ .regs = 38, .static = 64, .bound = 256 }, } +N/A limits dsv41_sampling dsv41_sampling_stats_fin sm_80: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_fin sm_86: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_fin sm_89: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_fin sm_120: below_minimum +PASS limits sm_121 dsv41_sampling dsv41_sampling_stats_fin: 40 registers, 0 static shared bytes, bound 32 (record stale) +RECORD dsv41_sampling dsv41_sampling_stats_fin: .usage = .{ .{ .regs = 40, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree_round tf_round_bf16_argmax: 25 registers, 3072 static shared bytes, bound 256 (record stale) +PASS limits sm_86 tree_round tf_round_bf16_argmax: 25 registers, 3072 static shared bytes, bound 256 (record stale) +PASS limits sm_89 tree_round tf_round_bf16_argmax: 25 registers, 3072 static shared bytes, bound 256 (record stale) +PASS limits sm_120 tree_round tf_round_bf16_argmax: 23 registers, 3072 static shared bytes, bound 256 (record stale) +PASS limits sm_121 tree_round tf_round_bf16_argmax: 23 registers, 3072 static shared bytes, bound 256 (record stale) +RECORD tree_round tf_round_bf16_argmax: .usage = .{ .{ .regs = 25, .static = 3072, .bound = 256 }, .{ .regs = 25, .static = 3072, .bound = 256 }, .{ .regs = 25, .static = 3072, .bound = 256 }, .{ .regs = 23, .static = 3072, .bound = 256 }, .{ .regs = 23, .static = 3072, .bound = 256 }, } +PASS limits sm_80 tree_round tf_tree_round_match: 25 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 tree_round tf_tree_round_match: 26 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 tree_round tf_tree_round_match: 26 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 tree_round tf_tree_round_match: 34 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 tree_round tf_tree_round_match: 34 registers, 0 static shared bytes, bound 0 (record stale) +RECORD tree_round tf_tree_round_match: .usage = .{ .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree_round tf_round_keep_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 tree_round tf_round_keep_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 tree_round tf_round_keep_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 tree_round tf_round_keep_rows: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 tree_round tf_round_keep_rows: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD tree_round tf_round_keep_rows: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +RECORD tree_round tf_round_gather_rows: .usage = .{ .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD tree_round tf_round_keep_head: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 act_rows tf_swiglu_rows: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_swiglu_rows: 35 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_swiglu_rows: 35 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_swiglu_rows: 34 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_swiglu_rows: 34 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_swiglu_rows: .usage = .{ .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 35, .static = 0, .bound = 256 }, .{ .regs = 35, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_swiglu_rows, 16 rows or fewer: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_swiglu_rows, 16 rows or fewer: 35 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_swiglu_rows, 16 rows or fewer: 35 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_swiglu_rows, 16 rows or fewer: 34 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_swiglu_rows, 16 rows or fewer: 34 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_swiglu_rows, 16 rows or fewer: .usage = .{ .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 35, .static = 0, .bound = 256 }, .{ .regs = 35, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_gate_rows: 32 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_gate_rows: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_gate_rows: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_gate_rows: 34 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_gate_rows: 34 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_gate_rows: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_gate_rows, 16 rows or fewer: 32 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_gate_rows, 16 rows or fewer: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_gate_rows, 16 rows or fewer: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_gate_rows, 16 rows or fewer: 34 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_gate_rows, 16 rows or fewer: 34 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_gate_rows, 16 rows or fewer: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_group_sums_rows: 12 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_group_sums_rows: 12 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_group_sums_rows: 12 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_group_sums_rows: 16 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_group_sums_rows: 16 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_group_sums_rows: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_dconv_rows: 32 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_dconv_rows: 37 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_dconv_rows: 37 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_dconv_rows: 37 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_dconv_rows: 37 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_dconv_rows: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 37, .static = 0, .bound = 256 }, .{ .regs = 37, .static = 0, .bound = 256 }, .{ .regs = 37, .static = 0, .bound = 256 }, .{ .regs = 37, .static = 0, .bound = 256 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_64: 44 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_64: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_64: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_64: 44 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_64: 44 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_64: .usage = .{ .{ .regs = 44, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 44, .static = 0, .bound = 128 }, .{ .regs = 44, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 44 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 44 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 44 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_64, 16 rows or fewer: .usage = .{ .{ .regs = 44, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 44, .static = 0, .bound = 128 }, .{ .regs = 44, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_128: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_128: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_128: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_128: 77 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_128: 77 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_128: .usage = .{ .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 77, .static = 0, .bound = 128 }, .{ .regs = 77, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 77 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 77 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_128, 16 rows or fewer: .usage = .{ .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 77, .static = 0, .bound = 128 }, .{ .regs = 77, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_256: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_256: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_256: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_256: 145 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_256: 145 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_256: .usage = .{ .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 145, .static = 0, .bound = 128 }, .{ .regs = 145, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 145 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 145 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_256, 16 rows or fewer: .usage = .{ .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 145, .static = 0, .bound = 128 }, .{ .regs = 145, .static = 0, .bound = 128 }, } +PASS limits sm_80 qk_rows tf_qk_rows at dim 64: 30 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_86 qk_rows tf_qk_rows at dim 64: 36 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_89 qk_rows tf_qk_rows at dim 64: 36 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_120 qk_rows tf_qk_rows at dim 64: 29 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_121 qk_rows tf_qk_rows at dim 64: 29 registers, 32 static shared bytes, bound 256 (record stale) +RECORD qk_rows tf_qk_rows at dim 64: .usage = .{ .{ .regs = 30, .static = 32, .bound = 256 }, .{ .regs = 36, .static = 32, .bound = 256 }, .{ .regs = 36, .static = 32, .bound = 256 }, .{ .regs = 29, .static = 32, .bound = 256 }, .{ .regs = 29, .static = 32, .bound = 256 }, } +PASS limits sm_80 qk_rows tf_qk_rows at dim 256: 30 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_86 qk_rows tf_qk_rows at dim 256: 36 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_89 qk_rows tf_qk_rows at dim 256: 36 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_120 qk_rows tf_qk_rows at dim 256: 29 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_121 qk_rows tf_qk_rows at dim 256: 29 registers, 32 static shared bytes, bound 256 (record stale) +RECORD qk_rows tf_qk_rows at dim 256: .usage = .{ .{ .regs = 30, .static = 32, .bound = 256 }, .{ .regs = 36, .static = 32, .bound = 256 }, .{ .regs = 36, .static = 32, .bound = 256 }, .{ .regs = 29, .static = 32, .bound = 256 }, .{ .regs = 29, .static = 32, .bound = 256 }, } +PASS limits sm_80 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +RECORD qk_rows tf_qk_rows_w: .usage = .{ .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 affine4 verify32 one CTA a tile: 79 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 affine4 verify32 one CTA a tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 affine4 verify32 one CTA a tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 affine4 verify32 one CTA a tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 affine4 verify32 one CTA a tile: 92 registers, 0 static shared bytes, bound 128 (record current) +RECORD affine4 verify32 one CTA a tile: .usage = .{ .{ .regs = 79, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, } +N/A limits affine4 verify32 cluster of 8 sm_80: below_minimum +N/A limits affine4 verify32 cluster of 8 sm_86: below_minimum +N/A limits affine4 verify32 cluster of 8 sm_89: below_minimum +PASS limits sm_120 affine4 verify32 cluster of 8: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 affine4 verify32 cluster of 8: 92 registers, 0 static shared bytes, bound 128 (record current) +RECORD affine4 verify32 cluster of 8: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, } +PASS limits sm_80 affine4 verify64 one CTA a tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 affine4 verify64 one CTA a tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 affine4 verify64 one CTA a tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 affine4 verify64 one CTA a tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 affine4 verify64 one CTA a tile: 124 registers, 0 static shared bytes, bound 128 (record current) +RECORD affine4 verify64 one CTA a tile: .usage = .{ .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, } +N/A limits affine4 verify64 cluster of 8 sm_80: below_minimum +N/A limits affine4 verify64 cluster of 8 sm_86: below_minimum +N/A limits affine4 verify64 cluster of 8 sm_89: below_minimum +PASS limits sm_120 affine4 verify64 cluster of 8: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 affine4 verify64 cluster of 8: 124 registers, 0 static shared bytes, bound 128 (record current) +RECORD affine4 verify64 cluster of 8: .usage = .{ .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, } +PASS limits sm_80 affine4 verify128 one CTA a tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 affine4 verify128 one CTA a tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 affine4 verify128 one CTA a tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 affine4 verify128 one CTA a tile: 240 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 affine4 verify128 one CTA a tile: 240 registers, 0 static shared bytes, bound 128 (record current) +RECORD affine4 verify128 one CTA a tile: .usage = .{ .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 240, .static = 0, .bound = 128 }, .{ .regs = 240, .static = 0, .bound = 128 }, } +N/A limits affine4 verify128 cluster of 8 sm_80: below_minimum +N/A limits affine4 verify128 cluster of 8 sm_86: below_minimum +N/A limits affine4 verify128 cluster of 8 sm_89: below_minimum +PASS limits sm_120 affine4 verify128 cluster of 8: 240 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 affine4 verify128 cluster of 8: 240 registers, 0 static shared bytes, bound 128 (record current) +RECORD affine4 verify128 cluster of 8: .usage = .{ .{ .regs = 240, .static = 0, .bound = 128 }, .{ .regs = 240, .static = 0, .bound = 128 }, } +RED limits sm_80 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_80: launched at 512 threads past its __launch_bounds__ 256 +RED limits sm_86 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_86: launched at 512 threads past its __launch_bounds__ 256 +RED limits sm_89 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_89: launched at 512 threads past its __launch_bounds__ 256 +RED limits sm_120 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_120: launched at 512 threads past its __launch_bounds__ 256 +RED limits sm_121 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_121: launched at 512 threads past its __launch_bounds__ 256 +RECORD control: past its bounds: .usage = .{ .{ .regs = 10, .static = 0, .bound = 256 }, .{ .regs = 10, .static = 0, .bound = 256 }, .{ .regs = 10, .static = 0, .bound = 256 }, .{ .regs = 10, .static = 0, .bound = 256 }, .{ .regs = 10, .static = 0, .bound = 256 }, } +RED limits sm_80 control: heavy at 1,024 threads: 168 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_80: 168 registers x 1024 threads need 172032, a block has 65536 +RED limits sm_86 control: heavy at 1,024 threads: 166 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_86: 166 registers x 1024 threads need 172032, a block has 65536 +RED limits sm_89 control: heavy at 1,024 threads: 166 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_89: 166 registers x 1024 threads need 172032, a block has 65536 +RED limits sm_120 control: heavy at 1,024 threads: 166 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_120: 166 registers x 1024 threads need 172032, a block has 65536 +RED limits sm_121 control: heavy at 1,024 threads: 166 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_121: 166 registers x 1024 threads need 172032, a block has 65536 +RECORD control: heavy at 1,024 threads: .usage = .{ .{ .regs = 168, .static = 0, .bound = 0 }, .{ .regs = 166, .static = 0, .bound = 0 }, .{ .regs = 166, .static = 0, .bound = 0 }, .{ .regs = 166, .static = 0, .bound = 0 }, .{ .regs = 166, .static = 0, .bound = 0 }, } +FAIL kernel-contract-limits: 915 problems +FAIL kernel-contract-limits: TestFailed diff --git a/docs/receipts/glm-kv-pages-0b094b74/gate5/limits-nvcc13.3.log b/docs/receipts/glm-kv-pages-0b094b74/gate5/limits-nvcc13.3.log new file mode 100644 index 0000000000..1d2a0eb013 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/gate5/limits-nvcc13.3.log @@ -0,0 +1,6982 @@ +INFO limits fatbins: images built by nvcc 13.3, checked against nvcc 13.3's records +PASS limits sm_80 flashnext_gdn flashnext_gdn_chain: 51 registers, 2060 static shared bytes, bound 1024 (record current) +PASS limits sm_86 flashnext_gdn flashnext_gdn_chain: 56 registers, 2060 static shared bytes, bound 1024 (record current) +PASS limits sm_89 flashnext_gdn flashnext_gdn_chain: 56 registers, 2060 static shared bytes, bound 1024 (record current) +PASS limits sm_120 flashnext_gdn flashnext_gdn_chain: 61 registers, 2060 static shared bytes, bound 1024 (record stale) +PASS limits sm_121 flashnext_gdn flashnext_gdn_chain: 61 registers, 2060 static shared bytes, bound 1024 (record stale) +RECORD flashnext_gdn flashnext_gdn_chain: .usage = .{ .{ .regs = 51, .static = 2060, .bound = 1024 }, .{ .regs = 56, .static = 2060, .bound = 1024 }, .{ .regs = 56, .static = 2060, .bound = 1024 }, .{ .regs = 61, .static = 2060, .bound = 1024 }, .{ .regs = 61, .static = 2060, .bound = 1024 }, } +PASS limits sm_80 flashnext_hc fn_add_streams: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_add_streams: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_add_streams: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_add_streams: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_add_streams: 35 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_add_streams: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_attn_chunks: 72 registers, 640 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_attn_chunks: .usage = .{ .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, } +PASS limits sm_80 flashnext_attn fn_attn_chunks_ahead: 70 registers, 768 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_attn_chunks_ahead: 70 registers, 768 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_attn_chunks_ahead: 70 registers, 768 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_attn_chunks_ahead: 72 registers, 768 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_attn_chunks_ahead: 72 registers, 768 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_attn_chunks_ahead: .usage = .{ .{ .regs = 70, .static = 768, .bound = 128 }, .{ .regs = 70, .static = 768, .bound = 128 }, .{ .regs = 70, .static = 768, .bound = 128 }, .{ .regs = 72, .static = 768, .bound = 128 }, .{ .regs = 72, .static = 768, .bound = 128 }, } +PASS limits sm_80 flashnext_attn fn_attn_gate: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_attn_gate: 23 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_attn_gate: 23 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_attn_gate: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn fn_attn_gate: 22 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_attn fn_attn_gate: .usage = .{ .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 23, .static = 0, .bound = 256 }, .{ .regs = 23, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_attn fn_attn_merge: 26 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_attn_merge: 28 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_attn_merge: 28 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_attn_merge: 25 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 flashnext_attn fn_attn_merge: 25 registers, 0 static shared bytes, bound 256 (record stale) +RECORD flashnext_attn fn_attn_merge: .usage = .{ .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 28, .static = 0, .bound = 256 }, .{ .regs = 28, .static = 0, .bound = 256 }, .{ .regs = 25, .static = 0, .bound = 256 }, .{ .regs = 25, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_attn fn_attn_prep: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_attn_prep: 34 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_attn_prep: 34 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_attn_prep: 30 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn fn_attn_prep: 30 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_attn fn_attn_prep: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 34, .static = 128, .bound = 256 }, .{ .regs = 34, .static = 128, .bound = 256 }, .{ .regs = 30, .static = 128, .bound = 256 }, .{ .regs = 30, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_gdn_front: 26 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_front: 26 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_front: 26 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_front: 30 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_front: 30 registers, 1024 static shared bytes, bound 256 (record current) +RECORD flashnext_batch fn_gdn_front: .usage = .{ .{ .regs = 26, .static = 1024, .bound = 256 }, .{ .regs = 26, .static = 1024, .bound = 256 }, .{ .regs = 26, .static = 1024, .bound = 256 }, .{ .regs = 30, .static = 1024, .bound = 256 }, .{ .regs = 30, .static = 1024, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_gdn_norm: 24 registers, 528 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_norm: 26 registers, 528 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_norm: 26 registers, 528 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_norm: 24 registers, 528 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_norm: 24 registers, 528 static shared bytes, bound 128 (record current) +RECORD flashnext_batch fn_gdn_norm: .usage = .{ .{ .regs = 24, .static = 528, .bound = 128 }, .{ .regs = 26, .static = 528, .bound = 128 }, .{ .regs = 26, .static = 528, .bound = 128 }, .{ .regs = 24, .static = 528, .bound = 128 }, .{ .regs = 24, .static = 528, .bound = 128 }, } +PASS limits sm_80 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_rows: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD flashnext_batch fn_gdn_rows: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_rows_to: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD flashnext_batch fn_gdn_rows_to: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 flashnext_hc fn_hc_dense_mix: 22 registers, 128 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_hc_dense_mix: 36 registers, 128 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_hc_dense_mix: 36 registers, 128 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_hc_dense_mix: 36 registers, 128 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_hc_dense_mix: 36 registers, 128 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_hc_dense_mix: .usage = .{ .{ .regs = 22, .static = 128, .bound = 0 }, .{ .regs = 36, .static = 128, .bound = 0 }, .{ .regs = 36, .static = 128, .bound = 0 }, .{ .regs = 36, .static = 128, .bound = 0 }, .{ .regs = 36, .static = 128, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_hc_down: 56 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_hc fn_hc_down: 54 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_hc fn_hc_down: 54 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_hc fn_hc_down: 64 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_hc fn_hc_down: 64 registers, 64 static shared bytes, bound 128 (record current) +RECORD flashnext_hc fn_hc_down: .usage = .{ .{ .regs = 56, .static = 64, .bound = 128 }, .{ .regs = 54, .static = 64, .bound = 128 }, .{ .regs = 54, .static = 64, .bound = 128 }, .{ .regs = 64, .static = 64, .bound = 128 }, .{ .regs = 64, .static = 64, .bound = 128 }, } +PASS limits sm_80 flashnext_hc fn_hc_normalize: 32 registers, 16 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_hc_normalize: 40 registers, 16 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_hc_normalize: 40 registers, 16 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_hc_normalize: 40 registers, 16 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_hc_normalize: 40 registers, 16 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_hc_normalize: .usage = .{ .{ .regs = 32, .static = 16, .bound = 0 }, .{ .regs = 40, .static = 16, .bound = 0 }, .{ .regs = 40, .static = 16, .bound = 0 }, .{ .regs = 40, .static = 16, .bound = 0 }, .{ .regs = 40, .static = 16, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_hc_reduce_act: 32 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_86 flashnext_hc fn_hc_reduce_act: 63 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_89 flashnext_hc fn_hc_reduce_act: 63 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_120 flashnext_hc fn_hc_reduce_act: 60 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_121 flashnext_hc fn_hc_reduce_act: 60 registers, 4096 static shared bytes, bound 1024 (record current) +RECORD flashnext_hc fn_hc_reduce_act: .usage = .{ .{ .regs = 32, .static = 4096, .bound = 1024 }, .{ .regs = 63, .static = 4096, .bound = 1024 }, .{ .regs = 63, .static = 4096, .bound = 1024 }, .{ .regs = 60, .static = 4096, .bound = 1024 }, .{ .regs = 60, .static = 4096, .bound = 1024 }, } +PASS limits sm_80 flashnext_hc fn_hc_upmix: 69 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_upmix: 64 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_upmix: 64 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_upmix: 62 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_upmix: 62 registers, 2048 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_upmix: .usage = .{ .{ .regs = 69, .static = 2048, .bound = 256 }, .{ .regs = 64, .static = 2048, .bound = 256 }, .{ .regs = 64, .static = 2048, .bound = 256 }, .{ .regs = 62, .static = 2048, .bound = 256 }, .{ .regs = 62, .static = 2048, .bound = 256 }, } +PASS limits sm_80 flashnext_hc fn_hc_writeback0: 30 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_writeback0: 30 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_writeback0: 30 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_writeback0: 37 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_writeback0: 37 registers, 256 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_writeback0: .usage = .{ .{ .regs = 30, .static = 256, .bound = 256 }, .{ .regs = 30, .static = 256, .bound = 256 }, .{ .regs = 30, .static = 256, .bound = 256 }, .{ .regs = 37, .static = 256, .bound = 256 }, .{ .regs = 37, .static = 256, .bound = 256 }, } +PASS limits sm_80 flashnext_hc fn_hc_writeback_moe: 32 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_writeback_moe: 40 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_writeback_moe: 40 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_writeback_moe: 47 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_writeback_moe: 47 registers, 256 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_writeback_moe: .usage = .{ .{ .regs = 32, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, .{ .regs = 47, .static = 256, .bound = 256 }, .{ .regs = 47, .static = 256, .bound = 256 }, } +PASS limits sm_80 flashnext_hc fn_hc_writeback_slices: 32 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_writeback_slices: 38 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_writeback_slices: 38 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_writeback_slices: 40 registers, 256 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_writeback_slices: 40 registers, 256 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_writeback_slices: .usage = .{ .{ .regs = 32, .static = 256, .bound = 256 }, .{ .regs = 38, .static = 256, .bound = 256 }, .{ .regs = 38, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, .{ .regs = 40, .static = 256, .bound = 256 }, } +PASS limits sm_80 flashnext_hc fn_ple_conv: 23 registers, 0 static shared bytes, bound 512 (record current) +PASS limits sm_86 flashnext_hc fn_ple_conv: 35 registers, 0 static shared bytes, bound 512 (record current) +PASS limits sm_89 flashnext_hc fn_ple_conv: 35 registers, 0 static shared bytes, bound 512 (record current) +PASS limits sm_120 flashnext_hc fn_ple_conv: 36 registers, 0 static shared bytes, bound 512 (record current) +PASS limits sm_121 flashnext_hc fn_ple_conv: 36 registers, 0 static shared bytes, bound 512 (record current) +RECORD flashnext_hc fn_ple_conv: .usage = .{ .{ .regs = 23, .static = 0, .bound = 512 }, .{ .regs = 35, .static = 0, .bound = 512 }, .{ .regs = 35, .static = 0, .bound = 512 }, .{ .regs = 36, .static = 0, .bound = 512 }, .{ .regs = 36, .static = 0, .bound = 512 }, } +PASS limits sm_80 flashnext_hc fn_ple_gate: 32 registers, 128 static shared bytes, bound 512 (record current) +PASS limits sm_86 flashnext_hc fn_ple_gate: 34 registers, 128 static shared bytes, bound 512 (record current) +PASS limits sm_89 flashnext_hc fn_ple_gate: 34 registers, 128 static shared bytes, bound 512 (record current) +PASS limits sm_120 flashnext_hc fn_ple_gate: 33 registers, 128 static shared bytes, bound 512 (record current) +PASS limits sm_121 flashnext_hc fn_ple_gate: 33 registers, 128 static shared bytes, bound 512 (record current) +RECORD flashnext_hc fn_ple_gate: .usage = .{ .{ .regs = 32, .static = 128, .bound = 512 }, .{ .regs = 34, .static = 128, .bound = 512 }, .{ .regs = 34, .static = 128, .bound = 512 }, .{ .regs = 33, .static = 128, .bound = 512 }, .{ .regs = 33, .static = 128, .bound = 512 }, } +PASS limits sm_80 flashnext_attn fn_qsa_pool: 32 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_qsa_pool: 40 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_qsa_pool: 40 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_qsa_pool: 39 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_qsa_pool: 39 registers, 1152 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_qsa_pool: .usage = .{ .{ .regs = 32, .static = 1152, .bound = 128 }, .{ .regs = 40, .static = 1152, .bound = 128 }, .{ .regs = 40, .static = 1152, .bound = 128 }, .{ .regs = 39, .static = 1152, .bound = 128 }, .{ .regs = 39, .static = 1152, .bound = 128 }, } +PASS limits sm_80 flashnext_attn fn_qsa_scores: 40 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_qsa_scores: 46 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_qsa_scores: 46 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_qsa_scores: 48 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn fn_qsa_scores: 48 registers, 4096 static shared bytes, bound 256 (record current) +RECORD flashnext_attn fn_qsa_scores: .usage = .{ .{ .regs = 40, .static = 4096, .bound = 256 }, .{ .regs = 46, .static = 4096, .bound = 256 }, .{ .regs = 46, .static = 4096, .bound = 256 }, .{ .regs = 48, .static = 4096, .bound = 256 }, .{ .regs = 48, .static = 4096, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_qsa_select: 32 registers, 1168 static shared bytes, bound 1024 (record current) +PASS limits sm_86 flashnext_batch fn_qsa_select: 50 registers, 1168 static shared bytes, bound 1024 (record current) +PASS limits sm_89 flashnext_batch fn_qsa_select: 50 registers, 1168 static shared bytes, bound 1024 (record current) +PASS limits sm_120 flashnext_batch fn_qsa_select: 44 registers, 1168 static shared bytes, bound 1024 (record current) +PASS limits sm_121 flashnext_batch fn_qsa_select: 44 registers, 1168 static shared bytes, bound 1024 (record current) +RECORD flashnext_batch fn_qsa_select: .usage = .{ .{ .regs = 32, .static = 1168, .bound = 1024 }, .{ .regs = 50, .static = 1168, .bound = 1024 }, .{ .regs = 50, .static = 1168, .bound = 1024 }, .{ .regs = 44, .static = 1168, .bound = 1024 }, .{ .regs = 44, .static = 1168, .bound = 1024 }, } +PASS limits sm_80 flashnext_mtp fn_row_best: 17 registers, 392 static shared bytes, bound 1024 (record current) +PASS limits sm_86 flashnext_mtp fn_row_best: 17 registers, 392 static shared bytes, bound 1024 (record current) +PASS limits sm_89 flashnext_mtp fn_row_best: 17 registers, 392 static shared bytes, bound 1024 (record current) +PASS limits sm_120 flashnext_mtp fn_row_best: 21 registers, 392 static shared bytes, bound 1024 (record current) +PASS limits sm_121 flashnext_mtp fn_row_best: 21 registers, 392 static shared bytes, bound 1024 (record current) +RECORD flashnext_mtp fn_row_best: .usage = .{ .{ .regs = 17, .static = 392, .bound = 1024 }, .{ .regs = 17, .static = 392, .bound = 1024 }, .{ .regs = 17, .static = 392, .bound = 1024 }, .{ .regs = 21, .static = 392, .bound = 1024 }, .{ .regs = 21, .static = 392, .bound = 1024 }, } +PASS limits sm_80 flashnext_hc fn_shared_scatter: 12 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_shared_scatter: 12 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_shared_scatter: 12 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_shared_scatter: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_shared_scatter: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_shared_scatter: .usage = .{ .{ .regs = 12, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_shared_swiglu: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_shared_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_shared_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_shared_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_shared_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_shared_swiglu: .usage = .{ .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_batch fn_shift_window: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_batch fn_shift_window: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_batch fn_shift_window: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_batch fn_shift_window: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_batch fn_shift_window: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_batch fn_shift_window: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_batch fn_sum_slices: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_batch fn_sum_slices: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_batch fn_sum_slices: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_batch fn_sum_slices: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_batch fn_sum_slices: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_batch fn_sum_slices: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 embed_rows tf_embed_rows_bf16_copies: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 embed_rows tf_embed_rows_bf16_copies: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 embed_rows tf_embed_rows_bf16_copies: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 embed_rows tf_embed_rows_bf16_copies: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 embed_rows tf_embed_rows_bf16_copies: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD embed_rows tf_embed_rows_bf16_copies: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 norms tf_rms_rows_sums: 31 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_86 norms tf_rms_rows_sums: 37 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_89 norms tf_rms_rows_sums: 37 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_120 norms tf_rms_rows_sums: 34 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_121 norms tf_rms_rows_sums: 34 registers, 132 static shared bytes, bound 1024 (record current) +RECORD norms tf_rms_rows_sums: .usage = .{ .{ .regs = 31, .static = 132, .bound = 1024 }, .{ .regs = 37, .static = 132, .bound = 1024 }, .{ .regs = 37, .static = 132, .bound = 1024 }, .{ .regs = 34, .static = 132, .bound = 1024 }, .{ .regs = 34, .static = 132, .bound = 1024 }, } +PASS limits sm_80 route_rows tf_route_logits: 32 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 route_rows tf_route_logits: 39 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 route_rows tf_route_logits: 39 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 route_rows tf_route_logits: 35 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 route_rows tf_route_logits: 35 registers, 0 static shared bytes, bound 128 (record current) +RECORD route_rows tf_route_logits: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 39, .static = 0, .bound = 128 }, .{ .regs = 39, .static = 0, .bound = 128 }, .{ .regs = 35, .static = 0, .bound = 128 }, .{ .regs = 35, .static = 0, .bound = 128 }, } +PASS limits sm_80 route_rows tf_route_logits_few: 31 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 route_rows tf_route_logits_few: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 route_rows tf_route_logits_few: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 route_rows tf_route_logits_few: 34 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 route_rows tf_route_logits_few: 34 registers, 0 static shared bytes, bound 128 (record current) +RECORD route_rows tf_route_logits_few: .usage = .{ .{ .regs = 31, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 34, .static = 0, .bound = 128 }, .{ .regs = 34, .static = 0, .bound = 128 }, } +PASS limits sm_80 route_rows tf_route_topk: 39 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_86 route_rows tf_route_topk: 39 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_89 route_rows tf_route_topk: 39 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_120 route_rows tf_route_topk: 36 registers, 64 static shared bytes, bound 128 (record current) +PASS limits sm_121 route_rows tf_route_topk: 36 registers, 64 static shared bytes, bound 128 (record current) +RECORD route_rows tf_route_topk: .usage = .{ .{ .regs = 39, .static = 64, .bound = 128 }, .{ .regs = 39, .static = 64, .bound = 128 }, .{ .regs = 39, .static = 64, .bound = 128 }, .{ .regs = 36, .static = 64, .bound = 128 }, .{ .regs = 36, .static = 64, .bound = 128 }, } +PASS limits sm_80 norms tf_rms_rows_bf16: 79 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_86 norms tf_rms_rows_bf16: 79 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_89 norms tf_rms_rows_bf16: 79 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_120 norms tf_rms_rows_bf16: 80 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_121 norms tf_rms_rows_bf16: 80 registers, 32 static shared bytes, bound 256 (record current) +RECORD norms tf_rms_rows_bf16: .usage = .{ .{ .regs = 79, .static = 32, .bound = 256 }, .{ .regs = 79, .static = 32, .bound = 256 }, .{ .regs = 79, .static = 32, .bound = 256 }, .{ .regs = 80, .static = 32, .bound = 256 }, .{ .regs = 80, .static = 32, .bound = 256 }, } +PASS limits sm_80 norms tf_rms_rows: 32 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_86 norms tf_rms_rows: 46 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_89 norms tf_rms_rows: 46 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_120 norms tf_rms_rows: 43 registers, 132 static shared bytes, bound 1024 (record current) +PASS limits sm_121 norms tf_rms_rows: 43 registers, 132 static shared bytes, bound 1024 (record current) +RECORD norms tf_rms_rows: .usage = .{ .{ .regs = 32, .static = 132, .bound = 1024 }, .{ .regs = 46, .static = 132, .bound = 1024 }, .{ .regs = 46, .static = 132, .bound = 1024 }, .{ .regs = 43, .static = 132, .bound = 1024 }, .{ .regs = 43, .static = 132, .bound = 1024 }, } +PASS limits sm_80 router tf_nemo_router: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 router tf_nemo_router: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 router tf_nemo_router: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 router tf_nemo_router: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 router tf_nemo_router: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD router tf_nemo_router: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 router tf_nemo_topk: 94 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 router tf_nemo_topk: 95 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 router tf_nemo_topk: 95 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 router tf_nemo_topk: 96 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 router tf_nemo_topk: 96 registers, 0 static shared bytes, bound 32 (record current) +RECORD router tf_nemo_topk: .usage = .{ .{ .regs = 94, .static = 0, .bound = 32 }, .{ .regs = 95, .static = 0, .bound = 32 }, .{ .regs = 95, .static = 0, .bound = 32 }, .{ .regs = 96, .static = 0, .bound = 32 }, .{ .regs = 96, .static = 0, .bound = 32 }, } +PASS limits sm_80 router tf_nemo_topk_wide: 168 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 router tf_nemo_topk_wide: 168 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 router tf_nemo_topk_wide: 168 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 router tf_nemo_topk_wide: 161 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 router tf_nemo_topk_wide: 161 registers, 0 static shared bytes, bound 32 (record stale) +RECORD router tf_nemo_topk_wide: .usage = .{ .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 161, .static = 0, .bound = 32 }, .{ .regs = 161, .static = 0, .bound = 32 }, } +PASS limits sm_80 latent tf_super_topk_few: 32 registers, 4352 static shared bytes, bound 1024 (record current) +PASS limits sm_86 latent tf_super_topk_few: 38 registers, 4352 static shared bytes, bound 1024 (record current) +PASS limits sm_89 latent tf_super_topk_few: 38 registers, 4352 static shared bytes, bound 1024 (record current) +PASS limits sm_120 latent tf_super_topk_few: 32 registers, 4352 static shared bytes, bound 1024 (record current) +PASS limits sm_121 latent tf_super_topk_few: 32 registers, 4352 static shared bytes, bound 1024 (record current) +RECORD latent tf_super_topk_few: .usage = .{ .{ .regs = 32, .static = 4352, .bound = 1024 }, .{ .regs = 38, .static = 4352, .bound = 1024 }, .{ .regs = 38, .static = 4352, .bound = 1024 }, .{ .regs = 32, .static = 4352, .bound = 1024 }, .{ .regs = 32, .static = 4352, .bound = 1024 }, } +N/A limits qmm_group cluster of 8 sm_80: below_minimum +N/A limits qmm_group cluster of 8 sm_86: below_minimum +N/A limits qmm_group cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmm_group cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmm_group cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmm_group cluster of 8: .usage = .{ .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmm_group one CTA a tile: 54 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmm_group one CTA a tile: 54 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmm_group one CTA a tile: 54 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmm_group one CTA a tile: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmm_group one CTA a tile: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmm_group one CTA a tile: .usage = .{ .{ .regs = 54, .static = 0, .bound = 128 }, .{ .regs = 54, .static = 0, .bound = 128 }, .{ .regs = 54, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 lane_gemv: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 lane_gemv: 58 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 lane_gemv: 58 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 lane_gemv: 66 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 lane_gemv: 66 registers, 0 static shared bytes, bound 128 (record current) +RECORD lane_gemv: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 58, .static = 0, .bound = 128 }, .{ .regs = 58, .static = 0, .bound = 128 }, .{ .regs = 66, .static = 0, .bound = 128 }, .{ .regs = 66, .static = 0, .bound = 128 }, } +PASS limits sm_80 lane_gemv split-K: 62 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_86 lane_gemv split-K: 56 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_89 lane_gemv split-K: 56 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_120 lane_gemv split-K: 72 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_121 lane_gemv split-K: 72 registers, 128 static shared bytes, bound 128 (record current) +RECORD lane_gemv split-K: .usage = .{ .{ .regs = 62, .static = 128, .bound = 128 }, .{ .regs = 56, .static = 128, .bound = 128 }, .{ .regs = 56, .static = 128, .bound = 128 }, .{ .regs = 72, .static = 128, .bound = 128 }, .{ .regs = 72, .static = 128, .bound = 128 }, } +PASS limits sm_80 prompt16 FP4: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 prompt16 FP4: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 prompt16 FP4: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 prompt16 FP4: 212 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 prompt16 FP4: 212 registers, 0 static shared bytes, bound 128 (record current) +RECORD prompt16 FP4: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, } +PASS limits sm_80 prompt16 FP8: 243 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 prompt16 FP8: 243 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 prompt16 FP8: 243 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 prompt16 FP8: 212 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 prompt16 FP8: 212 registers, 0 static shared bytes, bound 128 (record current) +RECORD prompt16 FP8: .usage = .{ .{ .regs = 243, .static = 0, .bound = 128 }, .{ .regs = 243, .static = 0, .bound = 128 }, .{ .regs = 243, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, } +PASS limits sm_80 prompt16 FP4 fp32 out: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 prompt16 FP4 fp32 out: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 prompt16 FP4 fp32 out: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 prompt16 FP4 fp32 out: 212 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 prompt16 FP4 fp32 out: 212 registers, 0 static shared bytes, bound 128 (record current) +RECORD prompt16 FP4 fp32 out: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, .{ .regs = 212, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP4 fp32 out 16-row tile: 88 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP4 fp32 out 16-row tile: 88 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP4 fp32 out 16-row tile: 96 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP4 fp32 out 16-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 16-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 16-row tile: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, .{ .regs = 88, .static = 0, .bound = 128 }, .{ .regs = 96, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +N/A limits qmmf FP4 fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP4 fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP4 fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP4 fp32 out 16-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 16-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP4 fp32 out 32-row tile: 106 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP4 fp32 out 32-row tile: 106 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP4 fp32 out 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP4 fp32 out 32-row tile: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 32-row tile: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 32-row tile: .usage = .{ .{ .regs = 106, .static = 0, .bound = 128 }, .{ .regs = 106, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +N/A limits qmmf FP4 fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP4 fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP4 fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP4 fp32 out 32-row tile, cluster of 8: 61 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 32-row tile, cluster of 8: 61 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 61, .static = 0, .bound = 128 }, .{ .regs = 61, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP4 fp32 out 64-row tile: 148 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP4 fp32 out 64-row tile: 148 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP4 fp32 out 64-row tile: 180 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP4 fp32 out 64-row tile: 78 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 64-row tile: 78 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 64-row tile: .usage = .{ .{ .regs = 148, .static = 0, .bound = 128 }, .{ .regs = 148, .static = 0, .bound = 128 }, .{ .regs = 180, .static = 0, .bound = 128 }, .{ .regs = 78, .static = 0, .bound = 128 }, .{ .regs = 78, .static = 0, .bound = 128 }, } +N/A limits qmmf FP4 fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP4 fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP4 fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP4 fp32 out 64-row tile, cluster of 8: 78 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out 64-row tile, cluster of 8: 78 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 78, .static = 0, .bound = 128 }, .{ .regs = 78, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP4 fp32 out fused slices: 182 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP4 fp32 out fused slices: 182 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP4 fp32 out fused slices: 228 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP4 fp32 out fused slices: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP4 fp32 out fused slices: 168 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP4 fp32 out fused slices: .usage = .{ .{ .regs = 182, .static = 0, .bound = 128 }, .{ .regs = 182, .static = 0, .bound = 128 }, .{ .regs = 228, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 16-row tile: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 16-row tile: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 16-row tile: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 16-row tile: 60 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 16-row tile: 60 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 16-row tile: .usage = .{ .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 60, .static = 0, .bound = 128 }, .{ .regs = 60, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 16-row tile, cluster of 8: .usage = .{ .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 32-row tile: 57 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 32-row tile: 57 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 32-row tile: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 32-row tile, cluster of 8: 66 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 32-row tile, cluster of 8: 66 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 32-row tile, cluster of 8: .usage = .{ .{ .regs = 66, .static = 0, .bound = 128 }, .{ .regs = 66, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 64-row tile: .usage = .{ .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 64-row tile, cluster of 8: 103 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 64-row tile, cluster of 8: 103 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 64-row tile, cluster of 8: .usage = .{ .{ .regs = 103, .static = 0, .bound = 128 }, .{ .regs = 103, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 64-row tile, fused slices: 158 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 64-row tile, fused slices: 158 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 64-row tile, fused slices: .usage = .{ .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 158, .static = 0, .bound = 128 }, .{ .regs = 158, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf reduce_kernel: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qmmf reduce_kernel: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qmmf reduce_kernel: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qmmf reduce_kernel: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qmmf reduce_kernel: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD qmmf reduce_kernel: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 qmmf FP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8 fp32 out 16-row tile: 50 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 16-row tile: 50 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 16-row tile: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 50, .static = 0, .bound = 128 }, .{ .regs = 50, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8 fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8 fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8 fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8 fp32 out 16-row tile, cluster of 8: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 16-row tile, cluster of 8: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8 fp32 out 32-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8 fp32 out 32-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8 fp32 out 32-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8 fp32 out 32-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 32-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 32-row tile: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8 fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8 fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8 fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8 fp32 out 32-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 32-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8 fp32 out 64-row tile: 126 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8 fp32 out 64-row tile: 126 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8 fp32 out 64-row tile: 126 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8 fp32 out 64-row tile: 72 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 64-row tile: 72 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 64-row tile: .usage = .{ .{ .regs = 126, .static = 0, .bound = 128 }, .{ .regs = 126, .static = 0, .bound = 128 }, .{ .regs = 126, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8 fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8 fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8 fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8 fp32 out 64-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 64-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8 fp32 out 64-row tile, fused slices: 179 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8 fp32 out 64-row tile, fused slices: 179 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8 fp32 out 64-row tile, fused slices: 179 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8 fp32 out 64-row tile, fused slices: 140 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8 fp32 out 64-row tile, fused slices: 140 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8 fp32 out 64-row tile, fused slices: .usage = .{ .{ .regs = 179, .static = 0, .bound = 128 }, .{ .regs = 179, .static = 0, .bound = 128 }, .{ .regs = 179, .static = 0, .bound = 128 }, .{ .regs = 140, .static = 0, .bound = 128 }, .{ .regs = 140, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 fp32 out 16-row tile: 60 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 16-row tile: 60 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 16-row tile: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 60, .static = 0, .bound = 128 }, .{ .regs = 60, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 fp32 out 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 fp32 out 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 fp32 out 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 fp32 out 32-row tile: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 fp32 out 32-row tile: 57 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 32-row tile: 57 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 32-row tile: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 fp32 out 32-row tile, cluster of 8: 66 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 32-row tile, cluster of 8: 66 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 66, .static = 0, .bound = 128 }, .{ .regs = 66, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 fp32 out 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 fp32 out 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 fp32 out 64-row tile: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 fp32 out 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 64-row tile: .usage = .{ .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, } +N/A limits qmmf MXFP8 fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf MXFP8 fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf MXFP8 fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf MXFP8 fp32 out 64-row tile, cluster of 8: 103 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 64-row tile, cluster of 8: 103 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 103, .static = 0, .bound = 128 }, .{ .regs = 103, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf MXFP8 fp32 out 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf MXFP8 fp32 out 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf MXFP8 fp32 out 64-row tile, fused slices: 229 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf MXFP8 fp32 out 64-row tile, fused slices: 158 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf MXFP8 fp32 out 64-row tile, fused slices: 158 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf MXFP8 fp32 out 64-row tile, fused slices: .usage = .{ .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 229, .static = 0, .bound = 128 }, .{ .regs = 158, .static = 0, .bound = 128 }, .{ .regs = 158, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8G fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8G fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8G fp32 out 16-row tile: 92 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8G fp32 out 16-row tile: 58 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 16-row tile: 58 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 16-row tile: .usage = .{ .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 58, .static = 0, .bound = 128 }, .{ .regs = 58, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8G fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8G fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8G fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8G fp32 out 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 16-row tile, cluster of 8: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8G fp32 out 32-row tile: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8G fp32 out 32-row tile: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8G fp32 out 32-row tile: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8G fp32 out 32-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 32-row tile: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 32-row tile: .usage = .{ .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8G fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8G fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8G fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8G fp32 out 32-row tile, cluster of 8: 57 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 32-row tile, cluster of 8: 57 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 57, .static = 0, .bound = 128 }, .{ .regs = 57, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8G fp32 out 64-row tile: 164 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8G fp32 out 64-row tile: 164 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8G fp32 out 64-row tile: 164 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8G fp32 out 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 64-row tile: 108 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 64-row tile: .usage = .{ .{ .regs = 164, .static = 0, .bound = 128 }, .{ .regs = 164, .static = 0, .bound = 128 }, .{ .regs = 164, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, .{ .regs = 108, .static = 0, .bound = 128 }, } +N/A limits qmmf FP8G fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf FP8G fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf FP8G fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf FP8G fp32 out 64-row tile, cluster of 8: 122 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 64-row tile, cluster of 8: 122 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 122, .static = 0, .bound = 128 }, .{ .regs = 122, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf FP8G fp32 out 64-row tile, fused slices: 196 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf FP8G fp32 out 64-row tile, fused slices: 196 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf FP8G fp32 out 64-row tile, fused slices: 196 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf FP8G fp32 out 64-row tile, fused slices: 155 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf FP8G fp32 out 64-row tile, fused slices: 155 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf FP8G fp32 out 64-row tile, fused slices: .usage = .{ .{ .regs = 196, .static = 0, .bound = 128 }, .{ .regs = 196, .static = 0, .bound = 128 }, .{ .regs = 196, .static = 0, .bound = 128 }, .{ .regs = 155, .static = 0, .bound = 128 }, .{ .regs = 155, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf BF16 fp32 out 16-row tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf BF16 fp32 out 16-row tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf BF16 fp32 out 16-row tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf BF16 fp32 out 16-row tile: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 16-row tile: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 16-row tile: .usage = .{ .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +N/A limits qmmf BF16 fp32 out 16-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf BF16 fp32 out 16-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf BF16 fp32 out 16-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf BF16 fp32 out 16-row tile, cluster of 8: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 16-row tile, cluster of 8: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 16-row tile, cluster of 8: .usage = .{ .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf BF16 fp32 out 32-row tile: 93 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf BF16 fp32 out 32-row tile: 93 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf BF16 fp32 out 32-row tile: 93 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf BF16 fp32 out 32-row tile: 55 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 32-row tile: 55 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 32-row tile: .usage = .{ .{ .regs = 93, .static = 0, .bound = 128 }, .{ .regs = 93, .static = 0, .bound = 128 }, .{ .regs = 93, .static = 0, .bound = 128 }, .{ .regs = 55, .static = 0, .bound = 128 }, .{ .regs = 55, .static = 0, .bound = 128 }, } +N/A limits qmmf BF16 fp32 out 32-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf BF16 fp32 out 32-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf BF16 fp32 out 32-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf BF16 fp32 out 32-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 32-row tile, cluster of 8: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 32-row tile, cluster of 8: .usage = .{ .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf BF16 fp32 out 64-row tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf BF16 fp32 out 64-row tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf BF16 fp32 out 64-row tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf BF16 fp32 out 64-row tile: 72 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 64-row tile: 72 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 64-row tile: .usage = .{ .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, } +N/A limits qmmf BF16 fp32 out 64-row tile, cluster of 8 sm_80: below_minimum +N/A limits qmmf BF16 fp32 out 64-row tile, cluster of 8 sm_86: below_minimum +N/A limits qmmf BF16 fp32 out 64-row tile, cluster of 8 sm_89: below_minimum +PASS limits sm_120 qmmf BF16 fp32 out 64-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 64-row tile, cluster of 8: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 64-row tile, cluster of 8: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf BF16 fp32 out 64-row tile, fused slices: 180 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qmmf BF16 fp32 out 64-row tile, fused slices: 180 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qmmf BF16 fp32 out 64-row tile, fused slices: 180 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qmmf BF16 fp32 out 64-row tile, fused slices: 140 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qmmf BF16 fp32 out 64-row tile, fused slices: 140 registers, 0 static shared bytes, bound 128 (record current) +RECORD qmmf BF16 fp32 out 64-row tile, fused slices: .usage = .{ .{ .regs = 180, .static = 0, .bound = 128 }, .{ .regs = 180, .static = 0, .bound = 128 }, .{ .regs = 180, .static = 0, .bound = 128 }, .{ .regs = 140, .static = 0, .bound = 128 }, .{ .regs = 140, .static = 0, .bound = 128 }, } +PASS limits sm_80 qmmf reduce_kernel: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qmmf reduce_kernel: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qmmf reduce_kernel: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qmmf reduce_kernel: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qmmf reduce_kernel: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD qmmf reduce_kernel: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits ck tf_ck4_up_relu2_few sm_80: below_minimum +N/A limits ck tf_ck4_up_relu2_few sm_86: below_minimum +N/A limits ck tf_ck4_up_relu2_few sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_up_relu2_few: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 ck tf_ck4_up_relu2_few: 124 registers, 0 static shared bytes, bound 128 (record current) +RECORD ck tf_ck4_up_relu2_few: .usage = .{ .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, } +N/A limits ck tf_ck4_down_f32_few sm_80: below_minimum +N/A limits ck tf_ck4_down_f32_few sm_86: below_minimum +N/A limits ck tf_ck4_down_f32_few sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_down_f32_few: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 ck tf_ck4_down_f32_few: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD ck tf_ck4_down_f32_few: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 128, .static = 0, .bound = 128 }, } +PASS limits sm_80 mamba tf_nemo_scan: 138 registers, 41088 static shared bytes, bound 128 (record current) +PASS limits sm_86 mamba tf_nemo_scan: 144 registers, 41088 static shared bytes, bound 128 (record current) +PASS limits sm_89 mamba tf_nemo_scan: 144 registers, 41088 static shared bytes, bound 128 (record current) +PASS limits sm_120 mamba tf_nemo_scan: 115 registers, 41088 static shared bytes, bound 128 (record stale) +PASS limits sm_121 mamba tf_nemo_scan: 115 registers, 41088 static shared bytes, bound 128 (record stale) +RECORD mamba tf_nemo_scan: .usage = .{ .{ .regs = 138, .static = 41088, .bound = 128 }, .{ .regs = 144, .static = 41088, .bound = 128 }, .{ .regs = 144, .static = 41088, .bound = 128 }, .{ .regs = 115, .static = 41088, .bound = 128 }, .{ .regs = 115, .static = 41088, .bound = 128 }, } +PASS limits sm_80 rank_sum tf_rank_sum: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 rank_sum tf_rank_sum: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 rank_sum tf_rank_sum: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 rank_sum tf_rank_sum: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 rank_sum tf_rank_sum: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD rank_sum tf_rank_sum: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit vision_kernel<80, 8> (RADIO): 143 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit vision_kernel<80, 8> (RADIO): 143 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit vision_kernel<80, 8> (RADIO): 143 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit vision_kernel<80, 8> (RADIO): 122 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit vision_kernel<80, 8> (RADIO): 122 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit vision_kernel<80, 8> (RADIO): .usage = .{ .{ .regs = 143, .static = 0, .bound = 256 }, .{ .regs = 143, .static = 0, .bound = 256 }, .{ .regs = 143, .static = 0, .bound = 256 }, .{ .regs = 122, .static = 0, .bound = 256 }, .{ .regs = 122, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit full_kernel<72> (SigLIP2): 158 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit full_kernel<72> (SigLIP2): 158 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit full_kernel<72> (SigLIP2): 158 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit full_kernel<72> (SigLIP2): 169 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit full_kernel<72> (SigLIP2): 169 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit full_kernel<72> (SigLIP2): .usage = .{ .{ .regs = 158, .static = 0, .bound = 256 }, .{ .regs = 158, .static = 0, .bound = 256 }, .{ .regs = 158, .static = 0, .bound = 256 }, .{ .regs = 169, .static = 0, .bound = 256 }, .{ .regs = 169, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit tf_vit_layer_norm: 38 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit tf_vit_layer_norm: 38 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit tf_vit_layer_norm: 38 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit tf_vit_layer_norm: 40 registers, 32 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit tf_vit_layer_norm: 40 registers, 32 static shared bytes, bound 256 (record current) +RECORD vit tf_vit_layer_norm: .usage = .{ .{ .regs = 38, .static = 32, .bound = 256 }, .{ .regs = 38, .static = 32, .bound = 256 }, .{ .regs = 38, .static = 32, .bound = 256 }, .{ .regs = 40, .static = 32, .bound = 256 }, .{ .regs = 40, .static = 32, .bound = 256 }, } +PASS limits sm_80 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit tf_vit_pointwise: 14 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit tf_vit_pointwise: .usage = .{ .{ .regs = 14, .static = 0, .bound = 256 }, .{ .regs = 14, .static = 0, .bound = 256 }, .{ .regs = 14, .static = 0, .bound = 256 }, .{ .regs = 14, .static = 0, .bound = 256 }, .{ .regs = 14, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit tf_vit_bias: 12 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit tf_vit_bias: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, } +PASS limits sm_80 vit tf_vit_unshuffle: 29 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 vit tf_vit_unshuffle: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 vit tf_vit_unshuffle: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 vit tf_vit_unshuffle: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 vit tf_vit_unshuffle: 34 registers, 0 static shared bytes, bound 256 (record current) +RECORD vit tf_vit_unshuffle: .usage = .{ .{ .regs = 29, .static = 0, .bound = 256 }, .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_embed: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_embed: 33 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_embed: 33 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_embed: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_embed: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_embed: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_layernorm: 23 registers, 132 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_layernorm: 26 registers, 132 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_layernorm: 26 registers, 132 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_layernorm: 23 registers, 132 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_layernorm: 23 registers, 132 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_layernorm: .usage = .{ .{ .regs = 23, .static = 132, .bound = 256 }, .{ .regs = 26, .static = 132, .bound = 256 }, .{ .regs = 26, .static = 132, .bound = 256 }, .{ .regs = 23, .static = 132, .bound = 256 }, .{ .regs = 23, .static = 132, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_qkv_rope: 32 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qwen_vision tf_qv_qkv_rope: 37 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qwen_vision tf_qv_qkv_rope: 37 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qwen_vision tf_qv_qkv_rope: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qwen_vision tf_qv_qkv_rope: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD qwen_vision tf_qv_qkv_rope: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 37, .static = 0, .bound = 128 }, .{ .regs = 37, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen_vision tf_qv_attention72: 180 registers, 36864 static shared bytes, bound 64 (record current) +PASS limits sm_86 qwen_vision tf_qv_attention72: 180 registers, 36864 static shared bytes, bound 64 (record current) +PASS limits sm_89 qwen_vision tf_qv_attention72: 180 registers, 36864 static shared bytes, bound 64 (record current) +PASS limits sm_120 qwen_vision tf_qv_attention72: 186 registers, 36864 static shared bytes, bound 64 (record current) +PASS limits sm_121 qwen_vision tf_qv_attention72: 186 registers, 36864 static shared bytes, bound 64 (record current) +RECORD qwen_vision tf_qv_attention72: .usage = .{ .{ .regs = 180, .static = 36864, .bound = 64 }, .{ .regs = 180, .static = 36864, .bound = 64 }, .{ .regs = 180, .static = 36864, .bound = 64 }, .{ .regs = 186, .static = 36864, .bound = 64 }, .{ .regs = 186, .static = 36864, .bound = 64 }, } +PASS limits sm_80 qwen_vision tf_qv_attention64: 164 registers, 32768 static shared bytes, bound 64 (record current) +PASS limits sm_86 qwen_vision tf_qv_attention64: 164 registers, 32768 static shared bytes, bound 64 (record current) +PASS limits sm_89 qwen_vision tf_qv_attention64: 164 registers, 32768 static shared bytes, bound 64 (record current) +PASS limits sm_120 qwen_vision tf_qv_attention64: 162 registers, 32768 static shared bytes, bound 64 (record current) +PASS limits sm_121 qwen_vision tf_qv_attention64: 162 registers, 32768 static shared bytes, bound 64 (record current) +RECORD qwen_vision tf_qv_attention64: .usage = .{ .{ .regs = 164, .static = 32768, .bound = 64 }, .{ .regs = 164, .static = 32768, .bound = 64 }, .{ .regs = 164, .static = 32768, .bound = 64 }, .{ .regs = 162, .static = 32768, .bound = 64 }, .{ .regs = 162, .static = 32768, .bound = 64 }, } +PASS limits sm_80 qwen_vision tf_qv_bias_gelu: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_bias_gelu: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_bias_gelu: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_bias_gelu: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_bias_gelu: 20 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_bias_gelu: .usage = .{ .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_bias_residual: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_bias_residual: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_bias_residual: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_bias_residual: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_bias_residual: 20 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_bias_residual: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_bias_cast: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen_vision tf_qv_bias_cast: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen_vision tf_qv_bias_cast: 12 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen_vision tf_qv_bias_cast: 18 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen_vision tf_qv_bias_cast: 18 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen_vision tf_qv_bias_cast: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 18, .static = 0, .bound = 256 }, .{ .regs = 18, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen_vision tf_qv_splice: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qwen_vision tf_qv_splice: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qwen_vision tf_qv_splice: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qwen_vision tf_qv_splice: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qwen_vision tf_qv_splice: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD qwen_vision tf_qv_splice: .usage = .{ .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_attn fn_attn_prep_media: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn fn_attn_prep_media: 34 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn fn_attn_prep_media: 34 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn fn_attn_prep_media: 30 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn fn_attn_prep_media: 30 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_attn fn_attn_prep_media: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 34, .static = 128, .bound = 256 }, .{ .regs = 34, .static = 128, .bound = 256 }, .{ .regs = 30, .static = 128, .bound = 256 }, .{ .regs = 30, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_attn fn_qsa_pool_media: 32 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_qsa_pool_media: 40 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_qsa_pool_media: 40 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_qsa_pool_media: 39 registers, 1152 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_qsa_pool_media: 39 registers, 1152 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_qsa_pool_media: .usage = .{ .{ .regs = 32, .static = 1152, .bound = 128 }, .{ .regs = 40, .static = 1152, .bound = 128 }, .{ .regs = 40, .static = 1152, .bound = 128 }, .{ .regs = 39, .static = 1152, .bound = 128 }, .{ .regs = 39, .static = 1152, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_rot_in: 19 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_rot_in: 22 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_rot_in: 22 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_rot_in: 22 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_rot_in: 22 registers, 0 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_rot_in: .usage = .{ .{ .regs = 19, .static = 0, .bound = 128 }, .{ .regs = 22, .static = 0, .bound = 128 }, .{ .regs = 22, .static = 0, .bound = 128 }, .{ .regs = 22, .static = 0, .bound = 128 }, .{ .regs = 22, .static = 0, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_2_4: 154 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_2_4: 154 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_2_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 154, .static = 16, .bound = 128 }, .{ .regs = 154, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_2_4: 154 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_2_4: 154 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_4_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 154, .static = 16, .bound = 128 }, .{ .regs = 154, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_2_4: 159 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_2_4: 159 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_6_2_4: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 159, .static = 16, .bound = 128 }, .{ .regs = 159, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_2_4: 156 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_2_4: 156 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_8_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_10_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_12_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_2_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_2_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_14_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_2_4: 125 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_2_4: 125 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_16_2_4: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 125, .static = 16, .bound = 128 }, .{ .regs = 125, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_3_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_3_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_3_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_3_2_4: 156 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_3_2_4: 156 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_3_2_4: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_5_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_5_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_5_2_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_5_2_4: 157 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_5_2_4: 157 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_5_2_4: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 157, .static = 16, .bound = 128 }, .{ .regs = 157, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_7_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_7_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_9_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_9_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_11_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_11_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_13_2_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_13_2_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_13_2_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_13_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_13_2_4: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_13_2_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_1_4: 158 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_1_4: 158 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_2_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 158, .static = 16, .bound = 128 }, .{ .regs = 158, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_1_4: 158 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_1_4: 158 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_4_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 158, .static = 16, .bound = 128 }, .{ .regs = 158, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_1_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_1_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_1_4: 164 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_1_4: 162 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_1_4: 162 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_6_1_4: .usage = .{ .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 164, .static = 16, .bound = 128 }, .{ .regs = 162, .static = 16, .bound = 128 }, .{ .regs = 162, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_1_4: 160 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_1_4: 160 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_8_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 160, .static = 16, .bound = 128 }, .{ .regs = 160, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_10_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_12_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_1_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_1_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_1_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_14_1_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_1_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_1_4: 123 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_1_4: 123 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_16_1_4: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 123, .static = 16, .bound = 128 }, .{ .regs = 123, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_0_4: 255 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_0_4: 255 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_0_4: 255 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_0_4: 156 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_0_4: 156 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_2_0_4: .usage = .{ .{ .regs = 255, .static = 16, .bound = 128 }, .{ .regs = 255, .static = 16, .bound = 128 }, .{ .regs = 255, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_0_4: 255 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_0_4: 255 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_0_4: 255 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_0_4: 156 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_0_4: 156 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_4_0_4: .usage = .{ .{ .regs = 255, .static = 16, .bound = 128 }, .{ .regs = 255, .static = 16, .bound = 128 }, .{ .regs = 255, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_0_4: 160 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_0_4: 160 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_6_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 160, .static = 16, .bound = 128 }, .{ .regs = 160, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_0_4: 254 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_0_4: 254 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_0_4: 254 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_0_4: 162 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_0_4: 162 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_8_0_4: .usage = .{ .{ .regs = 254, .static = 16, .bound = 128 }, .{ .regs = 254, .static = 16, .bound = 128 }, .{ .regs = 254, .static = 16, .bound = 128 }, .{ .regs = 162, .static = 16, .bound = 128 }, .{ .regs = 162, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_10_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_12_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 168, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_0_4: 168 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_0_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_0_4: 167 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_0_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_0_4: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_14_0_4: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 167, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 128, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_0_4: 128 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_0_4: 156 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_0_4: 156 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_0_4: 125 registers, 16 static shared bytes, bound 128 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_0_4: 125 registers, 16 static shared bytes, bound 128 (record current) +RECORD exl3_linear tf_exl3_linear_16_0_4: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, .{ .regs = 156, .static = 16, .bound = 128 }, .{ .regs = 125, .static = 16, .bound = 128 }, .{ .regs = 125, .static = 16, .bound = 128 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_2_8: 192 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_2_8: 192 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_2_8: 192 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_2_2_8: .usage = .{ .{ .regs = 192, .static = 16, .bound = 256 }, .{ .regs = 192, .static = 16, .bound = 256 }, .{ .regs = 192, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_2_8: 192 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_2_8: 192 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_2_8: 192 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_4_2_8: .usage = .{ .{ .regs = 192, .static = 16, .bound = 256 }, .{ .regs = 192, .static = 16, .bound = 256 }, .{ .regs = 192, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_2_8: 159 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_6_2_8: .usage = .{ .{ .regs = 159, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_2_8: 199 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_2_8: 199 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_2_8: 199 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_8_2_8: .usage = .{ .{ .regs = 199, .static = 16, .bound = 256 }, .{ .regs = 199, .static = 16, .bound = 256 }, .{ .regs = 199, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_2_8: 178 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_2_8: 178 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_2_8: 178 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_2_8: 228 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_2_8: 228 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_10_2_8: .usage = .{ .{ .regs = 178, .static = 16, .bound = 256 }, .{ .regs = 178, .static = 16, .bound = 256 }, .{ .regs = 178, .static = 16, .bound = 256 }, .{ .regs = 228, .static = 16, .bound = 256 }, .{ .regs = 228, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_2_8: 226 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_2_8: 226 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_12_2_8: .usage = .{ .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 226, .static = 16, .bound = 256 }, .{ .regs = 226, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_2_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_2_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_2_8: 225 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_14_2_8: .usage = .{ .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 225, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_2_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_2_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_2_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_16_2_8: .usage = .{ .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_3_2_8: 159 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_3_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_3_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_3_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_3_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_3_2_8: .usage = .{ .{ .regs = 159, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_5_2_8: 160 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_5_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_5_2_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_5_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_5_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_5_2_8: .usage = .{ .{ .regs = 160, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_7_2_8: 179 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_7_2_8: 179 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_7_2_8: 179 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_7_2_8: 236 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_7_2_8: 236 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_7_2_8: .usage = .{ .{ .regs = 179, .static = 16, .bound = 256 }, .{ .regs = 179, .static = 16, .bound = 256 }, .{ .regs = 179, .static = 16, .bound = 256 }, .{ .regs = 236, .static = 16, .bound = 256 }, .{ .regs = 236, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_9_2_8: 176 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_9_2_8: 177 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_9_2_8: 177 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_9_2_8: 224 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_9_2_8: 224 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_9_2_8: .usage = .{ .{ .regs = 176, .static = 16, .bound = 256 }, .{ .regs = 177, .static = 16, .bound = 256 }, .{ .regs = 177, .static = 16, .bound = 256 }, .{ .regs = 224, .static = 16, .bound = 256 }, .{ .regs = 224, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_11_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_11_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_11_2_8: 181 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_11_2_8: 226 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_11_2_8: 226 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_11_2_8: .usage = .{ .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 181, .static = 16, .bound = 256 }, .{ .regs = 226, .static = 16, .bound = 256 }, .{ .regs = 226, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_13_2_8: 216 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_13_2_8: 216 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_13_2_8: 216 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_13_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_13_2_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_13_2_8: .usage = .{ .{ .regs = 216, .static = 16, .bound = 256 }, .{ .regs = 216, .static = 16, .bound = 256 }, .{ .regs = 216, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_1_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_1_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_1_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_2_1_8: .usage = .{ .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_1_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_1_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_1_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_4_1_8: .usage = .{ .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_1_8: 160 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_1_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_1_8: 161 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_6_1_8: .usage = .{ .{ .regs = 160, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 161, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_1_8: 233 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_1_8: 233 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_1_8: 233 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_8_1_8: .usage = .{ .{ .regs = 233, .static = 16, .bound = 256 }, .{ .regs = 233, .static = 16, .bound = 256 }, .{ .regs = 233, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_1_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_1_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_1_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_1_8: 241 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_1_8: 241 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_10_1_8: .usage = .{ .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 241, .static = 16, .bound = 256 }, .{ .regs = 241, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_1_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_1_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_1_8: 214 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_1_8: 240 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_1_8: 240 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_12_1_8: .usage = .{ .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 214, .static = 16, .bound = 256 }, .{ .regs = 240, .static = 16, .bound = 256 }, .{ .regs = 240, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_1_8: 227 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_1_8: 227 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_1_8: 227 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_14_1_8: .usage = .{ .{ .regs = 227, .static = 16, .bound = 256 }, .{ .regs = 227, .static = 16, .bound = 256 }, .{ .regs = 227, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_1_8: 220 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_1_8: 220 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_1_8: 220 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_1_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_16_1_8: .usage = .{ .{ .regs = 220, .static = 16, .bound = 256 }, .{ .regs = 220, .static = 16, .bound = 256 }, .{ .regs = 220, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_2_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_2_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_2_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_2_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_2_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_2_0_8: .usage = .{ .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_4_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_4_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_4_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_4_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_4_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_4_0_8: .usage = .{ .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_6_0_8: 174 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_6_0_8: 174 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_6_0_8: 174 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_6_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_6_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_6_0_8: .usage = .{ .{ .regs = 174, .static = 16, .bound = 256 }, .{ .regs = 174, .static = 16, .bound = 256 }, .{ .regs = 174, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_8_0_8: 255 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_8_0_8: 255 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_8_0_8: 255 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_8_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_8_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_8_0_8: .usage = .{ .{ .regs = 255, .static = 16, .bound = 256 }, .{ .regs = 255, .static = 16, .bound = 256 }, .{ .regs = 255, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_10_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_10_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_10_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_10_0_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_10_0_8: 231 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_10_0_8: .usage = .{ .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_12_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_12_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_12_0_8: 254 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_12_0_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_12_0_8: 231 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_12_0_8: .usage = .{ .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 254, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_14_0_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_14_0_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_14_0_8: 231 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_14_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_14_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_14_0_8: .usage = .{ .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 231, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_linear tf_exl3_linear_16_0_8: 220 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_linear tf_exl3_linear_16_0_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_linear tf_exl3_linear_16_0_8: 222 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_linear tf_exl3_linear_16_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_linear tf_exl3_linear_16_0_8: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD exl3_linear tf_exl3_linear_16_0_8: .usage = .{ .{ .regs = 220, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 222, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, .{ .regs = 128, .static = 16, .bound = 256 }, } +PASS limits sm_80 exl3_experts tf_exl3_group: 32 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_86 exl3_experts tf_exl3_group: 42 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_89 exl3_experts tf_exl3_group: 42 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_120 exl3_experts tf_exl3_group: 42 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_121 exl3_experts tf_exl3_group: 42 registers, 128 static shared bytes, bound 1024 (record current) +RECORD exl3_experts tf_exl3_group: .usage = .{ .{ .regs = 32, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, .{ .regs = 42, .static = 128, .bound = 1024 }, } +PASS limits sm_80 exl3_experts tf_exl3_experts_rot_in: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_experts_rot_in: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_experts_rot_in: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_experts_rot_in: 19 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_experts_rot_in: 19 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_experts_rot_in: .usage = .{ .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 19, .static = 0, .bound = 0 }, .{ .regs = 19, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_grouped_0: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_grouped_0: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_grouped_0: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_grouped_0: 127 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_grouped_0: 127 registers, 32832 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_grouped_0: .usage = .{ .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 127, .static = 32832, .bound = 0 }, .{ .regs = 127, .static = 32832, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_grouped_1: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_grouped_1: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_grouped_1: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_grouped_1: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_grouped_1: 128 registers, 32832 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_grouped_1: .usage = .{ .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_grouped_2: 128 registers, 32832 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_grouped_2: .usage = .{ .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, .{ .regs = 128, .static = 32832, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_gateup_epilogue: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_gateup_epilogue: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_gateup_epilogue: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_gateup_epilogue: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_gateup_epilogue: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_gateup_epilogue: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_down_combine: 32 registers, 16384 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_down_combine: 40 registers, 16384 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_down_combine: 40 registers, 16384 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_down_combine: 42 registers, 16384 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_down_combine: 42 registers, 16384 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_down_combine: .usage = .{ .{ .regs = 32, .static = 16384, .bound = 0 }, .{ .regs = 40, .static = 16384, .bound = 0 }, .{ .regs = 40, .static = 16384, .bound = 0 }, .{ .regs = 42, .static = 16384, .bound = 0 }, .{ .regs = 42, .static = 16384, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_plan_rotate: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_plan_rotate: 33 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_plan_rotate: 33 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_plan_rotate: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_plan_rotate: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_plan_rotate: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 33, .static = 0, .bound = 0 }, .{ .regs = 33, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 exl3_experts tf_exl3_plan_finish: 40 registers, 0 static shared bytes, bound 0 (record stale) +RECORD exl3_experts tf_exl3_plan_finish: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_prompt_rotate: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_experts tf_exl3_prompt_rotate: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_experts tf_exl3_prompt_rotate: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_experts tf_exl3_prompt_rotate: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_experts tf_exl3_prompt_rotate: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_experts tf_exl3_prompt_rotate: .usage = .{ .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_experts tf_exl3_prompt_gu: 128 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_experts tf_exl3_prompt_gu: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_experts tf_exl3_prompt_gu: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_experts tf_exl3_prompt_gu: 128 registers, 43264 static shared bytes, bound 256 (record stale) +PASS limits sm_121 exl3_experts tf_exl3_prompt_gu: 128 registers, 43264 static shared bytes, bound 256 (record stale) +RECORD exl3_experts tf_exl3_prompt_gu: .usage = .{ .{ .regs = 128, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 128, .static = 43264, .bound = 256 }, .{ .regs = 128, .static = 43264, .bound = 256 }, } +PASS limits sm_80 exl3_experts tf_exl3_prompt_down: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_experts tf_exl3_prompt_down: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_experts tf_exl3_prompt_down: 126 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_experts tf_exl3_prompt_down: 127 registers, 43264 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_experts tf_exl3_prompt_down: 127 registers, 43264 static shared bytes, bound 256 (record current) +RECORD exl3_experts tf_exl3_prompt_down: .usage = .{ .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 126, .static = 43264, .bound = 256 }, .{ .regs = 127, .static = 43264, .bound = 256 }, .{ .regs = 127, .static = 43264, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_tc_0: 128 registers, 8196 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_tc_0: .usage = .{ .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_tc_1: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_tc_1: 127 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_tc_1: 127 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_tc_1: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_tc_1: 128 registers, 8196 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_tc_1: .usage = .{ .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 127, .static = 8196, .bound = 256 }, .{ .regs = 127, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_tc_2: 128 registers, 8196 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_tc_2: .usage = .{ .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, .{ .regs = 128, .static = 8196, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_prompt_0: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_prompt_0: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_prompt_0: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_prompt_0: 75 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_prompt_0: 75 registers, 32768 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_prompt_0: .usage = .{ .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 75, .static = 32768, .bound = 256 }, .{ .regs = 75, .static = 32768, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_prompt_1: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_prompt_1: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_prompt_1: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_prompt_1: 77 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_prompt_1: 77 registers, 32768 static shared bytes, bound 256 (record current) +RECORD exl3_candidate tf_exl3_prompt_1: .usage = .{ .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 77, .static = 32768, .bound = 256 }, .{ .regs = 77, .static = 32768, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_prompt_2: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_prompt_2: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_prompt_2: 80 registers, 32768 static shared bytes, bound 256 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_prompt_2: 79 registers, 32768 static shared bytes, bound 256 (record stale) +PASS limits sm_121 exl3_candidate tf_exl3_prompt_2: 79 registers, 32768 static shared bytes, bound 256 (record stale) +RECORD exl3_candidate tf_exl3_prompt_2: .usage = .{ .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 80, .static = 32768, .bound = 256 }, .{ .regs = 79, .static = 32768, .bound = 256 }, .{ .regs = 79, .static = 32768, .bound = 256 }, } +PASS limits sm_80 exl3_candidate tf_exl3_rotate_f: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_rotate_f: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_rotate_f: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_rotate_f: 20 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_rotate_f: 20 registers, 0 static shared bytes, bound 32 (record current) +RECORD exl3_candidate tf_exl3_rotate_f: .usage = .{ .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 20, .static = 0, .bound = 32 }, .{ .regs = 20, .static = 0, .bound = 32 }, } +PASS limits sm_80 exl3_candidate tf_exl3_rotate_h: 26 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_rotate_h: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_rotate_h: 26 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_rotate_h: 20 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_rotate_h: 20 registers, 0 static shared bytes, bound 32 (record current) +RECORD exl3_candidate tf_exl3_rotate_h: .usage = .{ .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 20, .static = 0, .bound = 32 }, .{ .regs = 20, .static = 0, .bound = 32 }, } +PASS limits sm_80 exl3_candidate tf_exl3_codebook: 6 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_codebook: 6 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_codebook: 6 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_codebook: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_codebook: 8 registers, 0 static shared bytes, bound 0 (record current) +RECORD exl3_candidate tf_exl3_codebook: .usage = .{ .{ .regs = 6, .static = 0, .bound = 0 }, .{ .regs = 6, .static = 0, .bound = 0 }, .{ .regs = 6, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, } +PASS limits sm_80 exl3_candidate tf_exl3_unpack: 31 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 exl3_candidate tf_exl3_unpack: 31 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 exl3_candidate tf_exl3_unpack: 31 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 exl3_candidate tf_exl3_unpack: 21 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 exl3_candidate tf_exl3_unpack: 21 registers, 0 static shared bytes, bound 32 (record current) +RECORD exl3_candidate tf_exl3_unpack: .usage = .{ .{ .regs = 31, .static = 0, .bound = 32 }, .{ .regs = 31, .static = 0, .bound = 32 }, .{ .regs = 31, .static = 0, .bound = 32 }, .{ .regs = 21, .static = 0, .bound = 32 }, .{ .regs = 21, .static = 0, .bound = 32 }, } +PASS limits sm_80 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 experts3 decode pair: 255 registers, 0 static shared bytes, bound 128 (record current) +RECORD experts3 decode pair: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, } +PASS limits sm_80 experts3 decode down: 198 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 experts3 decode down: 199 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 experts3 decode down: 199 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 experts3 decode down: 194 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 experts3 decode down: 194 registers, 0 static shared bytes, bound 128 (record current) +RECORD experts3 decode down: .usage = .{ .{ .regs = 198, .static = 0, .bound = 128 }, .{ .regs = 199, .static = 0, .bound = 128 }, .{ .regs = 199, .static = 0, .bound = 128 }, .{ .regs = 194, .static = 0, .bound = 128 }, .{ .regs = 194, .static = 0, .bound = 128 }, } +PASS limits sm_80 experts3_prefill pair: 179 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 experts3_prefill pair: 179 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 experts3_prefill pair: 179 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 experts3_prefill pair: 150 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 experts3_prefill pair: 150 registers, 0 static shared bytes, bound 256 (record current) +RECORD experts3_prefill pair: .usage = .{ .{ .regs = 179, .static = 0, .bound = 256 }, .{ .regs = 179, .static = 0, .bound = 256 }, .{ .regs = 179, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, } +PASS limits sm_80 experts3_prefill down: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 experts3_prefill down: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 experts3_prefill down: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 experts3_prefill down: 98 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 experts3_prefill down: 98 registers, 0 static shared bytes, bound 256 (record current) +RECORD experts3_prefill down: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 98, .static = 0, .bound = 256 }, .{ .regs = 98, .static = 0, .bound = 256 }, } +PASS limits sm_80 experts3 tf_experts3_widen: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 experts3 tf_experts3_widen: 34 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 experts3 tf_experts3_widen: 34 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 experts3 tf_experts3_widen: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 experts3 tf_experts3_widen: 36 registers, 0 static shared bytes, bound 0 (record current) +RECORD experts3 tf_experts3_widen: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, } +PASS limits sm_80 experts3 tf_experts3_planes: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 experts3 tf_experts3_planes: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 experts3 tf_experts3_planes: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 experts3 tf_experts3_planes: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 experts3 tf_experts3_planes: 36 registers, 0 static shared bytes, bound 0 (record current) +RECORD experts3 tf_experts3_planes: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, } +PASS limits sm_80 h3 h3_norm: 25 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_norm: 26 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_norm: 26 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_norm: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_norm: 32 registers, 128 static shared bytes, bound 256 (record current) +RECORD h3 h3_norm: .usage = .{ .{ .regs = 25, .static = 128, .bound = 256 }, .{ .regs = 26, .static = 128, .bound = 256 }, .{ .regs = 26, .static = 128, .bound = 256 }, .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 32, .static = 128, .bound = 256 }, } +PASS limits sm_80 h3 h3_gate_add: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_gate_add: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_gate_add: 16 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_gate_add: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_gate_add: 22 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_gate_add: .usage = .{ .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_gate_mix: 18 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_gate_mix: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_gate_mix: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_gate_mix: 24 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_gate_mix: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_gate_mix: .usage = .{ .{ .regs = 18, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_swiglu: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_swiglu: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_swiglu: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_swiglu: 24 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_swiglu: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_swiglu: .usage = .{ .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_heads: 234 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_heads: 234 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_heads: 234 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_heads: 255 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_heads: 255 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_heads: .usage = .{ .{ .regs = 234, .static = 0, .bound = 256 }, .{ .regs = 234, .static = 0, .bound = 256 }, .{ .regs = 234, .static = 0, .bound = 256 }, .{ .regs = 255, .static = 0, .bound = 256 }, .{ .regs = 255, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_pool: 31 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 h3 h3_pool: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 h3 h3_pool: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 h3 h3_pool: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 h3 h3_pool: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD h3 h3_pool: .usage = .{ .{ .regs = 31, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 h3 h3_tile_scores: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 h3 h3_tile_scores: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 h3 h3_tile_scores: 20 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 h3 h3_tile_scores: 21 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 h3 h3_tile_scores: 21 registers, 0 static shared bytes, bound 256 (record current) +RECORD h3 h3_tile_scores: .usage = .{ .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 20, .static = 0, .bound = 256 }, .{ .regs = 21, .static = 0, .bound = 256 }, .{ .regs = 21, .static = 0, .bound = 256 }, } +PASS limits sm_80 h3 h3_coarse: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 h3 h3_coarse: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 h3 h3_coarse: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 h3 h3_coarse: 39 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 h3 h3_coarse: 39 registers, 0 static shared bytes, bound 0 (record current) +RECORD h3 h3_coarse: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, } +PASS limits sm_80 h3 h3_attention: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 h3 h3_attention: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 h3 h3_attention: 168 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 h3 h3_attention: 208 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 h3 h3_attention: 208 registers, 0 static shared bytes, bound 128 (record current) +RECORD h3 h3_attention: .usage = .{ .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 168, .static = 0, .bound = 128 }, .{ .regs = 208, .static = 0, .bound = 128 }, .{ .regs = 208, .static = 0, .bound = 128 }, } +PASS limits sm_80 h3 h3_topk: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 h3 h3_topk: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 h3 h3_topk: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 h3 h3_topk: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 h3 h3_topk: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD h3 h3_topk: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_gemv: 54 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 drama3 d3_gemv: 54 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 drama3 d3_gemv: 54 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 drama3 d3_gemv: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 drama3 d3_gemv: 64 registers, 0 static shared bytes, bound 256 (record current) +RECORD drama3 d3_gemv: .usage = .{ .{ .regs = 54, .static = 0, .bound = 256 }, .{ .regs = 54, .static = 0, .bound = 256 }, .{ .regs = 54, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 drama3 d3_rmsnorm: 19 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_rmsnorm: 20 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_rmsnorm: 20 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_rmsnorm: 18 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_rmsnorm: 18 registers, 144 static shared bytes, bound 0 (record current) +RECORD drama3 d3_rmsnorm: .usage = .{ .{ .regs = 19, .static = 144, .bound = 0 }, .{ .regs = 20, .static = 144, .bound = 0 }, .{ .regs = 20, .static = 144, .bound = 0 }, .{ .regs = 18, .static = 144, .bound = 0 }, .{ .regs = 18, .static = 144, .bound = 0 }, } +PASS limits sm_80 drama3 d3_add_rmsnorm: 19 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_add_rmsnorm: 21 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_add_rmsnorm: 21 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_add_rmsnorm: 24 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_add_rmsnorm: 24 registers, 144 static shared bytes, bound 0 (record current) +RECORD drama3 d3_add_rmsnorm: .usage = .{ .{ .regs = 19, .static = 144, .bound = 0 }, .{ .regs = 21, .static = 144, .bound = 0 }, .{ .regs = 21, .static = 144, .bound = 0 }, .{ .regs = 24, .static = 144, .bound = 0 }, .{ .regs = 24, .static = 144, .bound = 0 }, } +PASS limits sm_80 drama3 d3_embed_backbone: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_embed_backbone: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_embed_backbone: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_embed_backbone: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_embed_backbone: 24 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_embed_backbone: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_conv_silu: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_conv_silu: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_conv_silu: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_conv_silu: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_conv_silu: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_conv_silu: .usage = .{ .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +PASS limits sm_86 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +PASS limits sm_89 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +PASS limits sm_120 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +PASS limits sm_121 drama3 d3_gdn: 255 registers, 1024 static shared bytes, bound 32 (record current) +RECORD drama3 d3_gdn: .usage = .{ .{ .regs = 255, .static = 1024, .bound = 32 }, .{ .regs = 255, .static = 1024, .bound = 32 }, .{ .regs = 255, .static = 1024, .bound = 32 }, .{ .regs = 255, .static = 1024, .bound = 32 }, .{ .regs = 255, .static = 1024, .bound = 32 }, } +PASS limits sm_80 drama3 d3_gated_norm: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_gated_norm: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_gated_norm: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_gated_norm: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_gated_norm: 26 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_gated_norm: .usage = .{ .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_qk_rope: 28 registers, 1168 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_qk_rope: 28 registers, 1168 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_qk_rope: 28 registers, 1168 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_qk_rope: 26 registers, 1168 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_qk_rope: 26 registers, 1168 static shared bytes, bound 0 (record current) +RECORD drama3 d3_qk_rope: .usage = .{ .{ .regs = 28, .static = 1168, .bound = 0 }, .{ .regs = 28, .static = 1168, .bound = 0 }, .{ .regs = 28, .static = 1168, .bound = 0 }, .{ .regs = 26, .static = 1168, .bound = 0 }, .{ .regs = 26, .static = 1168, .bound = 0 }, } +PASS limits sm_80 drama3 d3_attn_decode: 38 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_86 drama3 d3_attn_decode: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_89 drama3 d3_attn_decode: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_120 drama3 d3_attn_decode: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_121 drama3 d3_attn_decode: 40 registers, 8256 static shared bytes, bound 256 (record current) +RECORD drama3 d3_attn_decode: .usage = .{ .{ .regs = 38, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, } +PASS limits sm_80 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_86 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_89 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_120 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +PASS limits sm_121 drama3 d3_attn_partial: 40 registers, 8256 static shared bytes, bound 256 (record current) +RECORD drama3 d3_attn_partial: .usage = .{ .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, .{ .regs = 40, .static = 8256, .bound = 256 }, } +PASS limits sm_80 drama3 d3_attn_merge: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_attn_merge: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_q_group: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_q_group: .usage = .{ .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_causal_softmax: 17 registers, 272 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_causal_softmax: 19 registers, 272 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_causal_softmax: 19 registers, 272 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_causal_softmax: 20 registers, 272 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_causal_softmax: 20 registers, 272 static shared bytes, bound 0 (record current) +RECORD drama3 d3_causal_softmax: .usage = .{ .{ .regs = 17, .static = 272, .bound = 0 }, .{ .regs = 19, .static = 272, .bound = 0 }, .{ .regs = 19, .static = 272, .bound = 0 }, .{ .regs = 20, .static = 272, .bound = 0 }, .{ .regs = 20, .static = 272, .bound = 0 }, } +PASS limits sm_80 drama3 d3_attn_gate: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_attn_gate: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_attn_gate: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_attn_gate: 19 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_attn_gate: 19 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_attn_gate: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 19, .static = 0, .bound = 0 }, .{ .regs = 19, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_bf16_f32: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_bf16_f32: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_bf16_f32: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_bf16_f32: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_bf16_f32: 16 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_bf16_f32: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_router: 17 registers, 592 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_router: 20 registers, 592 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_router: 20 registers, 592 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_router: 26 registers, 592 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_router: 26 registers, 592 static shared bytes, bound 0 (record current) +RECORD drama3 d3_router: .usage = .{ .{ .regs = 17, .static = 592, .bound = 0 }, .{ .regs = 20, .static = 592, .bound = 0 }, .{ .regs = 20, .static = 592, .bound = 0 }, .{ .regs = 26, .static = 592, .bound = 0 }, .{ .regs = 26, .static = 592, .bound = 0 }, } +PASS limits sm_80 drama3 d3_swiglu: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_swiglu: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_swiglu: .usage = .{ .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_moe_combine: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_moe_combine: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_moe_combine: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_moe_combine: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_moe_combine: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_moe_combine: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_act_quant: 23 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_act_quant: 25 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_act_quant: 23 registers, 144 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_act_quant: 22 registers, 144 static shared bytes, bound 0 (record stale) +PASS limits sm_121 drama3 d3_act_quant: 22 registers, 144 static shared bytes, bound 0 (record stale) +RECORD drama3 d3_act_quant: .usage = .{ .{ .regs = 23, .static = 144, .bound = 0 }, .{ .regs = 25, .static = 144, .bound = 0 }, .{ .regs = 23, .static = 144, .bound = 0 }, .{ .regs = 22, .static = 144, .bound = 0 }, .{ .regs = 22, .static = 144, .bound = 0 }, } +PASS limits sm_80 drama3 d3_tower_mix: 40 registers, 8336 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_tower_mix: 40 registers, 8336 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_tower_mix: 40 registers, 8336 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_tower_mix: 36 registers, 8336 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_tower_mix: 36 registers, 8336 static shared bytes, bound 0 (record current) +RECORD drama3 d3_tower_mix: .usage = .{ .{ .regs = 40, .static = 8336, .bound = 0 }, .{ .regs = 40, .static = 8336, .bound = 0 }, .{ .regs = 40, .static = 8336, .bound = 0 }, .{ .regs = 36, .static = 8336, .bound = 0 }, .{ .regs = 36, .static = 8336, .bound = 0 }, } +PASS limits sm_80 drama3 d3_sample_main: 31 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_86 drama3 d3_sample_main: 37 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_89 drama3 d3_sample_main: 37 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_120 drama3 d3_sample_main: 47 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_121 drama3 d3_sample_main: 47 registers, 400 static shared bytes, bound 1024 (record current) +RECORD drama3 d3_sample_main: .usage = .{ .{ .regs = 31, .static = 400, .bound = 1024 }, .{ .regs = 37, .static = 400, .bound = 1024 }, .{ .regs = 37, .static = 400, .bound = 1024 }, .{ .regs = 47, .static = 400, .bound = 1024 }, .{ .regs = 47, .static = 400, .bound = 1024 }, } +PASS limits sm_80 drama3 d3_sample_main_fast: 31 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_86 drama3 d3_sample_main_fast: 35 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_89 drama3 d3_sample_main_fast: 35 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_120 drama3 d3_sample_main_fast: 37 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_121 drama3 d3_sample_main_fast: 37 registers, 3616 static shared bytes, bound 1024 (record current) +RECORD drama3 d3_sample_main_fast: .usage = .{ .{ .regs = 31, .static = 3616, .bound = 1024 }, .{ .regs = 35, .static = 3616, .bound = 1024 }, .{ .regs = 35, .static = 3616, .bound = 1024 }, .{ .regs = 37, .static = 3616, .bound = 1024 }, .{ .regs = 37, .static = 3616, .bound = 1024 }, } +PASS limits sm_80 drama3 d3_sample_code: 30 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_86 drama3 d3_sample_code: 31 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_89 drama3 d3_sample_code: 31 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_120 drama3 d3_sample_code: 39 registers, 400 static shared bytes, bound 1024 (record current) +PASS limits sm_121 drama3 d3_sample_code: 39 registers, 400 static shared bytes, bound 1024 (record current) +RECORD drama3 d3_sample_code: .usage = .{ .{ .regs = 30, .static = 400, .bound = 1024 }, .{ .regs = 31, .static = 400, .bound = 1024 }, .{ .regs = 31, .static = 400, .bound = 1024 }, .{ .regs = 39, .static = 400, .bound = 1024 }, .{ .regs = 39, .static = 400, .bound = 1024 }, } +PASS limits sm_80 drama3 d3_sample_code_fast: 25 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_86 drama3 d3_sample_code_fast: 25 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_89 drama3 d3_sample_code_fast: 25 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_120 drama3 d3_sample_code_fast: 25 registers, 3616 static shared bytes, bound 1024 (record current) +PASS limits sm_121 drama3 d3_sample_code_fast: 25 registers, 3616 static shared bytes, bound 1024 (record current) +RECORD drama3 d3_sample_code_fast: .usage = .{ .{ .regs = 25, .static = 3616, .bound = 1024 }, .{ .regs = 25, .static = 3616, .bound = 1024 }, .{ .regs = 25, .static = 3616, .bound = 1024 }, .{ .regs = 25, .static = 3616, .bound = 1024 }, .{ .regs = 25, .static = 3616, .bound = 1024 }, } +PASS limits sm_80 drama3 d3_argmax_main: 12 registers, 256 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_argmax_main: 14 registers, 256 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_argmax_main: 14 registers, 256 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_argmax_main: 13 registers, 256 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_argmax_main: 13 registers, 256 static shared bytes, bound 0 (record current) +RECORD drama3 d3_argmax_main: .usage = .{ .{ .regs = 12, .static = 256, .bound = 0 }, .{ .regs = 14, .static = 256, .bound = 0 }, .{ .regs = 14, .static = 256, .bound = 0 }, .{ .regs = 13, .static = 256, .bound = 0 }, .{ .regs = 13, .static = 256, .bound = 0 }, } +PASS limits sm_80 drama3 d3_finish_frame: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_finish_frame: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_finish_frame: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_finish_frame: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_finish_frame: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_finish_frame: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_next_input: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_next_input: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3 d3_copy_i32: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3 d3_copy_i32: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3 d3_copy_i32: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3 d3_copy_i32: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3 d3_copy_i32: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3 d3_copy_i32: .usage = .{ .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_vq_gather: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_vq_gather: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_vq_gather: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_vq_gather: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_vq_gather: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_vq_gather: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_vq_encode: 31 registers, 424 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_vq_encode: 39 registers, 424 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_vq_encode: 39 registers, 424 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_vq_encode: 39 registers, 424 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_vq_encode: 39 registers, 424 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_vq_encode: .usage = .{ .{ .regs = 31, .static = 424, .bound = 0 }, .{ .regs = 39, .static = 424, .bound = 0 }, .{ .regs = 39, .static = 424, .bound = 0 }, .{ .regs = 39, .static = 424, .bound = 0 }, .{ .regs = 39, .static = 424, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_rmsnorm: 15 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_rmsnorm: 16 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_rmsnorm: 16 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_rmsnorm: 18 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_rmsnorm: 18 registers, 132 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_rmsnorm: .usage = .{ .{ .regs = 15, .static = 132, .bound = 0 }, .{ .regs = 16, .static = 132, .bound = 0 }, .{ .regs = 16, .static = 132, .bound = 0 }, .{ .regs = 18, .static = 132, .bound = 0 }, .{ .regs = 18, .static = 132, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_layernorm: 21 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_layernorm: 23 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_layernorm: 23 registers, 132 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_layernorm: 25 registers, 132 static shared bytes, bound 0 (record stale) +PASS limits sm_121 drama3_codec c_layernorm: 25 registers, 132 static shared bytes, bound 0 (record stale) +RECORD drama3_codec c_layernorm: .usage = .{ .{ .regs = 21, .static = 132, .bound = 0 }, .{ .regs = 23, .static = 132, .bound = 0 }, .{ .regs = 23, .static = 132, .bound = 0 }, .{ .regs = 25, .static = 132, .bound = 0 }, .{ .regs = 25, .static = 132, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_rope_kv: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_rope_kv: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_rope_kv: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_rope_kv: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_rope_kv: 24 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_rope_kv: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_attn: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_attn: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_attn: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_attn: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_attn: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_attn: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_add_gamma: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_add_gamma: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_add_gamma: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_add_gamma: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_add_gamma: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_add_gamma: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_silu_mul: 23 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_silu_mul: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_silu_mul: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_silu_mul: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_silu_mul: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_silu_mul: .usage = .{ .{ .regs = 23, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_gelu: 15 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_gelu: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_gelu: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_gelu: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_gelu: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_gelu: .usage = .{ .{ .regs = 15, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_bias: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_bias: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_bias: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_bias: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_bias: 26 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_bias: .usage = .{ .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_snake: 21 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_snake: 21 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_snake: 21 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_snake: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_snake: 26 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_snake: .usage = .{ .{ .regs = 21, .static = 0, .bound = 0 }, .{ .regs = 21, .static = 0, .bound = 0 }, .{ .regs = 21, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_dwconv7: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_dwconv7: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_dwconv7: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_dwconv7: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_dwconv7: 36 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_dwconv7: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_codec c_copy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 drama3_codec c_copy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 drama3_codec c_copy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 drama3_codec c_copy: 12 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 drama3_codec c_copy: 12 registers, 0 static shared bytes, bound 0 (record current) +RECORD drama3_codec c_copy: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, } +PASS limits sm_80 drama3_fp8 decode gate-up: 157 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 drama3_fp8 decode gate-up: 157 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 drama3_fp8 decode gate-up: 157 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 drama3_fp8 decode gate-up: 163 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 drama3_fp8 decode gate-up: 163 registers, 0 static shared bytes, bound 128 (record current) +RECORD drama3_fp8 decode gate-up: .usage = .{ .{ .regs = 157, .static = 0, .bound = 128 }, .{ .regs = 157, .static = 0, .bound = 128 }, .{ .regs = 157, .static = 0, .bound = 128 }, .{ .regs = 163, .static = 0, .bound = 128 }, .{ .regs = 163, .static = 0, .bound = 128 }, } +PASS limits sm_80 drama3_fp8 decode down: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 drama3_fp8 decode down: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 drama3_fp8 decode down: 116 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 drama3_fp8 decode down: 118 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 drama3_fp8 decode down: 118 registers, 0 static shared bytes, bound 128 (record current) +RECORD drama3_fp8 decode down: .usage = .{ .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 116, .static = 0, .bound = 128 }, .{ .regs = 118, .static = 0, .bound = 128 }, .{ .regs = 118, .static = 0, .bound = 128 }, } +PASS limits sm_80 drama3_fp8 prompt gate-up: 213 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_86 drama3_fp8 prompt gate-up: 212 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_89 drama3_fp8 prompt gate-up: 212 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_120 drama3_fp8 prompt gate-up: 212 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_121 drama3_fp8 prompt gate-up: 212 registers, 32768 static shared bytes, bound 128 (record current) +RECORD drama3_fp8 prompt gate-up: .usage = .{ .{ .regs = 213, .static = 32768, .bound = 128 }, .{ .regs = 212, .static = 32768, .bound = 128 }, .{ .regs = 212, .static = 32768, .bound = 128 }, .{ .regs = 212, .static = 32768, .bound = 128 }, .{ .regs = 212, .static = 32768, .bound = 128 }, } +PASS limits sm_80 drama3_fp8 prompt down: 213 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_86 drama3_fp8 prompt down: 211 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_89 drama3_fp8 prompt down: 211 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_120 drama3_fp8 prompt down: 213 registers, 32768 static shared bytes, bound 128 (record current) +PASS limits sm_121 drama3_fp8 prompt down: 213 registers, 32768 static shared bytes, bound 128 (record current) +RECORD drama3_fp8 prompt down: .usage = .{ .{ .regs = 213, .static = 32768, .bound = 128 }, .{ .regs = 211, .static = 32768, .bound = 128 }, .{ .regs = 211, .static = 32768, .bound = 128 }, .{ .regs = 213, .static = 32768, .bound = 128 }, .{ .regs = 213, .static = 32768, .bound = 128 }, } +PASS limits sm_80 router tf_nemo_router_tiles: 181 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 router tf_nemo_router_tiles: 184 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 router tf_nemo_router_tiles: 184 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 router tf_nemo_router_tiles: 183 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 router tf_nemo_router_tiles: 183 registers, 0 static shared bytes, bound 256 (record stale) +RECORD router tf_nemo_router_tiles: .usage = .{ .{ .regs = 181, .static = 0, .bound = 256 }, .{ .regs = 184, .static = 0, .bound = 256 }, .{ .regs = 184, .static = 0, .bound = 256 }, .{ .regs = 183, .static = 0, .bound = 256 }, .{ .regs = 183, .static = 0, .bound = 256 }, } +PASS limits sm_80 latent tf_super_combine_sums: 32 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 latent tf_super_combine_sums: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 latent tf_super_combine_sums: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 latent tf_super_combine_sums: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 latent tf_super_combine_sums: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD latent tf_super_combine_sums: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 latent tf_super_moe_out_sums: 26 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 latent tf_super_moe_out_sums: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 latent tf_super_moe_out_sums: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 latent tf_super_moe_out_sums: 24 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 latent tf_super_moe_out_sums: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD latent tf_super_moe_out_sums: .usage = .{ .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits ck tf_ck4_up_nvfp4_prompt sm_80: below_minimum +N/A limits ck tf_ck4_up_nvfp4_prompt sm_86: below_minimum +N/A limits ck tf_ck4_up_nvfp4_prompt sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_up_nvfp4_prompt: 115 registers, 46080 static shared bytes, bound 256 (record stale) +PASS limits sm_121 ck tf_ck4_up_nvfp4_prompt: 115 registers, 46080 static shared bytes, bound 256 (record stale) +RECORD ck tf_ck4_up_nvfp4_prompt: .usage = .{ .{ .regs = 115, .static = 46080, .bound = 256 }, .{ .regs = 115, .static = 46080, .bound = 256 }, } +N/A limits ck tf_ck4_down_f32_prompt sm_80: below_minimum +N/A limits ck tf_ck4_down_f32_prompt sm_86: below_minimum +N/A limits ck tf_ck4_down_f32_prompt sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_down_f32_prompt: 126 registers, 36864 static shared bytes, bound 256 (record current) +PASS limits sm_121 ck tf_ck4_down_f32_prompt: 126 registers, 36864 static shared bytes, bound 256 (record current) +RECORD ck tf_ck4_down_f32_prompt: .usage = .{ .{ .regs = 126, .static = 36864, .bound = 256 }, .{ .regs = 126, .static = 36864, .bound = 256 }, } +N/A limits ck tf_ck4_down_sum_prompt sm_80: below_minimum +N/A limits ck tf_ck4_down_sum_prompt sm_86: below_minimum +N/A limits ck tf_ck4_down_sum_prompt sm_89: below_minimum +PASS limits sm_120 ck tf_ck4_down_sum_prompt: 224 registers, 36864 static shared bytes, bound 256 (record current) +PASS limits sm_121 ck tf_ck4_down_sum_prompt: 224 registers, 36864 static shared bytes, bound 256 (record current) +RECORD ck tf_ck4_down_sum_prompt: .usage = .{ .{ .regs = 224, .static = 36864, .bound = 256 }, .{ .regs = 224, .static = 36864, .bound = 256 }, } +N/A limits ck tf_ck8_128 sm_80: below_minimum +N/A limits ck tf_ck8_128 sm_86: below_minimum +PASS limits sm_89 ck tf_ck8_128: 122 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 ck tf_ck8_128: 76 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 ck tf_ck8_128: 76 registers, 0 static shared bytes, bound 256 (record current) +RECORD ck tf_ck8_128: .usage = .{ .{ .regs = 122, .static = 0, .bound = 256 }, .{ .regs = 76, .static = 0, .bound = 256 }, .{ .regs = 76, .static = 0, .bound = 256 }, } +N/A limits ck tf_ck8_128c cluster of 8 sm_80: below_minimum +N/A limits ck tf_ck8_128c cluster of 8 sm_86: below_minimum +N/A limits ck tf_ck8_128c cluster of 8 sm_89: below_minimum +PASS limits sm_120 ck tf_ck8_128c cluster of 8: 104 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 ck tf_ck8_128c cluster of 8: 104 registers, 0 static shared bytes, bound 256 (record current) +RECORD ck tf_ck8_128c cluster of 8: .usage = .{ .{ .regs = 104, .static = 0, .bound = 256 }, .{ .regs = 104, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm16_16: 94 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 glm_mm16_16: 94 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 glm_mm16_16: 94 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 glm_mm16_16: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm16_16: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm16_16: .usage = .{ .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm16_32: 94 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 glm_mm16_32: 94 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 glm_mm16_32: 94 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 glm_mm16_32: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm16_32: 64 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm16_32: .usage = .{ .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm16_64: 95 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm16_64: 95 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm16_64: 95 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm16_64: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm16_64: 64 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm16_64: .usage = .{ .{ .regs = 95, .static = 0, .bound = 256 }, .{ .regs = 95, .static = 0, .bound = 256 }, .{ .regs = 95, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm16_128: 126 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_mm16_128: 126 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_mm16_128: 126 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_mm16_128: 100 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm16_128: 100 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm16_128: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, .{ .regs = 126, .static = 0, .bound = 256 }, .{ .regs = 126, .static = 0, .bound = 256 }, .{ .regs = 100, .static = 0, .bound = 256 }, .{ .regs = 100, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_tile16: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_tile16: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_tile16: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_tile16: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_tile16: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_tile16: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_mm4_16: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm4_16: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm4_16: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm4_16: 50 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm4_16: 50 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm4_16: .usage = .{ .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 50, .static = 0, .bound = 128 }, .{ .regs = 50, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm4_32: 122 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 glm_mm4_32: 120 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 glm_mm4_32: 120 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 glm_mm4_32: 65 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm4_32: 65 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm4_32: .usage = .{ .{ .regs = 122, .static = 0, .bound = 128 }, .{ .regs = 120, .static = 0, .bound = 128 }, .{ .regs = 120, .static = 0, .bound = 128 }, .{ .regs = 65, .static = 0, .bound = 128 }, .{ .regs = 65, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm4_64: 80 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm4_64: 80 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm4_64: 80 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm4_64: 61 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm4_64: 61 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm4_64: .usage = .{ .{ .regs = 80, .static = 0, .bound = 256 }, .{ .regs = 80, .static = 0, .bound = 256 }, .{ .regs = 80, .static = 0, .bound = 256 }, .{ .regs = 61, .static = 0, .bound = 256 }, .{ .regs = 61, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm4_128: 127 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_mm4_128: 124 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_mm4_128: 124 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_mm4_128: 106 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm4_128: 106 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm4_128: .usage = .{ .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 124, .static = 0, .bound = 256 }, .{ .regs = 124, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_tile4: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_tile4: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_tile4: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_tile4: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_tile4: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_tile4: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_mm6_16: 94 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm6_16: 92 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 glm_mm6_16: 92 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 glm_mm6_16: 54 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm6_16: 54 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm6_16: .usage = .{ .{ .regs = 94, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 92, .static = 0, .bound = 128 }, .{ .regs = 54, .static = 0, .bound = 128 }, .{ .regs = 54, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm6_32: 128 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 glm_mm6_32: 127 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 glm_mm6_32: 127 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 glm_mm6_32: 70 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm6_32: 70 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm6_32: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, .{ .regs = 127, .static = 0, .bound = 128 }, .{ .regs = 127, .static = 0, .bound = 128 }, .{ .regs = 70, .static = 0, .bound = 128 }, .{ .regs = 70, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm6_64: 97 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm6_64: 91 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm6_64: 91 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm6_64: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm6_64: 64 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm6_64: .usage = .{ .{ .regs = 97, .static = 0, .bound = 256 }, .{ .regs = 91, .static = 0, .bound = 256 }, .{ .regs = 91, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm6_128: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm6_128: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 glm_mm6_128: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 glm_mm6_128: 106 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm6_128: 106 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm6_128: .usage = .{ .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_tile6: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_tile6: 39 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_tile6: 39 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_tile6: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_tile6: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_tile6: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_mm8_16: 80 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 glm_mm8_16: 80 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 glm_mm8_16: 80 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 glm_mm8_16: 58 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm8_16: 58 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm8_16: .usage = .{ .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 58, .static = 0, .bound = 128 }, .{ .regs = 58, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm8_32: 126 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 glm_mm8_32: 124 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 glm_mm8_32: 124 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 glm_mm8_32: 74 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 glm_mm8_32: 74 registers, 0 static shared bytes, bound 128 (record current) +RECORD glm_mm8_32: .usage = .{ .{ .regs = 126, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 74, .static = 0, .bound = 128 }, .{ .regs = 74, .static = 0, .bound = 128 }, } +PASS limits sm_80 glm_mm8_64: 92 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 glm_mm8_64: 93 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_mm8_64: 93 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_mm8_64: 63 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm8_64: 63 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm8_64: .usage = .{ .{ .regs = 92, .static = 0, .bound = 256 }, .{ .regs = 93, .static = 0, .bound = 256 }, .{ .regs = 93, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm8_128: 125 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_mm8_128: 125 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_mm8_128: 125 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_mm8_128: 106 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm8_128: 106 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm8_128: .usage = .{ .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, .{ .regs = 106, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_tile8: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_tile8: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_tile8: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_tile8: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_tile8: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_tile8: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_mm16_rows: 126 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_mm16_rows: 126 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_mm16_rows: 126 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_mm16_rows: 99 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 glm_mm16_rows: 99 registers, 0 static shared bytes, bound 256 (record current) +RECORD glm_mm16_rows: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, .{ .regs = 126, .static = 0, .bound = 256 }, .{ .regs = 126, .static = 0, .bound = 256 }, .{ .regs = 99, .static = 0, .bound = 256 }, .{ .regs = 99, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_mm_reduce: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 glm_mm_reduce: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 glm_mm_reduce: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 glm_mm_reduce: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 glm_mm_reduce: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD glm_mm_reduce: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_tap: 15 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_tap: 15 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_tap: 15 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_tap: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_tap: 36 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_tap: .usage = .{ .{ .regs = 15, .static = 0, .bound = 0 }, .{ .regs = 15, .static = 0, .bound = 0 }, .{ .regs = 15, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_block_qkv: 32 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_block_qkv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_block_qkv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_block_qkv: 32 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_block_qkv: 32 registers, 8 static shared bytes, bound 0 (record current) +RECORD df_block_qkv: .usage = .{ .{ .regs = 32, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 32, .static = 8, .bound = 0 }, .{ .regs = 32, .static = 8, .bound = 0 }, } +PASS limits sm_80 df_ctx_kv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_ctx_kv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_ctx_kv: 30 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_ctx_kv: 28 registers, 8 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_ctx_kv: 28 registers, 8 static shared bytes, bound 0 (record current) +RECORD df_ctx_kv: .usage = .{ .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 28, .static = 8, .bound = 0 }, .{ .regs = 28, .static = 8, .bound = 0 }, } +PASS limits sm_80 df_conv: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_conv: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_conv: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_conv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_conv: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_conv: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_swiglu: 22 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_swiglu: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_swiglu: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_swiglu: .usage = .{ .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_attn_part: 74 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 df_attn_part: 77 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 df_attn_part: 77 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 df_attn_part: 75 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 df_attn_part: 75 registers, 0 static shared bytes, bound 256 (record current) +RECORD df_attn_part: .usage = .{ .{ .regs = 74, .static = 0, .bound = 256 }, .{ .regs = 77, .static = 0, .bound = 256 }, .{ .regs = 77, .static = 0, .bound = 256 }, .{ .regs = 75, .static = 0, .bound = 256 }, .{ .regs = 75, .static = 0, .bound = 256 }, } +PASS limits sm_80 df_attn_merge: 39 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_attn_merge: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_attn_merge: .usage = .{ .{ .regs = 39, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_sum2: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_sum2: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_sum2: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_sum2: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_sum2: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_sum2: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 df_top16_part: 32 registers, 32864 static shared bytes, bound 256 (record current) +PASS limits sm_86 df_top16_part: 32 registers, 32864 static shared bytes, bound 256 (record current) +PASS limits sm_89 df_top16_part: 32 registers, 32864 static shared bytes, bound 256 (record current) +PASS limits sm_120 df_top16_part: 30 registers, 32864 static shared bytes, bound 256 (record current) +PASS limits sm_121 df_top16_part: 30 registers, 32864 static shared bytes, bound 256 (record current) +RECORD df_top16_part: .usage = .{ .{ .regs = 32, .static = 32864, .bound = 256 }, .{ .regs = 32, .static = 32864, .bound = 256 }, .{ .regs = 32, .static = 32864, .bound = 256 }, .{ .regs = 30, .static = 32864, .bound = 256 }, .{ .regs = 30, .static = 32864, .bound = 256 }, } +PASS limits sm_80 df_top16_merge: 23 registers, 8416 static shared bytes, bound 256 (record current) +PASS limits sm_86 df_top16_merge: 23 registers, 8416 static shared bytes, bound 256 (record current) +PASS limits sm_89 df_top16_merge: 23 registers, 8416 static shared bytes, bound 256 (record current) +PASS limits sm_120 df_top16_merge: 28 registers, 8416 static shared bytes, bound 256 (record current) +PASS limits sm_121 df_top16_merge: 28 registers, 8416 static shared bytes, bound 256 (record current) +RECORD df_top16_merge: .usage = .{ .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 28, .static = 8416, .bound = 256 }, .{ .regs = 28, .static = 8416, .bound = 256 }, } +PASS limits sm_80 df_gsum: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 df_gsum: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 df_gsum: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 df_gsum: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 df_gsum: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD df_gsum: .usage = .{ .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_glue tf_glm_lat_write: 13 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 glm_glue tf_glm_lat_write: 13 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 glm_glue tf_glm_lat_write: 13 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 glm_glue tf_glm_lat_write: 34 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 glm_glue tf_glm_lat_write: 34 registers, 0 static shared bytes, bound 0 (record stale) +RECORD glm_glue tf_glm_lat_write: .usage = .{ .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, } +PASS limits sm_80 glm_glue tf_glm_pool_keys: 25 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_86 glm_glue tf_glm_pool_keys: 26 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_89 glm_glue tf_glm_pool_keys: 26 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_120 glm_glue tf_glm_pool_keys: 24 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 glm_glue tf_glm_pool_keys: 24 registers, 0 static shared bytes, bound 32 (record stale) +RECORD glm_glue tf_glm_pool_keys: .usage = .{ .{ .regs = 25, .static = 0, .bound = 32 }, .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 24, .static = 0, .bound = 32 }, .{ .regs = 24, .static = 0, .bound = 32 }, } +PASS limits sm_80 glm_glue tf_glm_scores: 122 registers, 32768 static shared bytes, bound 128 (record stale) +PASS limits sm_86 glm_glue tf_glm_scores: 129 registers, 32768 static shared bytes, bound 128 (record stale) +PASS limits sm_89 glm_glue tf_glm_scores: 129 registers, 32768 static shared bytes, bound 128 (record stale) +PASS limits sm_120 glm_glue tf_glm_scores: 157 registers, 32768 static shared bytes, bound 128 (record stale) +PASS limits sm_121 glm_glue tf_glm_scores: 157 registers, 32768 static shared bytes, bound 128 (record stale) +RECORD glm_glue tf_glm_scores: .usage = .{ .{ .regs = 122, .static = 32768, .bound = 128 }, .{ .regs = 129, .static = 32768, .bound = 128 }, .{ .regs = 129, .static = 32768, .bound = 128 }, .{ .regs = 157, .static = 32768, .bound = 128 }, .{ .regs = 157, .static = 32768, .bound = 128 }, } +PASS limits sm_80 glm_glue tf_glm_chunks_16: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_glue tf_glm_chunks_16: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_glue tf_glm_chunks_16: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_glue tf_glm_chunks_16: 123 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 glm_glue tf_glm_chunks_16: 123 registers, 0 static shared bytes, bound 256 (record stale) +RECORD glm_glue tf_glm_chunks_16: .usage = .{ .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 123, .static = 0, .bound = 256 }, .{ .regs = 123, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_glue tf_glm_chunks_32: 150 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_glue tf_glm_chunks_32: 150 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_glue tf_glm_chunks_32: 150 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_glue tf_glm_chunks_32: 128 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 glm_glue tf_glm_chunks_32: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD glm_glue tf_glm_chunks_32: .usage = .{ .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_ops tf_glm_dsa_tokens: 25 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 glm_ops tf_glm_dsa_tokens: 25 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 glm_ops tf_glm_dsa_tokens: 25 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 glm_ops tf_glm_dsa_tokens: 22 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 glm_ops tf_glm_dsa_tokens: 22 registers, 0 static shared bytes, bound 0 (record stale) +RECORD glm_ops tf_glm_dsa_tokens: .usage = .{ .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree attention tf_tree_paths: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 tree attention tf_tree_paths: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 tree attention tf_tree_paths: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 tree attention tf_tree_paths: 22 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 tree attention tf_tree_paths: 22 registers, 0 static shared bytes, bound 0 (record stale) +RECORD tree attention tf_tree_paths: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree attention tf_tree_shared: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 tree attention tf_tree_shared: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 tree attention tf_tree_shared: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 tree attention tf_tree_shared: 213 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 tree attention tf_tree_shared: 213 registers, 0 static shared bytes, bound 128 (record stale) +RECORD tree attention tf_tree_shared: .usage = .{ .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 213, .static = 0, .bound = 128 }, .{ .regs = 213, .static = 0, .bound = 128 }, } +PASS limits sm_80 tree attention tf_tree_shared_w1: 193 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_shared_w1: 193 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_shared_w1: 193 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_shared_w1: 168 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 tree attention tf_tree_shared_w1: 168 registers, 0 static shared bytes, bound 32 (record stale) +RECORD tree attention tf_tree_shared_w1: .usage = .{ .{ .regs = 193, .static = 0, .bound = 32 }, .{ .regs = 193, .static = 0, .bound = 32 }, .{ .regs = 193, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_shared_w1s3: 190 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_shared_w1s3: 190 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_shared_w1s3: 190 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_shared_w1s3: 168 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 tree attention tf_tree_shared_w1s3: 168 registers, 0 static shared bytes, bound 32 (record stale) +RECORD tree attention tf_tree_shared_w1s3: .usage = .{ .{ .regs = 190, .static = 0, .bound = 32 }, .{ .regs = 190, .static = 0, .bound = 32 }, .{ .regs = 190, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_part: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 tree attention tf_tree_part: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 tree attention tf_tree_part: 202 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 tree attention tf_tree_part: 228 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 tree attention tf_tree_part: 228 registers, 0 static shared bytes, bound 128 (record current) +RECORD tree attention tf_tree_part: .usage = .{ .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 202, .static = 0, .bound = 128 }, .{ .regs = 228, .static = 0, .bound = 128 }, .{ .regs = 228, .static = 0, .bound = 128 }, } +PASS limits sm_80 tree attention tf_tree_part_w1: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_part_w1: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_part_w1: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_part_w1: 168 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_part_w1: 168 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_part_w1: .usage = .{ .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_part_w1s3: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_part_w1s3: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_part_w1s3: 197 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_part_w1s3: 168 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 tree attention tf_tree_part_w1s3: 168 registers, 0 static shared bytes, bound 32 (record stale) +RECORD tree attention tf_tree_part_w1s3: .usage = .{ .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 197, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, .{ .regs = 168, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 tree attention tf_tree_shared_256: 255 registers, 0 static shared bytes, bound 128 (record current) +RECORD tree attention tf_tree_shared_256: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, } +PASS limits sm_80 tree attention tf_tree_shared_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_shared_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_shared_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_shared_256_w1: 253 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 tree attention tf_tree_shared_256_w1: 253 registers, 0 static shared bytes, bound 32 (record stale) +RECORD tree attention tf_tree_shared_256_w1: .usage = .{ .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 253, .static = 0, .bound = 32 }, .{ .regs = 253, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_shared_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_shared_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_shared_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_shared_256_w1s3: 252 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 tree attention tf_tree_shared_256_w1s3: 252 registers, 0 static shared bytes, bound 32 (record stale) +RECORD tree attention tf_tree_shared_256_w1s3: .usage = .{ .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 252, .static = 0, .bound = 32 }, .{ .regs = 252, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 tree attention tf_tree_part_256: 255 registers, 0 static shared bytes, bound 128 (record current) +RECORD tree attention tf_tree_part_256: .usage = .{ .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, .{ .regs = 255, .static = 0, .bound = 128 }, } +PASS limits sm_80 tree attention tf_tree_part_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_part_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_part_256_w1: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_part_256_w1: 255 registers, 0 static shared bytes, bound 32 (record stale) +PASS limits sm_121 tree attention tf_tree_part_256_w1: 255 registers, 0 static shared bytes, bound 32 (record stale) +RECORD tree attention tf_tree_part_256_w1: .usage = .{ .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_part_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tree attention tf_tree_part_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tree attention tf_tree_part_256_w1s3: 255 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tree attention tf_tree_part_256_w1s3: 251 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tree attention tf_tree_part_256_w1s3: 251 registers, 0 static shared bytes, bound 32 (record current) +RECORD tree attention tf_tree_part_256_w1s3: .usage = .{ .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 255, .static = 0, .bound = 32 }, .{ .regs = 251, .static = 0, .bound = 32 }, .{ .regs = 251, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree attention tf_tree_merge: 114 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 tree attention tf_tree_merge: 114 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 tree attention tf_tree_merge: 114 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 tree attention tf_tree_merge: 110 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 tree attention tf_tree_merge: 110 registers, 0 static shared bytes, bound 256 (record stale) +RECORD tree attention tf_tree_merge: .usage = .{ .{ .regs = 114, .static = 0, .bound = 256 }, .{ .regs = 114, .static = 0, .bound = 256 }, .{ .regs = 114, .static = 0, .bound = 256 }, .{ .regs = 110, .static = 0, .bound = 256 }, .{ .regs = 110, .static = 0, .bound = 256 }, } +PASS limits sm_80 block attention tf_block_attention (up to 8 query heads a key head): 230 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 block attention tf_block_attention (up to 8 query heads a key head): 230 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 block attention tf_block_attention (up to 8 query heads a key head): 230 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 block attention tf_block_attention (up to 8 query heads a key head): 236 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 block attention tf_block_attention (up to 8 query heads a key head): 236 registers, 0 static shared bytes, bound 256 (record stale) +RECORD block attention tf_block_attention (up to 8 query heads a key head): .usage = .{ .{ .regs = 230, .static = 0, .bound = 256 }, .{ .regs = 230, .static = 0, .bound = 256 }, .{ .regs = 230, .static = 0, .bound = 256 }, .{ .regs = 236, .static = 0, .bound = 256 }, .{ .regs = 236, .static = 0, .bound = 256 }, } +PASS limits sm_80 window attention tf_window_part: 55 registers, 45824 static shared bytes, bound 256 (record current) +PASS limits sm_86 window attention tf_window_part: 50 registers, 45824 static shared bytes, bound 256 (record current) +PASS limits sm_89 window attention tf_window_part: 50 registers, 45824 static shared bytes, bound 256 (record current) +PASS limits sm_120 window attention tf_window_part: 54 registers, 45824 static shared bytes, bound 256 (record stale) +PASS limits sm_121 window attention tf_window_part: 54 registers, 45824 static shared bytes, bound 256 (record stale) +RECORD window attention tf_window_part: .usage = .{ .{ .regs = 55, .static = 45824, .bound = 256 }, .{ .regs = 50, .static = 45824, .bound = 256 }, .{ .regs = 50, .static = 45824, .bound = 256 }, .{ .regs = 54, .static = 45824, .bound = 256 }, .{ .regs = 54, .static = 45824, .bound = 256 }, } +PASS limits sm_80 window attention tf_window_merge: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 window attention tf_window_merge: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 window attention tf_window_merge: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 window attention tf_window_merge: 38 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 window attention tf_window_merge: 38 registers, 0 static shared bytes, bound 256 (record stale) +RECORD window attention tf_window_merge: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, } +PASS limits sm_80 window attention tf_window_decode_tc: 176 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 window attention tf_window_decode_tc: 176 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 window attention tf_window_decode_tc: 176 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 window attention tf_window_decode_tc: 182 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 window attention tf_window_decode_tc: 182 registers, 0 static shared bytes, bound 128 (record stale) +RECORD window attention tf_window_decode_tc: .usage = .{ .{ .regs = 176, .static = 0, .bound = 128 }, .{ .regs = 176, .static = 0, .bound = 128 }, .{ .regs = 176, .static = 0, .bound = 128 }, .{ .regs = 182, .static = 0, .bound = 128 }, .{ .regs = 182, .static = 0, .bound = 128 }, } +PASS limits sm_80 window attention wattn_kernel<128, 8, 4, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 window attention wattn_kernel<128, 8, 4, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 window attention wattn_kernel<128, 8, 4, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 window attention wattn_kernel<128, 8, 4, 8>: 218 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 window attention wattn_kernel<128, 8, 4, 8>: 218 registers, 0 static shared bytes, bound 256 (record stale) +RECORD window attention wattn_kernel<128, 8, 4, 8>: .usage = .{ .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 218, .static = 0, .bound = 256 }, .{ .regs = 218, .static = 0, .bound = 256 }, } +PASS limits sm_80 window attention wattn_kernel<128, 8, 8, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 window attention wattn_kernel<128, 8, 8, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 window attention wattn_kernel<128, 8, 8, 8>: 223 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 window attention wattn_kernel<128, 8, 8, 8>: 218 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 window attention wattn_kernel<128, 8, 8, 8>: 218 registers, 0 static shared bytes, bound 256 (record stale) +RECORD window attention wattn_kernel<128, 8, 8, 8>: .usage = .{ .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 223, .static = 0, .bound = 256 }, .{ .regs = 218, .static = 0, .bound = 256 }, .{ .regs = 218, .static = 0, .bound = 256 }, } +PASS limits sm_80 moe tf_moe_topk_sigmoid: 32 registers, 16704 static shared bytes, bound 256 (record current) +PASS limits sm_86 moe tf_moe_topk_sigmoid: 48 registers, 16704 static shared bytes, bound 256 (record current) +PASS limits sm_89 moe tf_moe_topk_sigmoid: 48 registers, 16704 static shared bytes, bound 256 (record current) +PASS limits sm_120 moe tf_moe_topk_sigmoid: 48 registers, 16704 static shared bytes, bound 256 (record current) +PASS limits sm_121 moe tf_moe_topk_sigmoid: 48 registers, 16704 static shared bytes, bound 256 (record current) +RECORD moe tf_moe_topk_sigmoid: .usage = .{ .{ .regs = 32, .static = 16704, .bound = 256 }, .{ .regs = 48, .static = 16704, .bound = 256 }, .{ .regs = 48, .static = 16704, .bound = 256 }, .{ .regs = 48, .static = 16704, .bound = 256 }, .{ .regs = 48, .static = 16704, .bound = 256 }, } +PASS limits sm_80 moe tf_moe_topk_sigmoid_warp: 63 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 moe tf_moe_topk_sigmoid_warp: 63 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 moe tf_moe_topk_sigmoid_warp: 63 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 moe tf_moe_topk_sigmoid_warp: 63 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 moe tf_moe_topk_sigmoid_warp: 63 registers, 0 static shared bytes, bound 256 (record stale) +RECORD moe tf_moe_topk_sigmoid_warp: .usage = .{ .{ .regs = 63, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, .{ .regs = 63, .static = 0, .bound = 256 }, } +PASS limits sm_80 moe tf_moe_combine: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 moe tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 moe tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 moe tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 moe tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD moe tf_moe_combine: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 dense rows tf_dense_rows_1: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 dense rows tf_dense_rows_1: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 dense rows tf_dense_rows_1: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 dense rows tf_dense_rows_1: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 dense rows tf_dense_rows_1: 38 registers, 0 static shared bytes, bound 256 (record current) +RECORD dense rows tf_dense_rows_1: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, } +PASS limits sm_80 dense rows tf_dense_rows_4: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 dense rows tf_dense_rows_4: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 dense rows tf_dense_rows_4: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 dense rows tf_dense_rows_4: 40 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 dense rows tf_dense_rows_4: 40 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dense rows tf_dense_rows_4: .usage = .{ .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 dense rows tf_dense_rows_16: 79 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 dense rows tf_dense_rows_16: 79 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 dense rows tf_dense_rows_16: 79 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 dense rows tf_dense_rows_16: 96 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 dense rows tf_dense_rows_16: 96 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dense rows tf_dense_rows_16: .usage = .{ .{ .regs = 79, .static = 0, .bound = 256 }, .{ .regs = 79, .static = 0, .bound = 256 }, .{ .regs = 79, .static = 0, .bound = 256 }, .{ .regs = 96, .static = 0, .bound = 256 }, .{ .regs = 96, .static = 0, .bound = 256 }, } +PASS limits sm_80 dense rows tf_dense_rows_32: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 dense rows tf_dense_rows_32: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 dense rows tf_dense_rows_32: 127 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 dense rows tf_dense_rows_32: 180 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 dense rows tf_dense_rows_32: 180 registers, 0 static shared bytes, bound 256 (record current) +RECORD dense rows tf_dense_rows_32: .usage = .{ .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 180, .static = 0, .bound = 256 }, .{ .regs = 180, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_geglu_rows 256 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 act_rows tf_geglu_rows 256 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 act_rows tf_geglu_rows 256 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 act_rows tf_geglu_rows 256 threads: 33 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 act_rows tf_geglu_rows 256 threads: 33 registers, 0 static shared bytes, bound 256 (record current) +RECORD act_rows tf_geglu_rows 256 threads: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_geglu_rows 128 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 act_rows tf_geglu_rows 128 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 act_rows tf_geglu_rows 128 threads: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 act_rows tf_geglu_rows 128 threads: 33 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 act_rows tf_geglu_rows 128 threads: 33 registers, 0 static shared bytes, bound 256 (record current) +RECORD act_rows tf_geglu_rows 128 threads: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 act_rows tf_swiglu_rows_round 256 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +RECORD act_rows tf_swiglu_rows_round 256 threads: .usage = .{ .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 act_rows tf_swiglu_rows_round 128 threads: 34 registers, 0 static shared bytes, bound 256 (record current) +RECORD act_rows tf_swiglu_rows_round 128 threads: .usage = .{ .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 embed_rows tf_embed_rows_affine: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 embed_rows tf_embed_rows_affine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 embed_rows tf_embed_rows_affine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 embed_rows tf_embed_rows_affine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 embed_rows tf_embed_rows_affine: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD embed_rows tf_embed_rows_affine: .usage = .{ .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 qk_rows tf_qk_rows_once 512 threads: 30 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_86 qk_rows tf_qk_rows_once 512 threads: 36 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_89 qk_rows tf_qk_rows_once 512 threads: 36 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_120 qk_rows tf_qk_rows_once 512 threads: 34 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_121 qk_rows tf_qk_rows_once 512 threads: 34 registers, 64 static shared bytes, bound 512 (record current) +RECORD qk_rows tf_qk_rows_once 512 threads: .usage = .{ .{ .regs = 30, .static = 64, .bound = 512 }, .{ .regs = 36, .static = 64, .bound = 512 }, .{ .regs = 36, .static = 64, .bound = 512 }, .{ .regs = 34, .static = 64, .bound = 512 }, .{ .regs = 34, .static = 64, .bound = 512 }, } +PASS limits sm_80 qk_rows tf_qk_rows_round 512 threads: 30 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_86 qk_rows tf_qk_rows_round 512 threads: 36 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_89 qk_rows tf_qk_rows_round 512 threads: 36 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_120 qk_rows tf_qk_rows_round 512 threads: 34 registers, 64 static shared bytes, bound 512 (record current) +PASS limits sm_121 qk_rows tf_qk_rows_round 512 threads: 34 registers, 64 static shared bytes, bound 512 (record current) +RECORD qk_rows tf_qk_rows_round 512 threads: .usage = .{ .{ .regs = 30, .static = 64, .bound = 512 }, .{ .regs = 36, .static = 64, .bound = 512 }, .{ .regs = 36, .static = 64, .bound = 512 }, .{ .regs = 34, .static = 64, .bound = 512 }, .{ .regs = 34, .static = 64, .bound = 512 }, } +PASS limits sm_80 norms tf_add_rms_rows_bf16h_tail: 94 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_86 norms tf_add_rms_rows_bf16h_tail: 95 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_89 norms tf_add_rms_rows_bf16h_tail: 95 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_120 norms tf_add_rms_rows_bf16h_tail: 147 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_121 norms tf_add_rms_rows_bf16h_tail: 147 registers, 64 static shared bytes, bound 256 (record current) +RECORD norms tf_add_rms_rows_bf16h_tail: .usage = .{ .{ .regs = 94, .static = 64, .bound = 256 }, .{ .regs = 95, .static = 64, .bound = 256 }, .{ .regs = 95, .static = 64, .bound = 256 }, .{ .regs = 147, .static = 64, .bound = 256 }, .{ .regs = 147, .static = 64, .bound = 256 }, } +PASS limits sm_80 norms tf_add_rms_rows_bf16h_tail_round: 94 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_86 norms tf_add_rms_rows_bf16h_tail_round: 95 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_89 norms tf_add_rms_rows_bf16h_tail_round: 95 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_120 norms tf_add_rms_rows_bf16h_tail_round: 147 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_121 norms tf_add_rms_rows_bf16h_tail_round: 147 registers, 64 static shared bytes, bound 256 (record current) +RECORD norms tf_add_rms_rows_bf16h_tail_round: .usage = .{ .{ .regs = 94, .static = 64, .bound = 256 }, .{ .regs = 95, .static = 64, .bound = 256 }, .{ .regs = 95, .static = 64, .bound = 256 }, .{ .regs = 147, .static = 64, .bound = 256 }, .{ .regs = 147, .static = 64, .bound = 256 }, } +PASS limits sm_80 flashnext_fact factored prompt matmul 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_fact factored prompt matmul 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_fact factored prompt matmul 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_fact factored prompt matmul 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_fact factored prompt matmul 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_fact factored prompt matmul 64x128: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_fact factored prompt matmul 64x128 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_fact factored prompt matmul 64x128 folded: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_hcf hyper-connection down 32x192 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hcf hyper-connection down 32x192 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hcf hyper-connection down 32x192 folded: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hcf hyper-connection down 32x192 folded: 126 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hcf hyper-connection down 32x192 folded: 126 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_hcf hyper-connection down 32x192 folded: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 126, .static = 0, .bound = 256 }, .{ .regs = 126, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hcf hyper-connection up 64x128: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_hcf hyper-connection up 64x128: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_fexp factored experts gate and up: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_fexp factored experts gate and up: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_fexp factored experts gate and up: 125 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_fexp factored experts gate and up: 116 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_fexp factored experts gate and up: 116 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_fexp factored experts gate and up: .usage = .{ .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 125, .static = 0, .bound = 256 }, .{ .regs = 116, .static = 0, .bound = 256 }, .{ .regs = 116, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_fexp factored experts down: 136 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_fexp factored experts down: 136 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_fexp factored experts down: 136 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_fexp factored experts down: 120 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_fexp factored experts down: 120 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_fexp factored experts down: .usage = .{ .{ .regs = 136, .static = 0, .bound = 256 }, .{ .regs = 136, .static = 0, .bound = 256 }, .{ .regs = 136, .static = 0, .bound = 256 }, .{ .regs = 120, .static = 0, .bound = 256 }, .{ .regs = 120, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_gdn_chain_tpr: 129 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_batch fn_gdn_chain_tpr: 129 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_batch fn_gdn_chain_tpr: 129 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_batch fn_gdn_chain_tpr: 152 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_batch fn_gdn_chain_tpr: 152 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_batch fn_gdn_chain_tpr: .usage = .{ .{ .regs = 129, .static = 0, .bound = 256 }, .{ .regs = 129, .static = 0, .bound = 256 }, .{ .regs = 129, .static = 0, .bound = 256 }, .{ .regs = 152, .static = 0, .bound = 256 }, .{ .regs = 152, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_attn qsa_tile scores: 222 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_attn qsa_tile scores: 222 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_attn qsa_tile scores: 222 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_attn qsa_tile scores: 224 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_attn qsa_tile scores: 224 registers, 16384 static shared bytes, bound 256 (record current) +RECORD flashnext_attn qsa_tile scores: .usage = .{ .{ .regs = 222, .static = 16384, .bound = 256 }, .{ .regs = 222, .static = 16384, .bound = 256 }, .{ .regs = 222, .static = 16384, .bound = 256 }, .{ .regs = 224, .static = 16384, .bound = 256 }, .{ .regs = 224, .static = 16384, .bound = 256 }, } +PASS limits sm_80 flashnext_attn fn_attn_fused: 118 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_attn_fused: 118 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_attn_fused: 118 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_attn_fused: 122 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_attn_fused: 122 registers, 640 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_attn_fused: .usage = .{ .{ .regs = 118, .static = 640, .bound = 128 }, .{ .regs = 118, .static = 640, .bound = 128 }, .{ .regs = 118, .static = 640, .bound = 128 }, .{ .regs = 122, .static = 640, .bound = 128 }, .{ .regs = 122, .static = 640, .bound = 128 }, } +PASS limits sm_80 flashnext_attn fn_attn_chunks_one: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_attn fn_attn_chunks_one: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_attn fn_attn_chunks_one: 72 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_attn fn_attn_chunks_one: 96 registers, 640 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_attn fn_attn_chunks_one: 96 registers, 640 static shared bytes, bound 128 (record current) +RECORD flashnext_attn fn_attn_chunks_one: .usage = .{ .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 72, .static = 640, .bound = 128 }, .{ .regs = 96, .static = 640, .bound = 128 }, .{ .regs = 96, .static = 640, .bound = 128 }, } +PASS limits sm_80 flashnext_hc fn_hc_rinv: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_hc fn_hc_rinv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_hc fn_hc_rinv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_hc fn_hc_rinv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_hc fn_hc_rinv: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_hc fn_hc_rinv: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_hc_mix_h: 27 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_mix_h: 27 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_mix_h: 27 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_mix_h: 38 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_mix_h: 38 registers, 1024 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_mix_h: .usage = .{ .{ .regs = 27, .static = 1024, .bound = 256 }, .{ .regs = 27, .static = 1024, .bound = 256 }, .{ .regs = 27, .static = 1024, .bound = 256 }, .{ .regs = 38, .static = 1024, .bound = 256 }, .{ .regs = 38, .static = 1024, .bound = 256 }, } +PASS limits sm_80 route_rows tf_route_logits_wide: 47 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 route_rows tf_route_logits_wide: 47 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 route_rows tf_route_logits_wide: 47 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 route_rows tf_route_logits_wide: 58 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 route_rows tf_route_logits_wide: 58 registers, 0 static shared bytes, bound 256 (record current) +RECORD route_rows tf_route_logits_wide: .usage = .{ .{ .regs = 47, .static = 0, .bound = 256 }, .{ .regs = 47, .static = 0, .bound = 256 }, .{ .regs = 47, .static = 0, .bound = 256 }, .{ .regs = 58, .static = 0, .bound = 256 }, .{ .regs = 58, .static = 0, .bound = 256 }, } +PASS limits sm_80 gdn_train tf_gdn_front threads 128: 32 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_86 gdn_train tf_gdn_front threads 128: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_89 gdn_train tf_gdn_front threads 128: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_120 gdn_train tf_gdn_front threads 128: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_121 gdn_train tf_gdn_front threads 128: 40 registers, 1024 static shared bytes, bound 128 (record current) +RECORD gdn_train tf_gdn_front threads 128: .usage = .{ .{ .regs = 32, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, } +PASS limits sm_80 gdn_train tf_gdn_scan_fwd threads 256: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 gdn_train tf_gdn_scan_fwd threads 256: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 gdn_train tf_gdn_scan_fwd threads 256: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 gdn_train tf_gdn_scan_fwd threads 256: 60 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 gdn_train tf_gdn_scan_fwd threads 256: 60 registers, 1152 static shared bytes, bound 256 (record current) +RECORD gdn_train tf_gdn_scan_fwd threads 256: .usage = .{ .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 60, .static = 1152, .bound = 256 }, .{ .regs = 60, .static = 1152, .bound = 256 }, } +PASS limits sm_80 gdn_train tf_gdn_norm_back threads 128: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_86 gdn_train tf_gdn_norm_back threads 128: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_89 gdn_train tf_gdn_norm_back threads 128: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_120 gdn_train tf_gdn_norm_back threads 128: 24 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_121 gdn_train tf_gdn_norm_back threads 128: 24 registers, 644 static shared bytes, bound 128 (record current) +RECORD gdn_train tf_gdn_norm_back threads 128: .usage = .{ .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 24, .static = 644, .bound = 128 }, .{ .regs = 24, .static = 644, .bound = 128 }, } +PASS limits sm_80 gdn_train tf_gdn_scan_back threads 256: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_86 gdn_train tf_gdn_scan_back threads 256: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_89 gdn_train tf_gdn_scan_back threads 256: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_120 gdn_train tf_gdn_scan_back threads 256: 102 registers, 9536 static shared bytes, bound 256 (record stale) +PASS limits sm_121 gdn_train tf_gdn_scan_back threads 256: 102 registers, 9536 static shared bytes, bound 256 (record stale) +RECORD gdn_train tf_gdn_scan_back threads 256: .usage = .{ .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 102, .static = 9536, .bound = 256 }, .{ .regs = 102, .static = 9536, .bound = 256 }, } +PASS limits sm_80 gdn_train tf_gdn_qk_back threads 128: 32 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_86 gdn_train tf_gdn_qk_back threads 128: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_89 gdn_train tf_gdn_qk_back threads 128: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_120 gdn_train tf_gdn_qk_back threads 128: 40 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_121 gdn_train tf_gdn_qk_back threads 128: 40 registers, 128 static shared bytes, bound 128 (record current) +RECORD gdn_train tf_gdn_qk_back threads 128: .usage = .{ .{ .regs = 32, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, } +PASS limits sm_80 gdn_train tf_gdn_conv_back threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_conv_back threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_conv_back threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_conv_back threads 256: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_conv_back threads 256: 35 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_conv_back threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_dequant threads 256: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_dequant threads 256: .usage = .{ .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 32: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 32: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 32: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 32: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 32: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 32: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 64: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 64: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 64: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 64: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 64: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 64: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 96: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 96: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 96: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 96: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 96: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 96: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 128: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 128: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 128: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 128: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 128: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 128: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 160: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 160: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 160: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 160: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 160: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 160: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 192: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 192: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 192: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 192: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 192: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 192: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 224: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 224: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 224: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 224: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 224: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 224: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 256: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 256: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 256: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 256: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 288: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 288: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 288: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 288: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 288: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 288: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 320: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 320: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 320: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 320: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 320: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 320: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 352: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 352: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 352: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 352: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 352: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 352: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 384: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 384: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 384: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 384: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 384: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 384: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 416: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 416: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 416: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 416: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 416: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 416: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 448: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 448: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 448: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 448: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 448: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 448: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 480: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 480: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 480: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 480: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 480: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 480: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 512: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 512: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 512: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 512: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 512: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 512: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 544: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 544: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 544: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 544: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 544: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 544: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 576: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 576: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 576: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 576: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 576: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 576: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 608: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 608: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 608: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 608: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 608: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 608: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 640: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 640: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 640: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 640: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 640: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 640: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 672: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 672: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 672: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 672: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 672: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 672: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 704: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 704: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 704: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 704: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 704: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 704: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 736: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 736: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 736: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 736: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 736: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 736: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 768: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 768: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 768: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 768: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 768: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 768: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 800: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 800: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 800: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 800: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 800: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 800: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 832: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 832: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 832: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 832: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 832: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 832: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 864: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 864: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 864: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 864: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 864: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 864: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 896: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 896: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 896: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 896: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 896: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 896: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 928: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 928: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 928: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 928: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 928: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 928: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 960: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 960: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 960: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 960: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 960: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 960: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 992: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 992: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 992: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 992: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 992: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 992: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 gdn_train tf_gdn_gates_back threads 1024: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 gdn_train tf_gdn_gates_back threads 1024: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 gdn_train tf_gdn_gates_back threads 1024: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 gdn_train tf_gdn_gates_back threads 1024: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 gdn_train tf_gdn_gates_back threads 1024: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD gdn_train tf_gdn_gates_back threads 1024: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_q4 tf_q4_back threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 train_q4 tf_q4_back threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 train_q4 tf_q4_back threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 train_q4 tf_q4_back threads 128: 60 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 train_q4 tf_q4_back threads 128: 60 registers, 10240 static shared bytes, bound 128 (record current) +RECORD train_q4 tf_q4_back threads 128: .usage = .{ .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 60, .static = 10240, .bound = 128 }, .{ .regs = 60, .static = 10240, .bound = 128 }, } +PASS limits sm_80 train_q4 tf_q4_back_bf threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 train_q4 tf_q4_back_bf threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 train_q4 tf_q4_back_bf threads 128: 54 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 train_q4 tf_q4_back_bf threads 128: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 train_q4 tf_q4_back_bf threads 128: 56 registers, 10240 static shared bytes, bound 128 (record current) +RECORD train_q4 tf_q4_back_bf threads 128: .usage = .{ .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 54, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, } +PASS limits sm_80 train_q4 tf_q4_sum_parts threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train_q4 tf_q4_sum_parts threads 256: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train_q4 tf_q4_sum_parts threads 256: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train_q4 tf_q4_sum_parts threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train_q4 tf_q4_sum_parts threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train_q4 tf_q4_sum_parts threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_q4 tf_q4_experts_mm threads 64: 72 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_86 train_q4 tf_q4_experts_mm threads 64: 90 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_89 train_q4 tf_q4_experts_mm threads 64: 90 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_120 train_q4 tf_q4_experts_mm threads 64: 79 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_121 train_q4 tf_q4_experts_mm threads 64: 79 registers, 10304 static shared bytes, bound 64 (record current) +RECORD train_q4 tf_q4_experts_mm threads 64: .usage = .{ .{ .regs = 72, .static = 10304, .bound = 64 }, .{ .regs = 90, .static = 10304, .bound = 64 }, .{ .regs = 90, .static = 10304, .bound = 64 }, .{ .regs = 79, .static = 10304, .bound = 64 }, .{ .regs = 79, .static = 10304, .bound = 64 }, } +PASS limits sm_80 train_q4 tf_q4_experts_nt threads 64: 72 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_86 train_q4 tf_q4_experts_nt threads 64: 91 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_89 train_q4 tf_q4_experts_nt threads 64: 91 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_120 train_q4 tf_q4_experts_nt threads 64: 78 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_121 train_q4 tf_q4_experts_nt threads 64: 78 registers, 10816 static shared bytes, bound 64 (record current) +RECORD train_q4 tf_q4_experts_nt threads 64: .usage = .{ .{ .regs = 72, .static = 10816, .bound = 64 }, .{ .regs = 91, .static = 10816, .bound = 64 }, .{ .regs = 91, .static = 10816, .bound = 64 }, .{ .regs = 78, .static = 10816, .bound = 64 }, .{ .regs = 78, .static = 10816, .bound = 64 }, } +PASS limits sm_80 train_q4 tf_q4_swiglu_back threads 128: 40 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_86 train_q4 tf_q4_swiglu_back threads 128: 40 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_89 train_q4 tf_q4_swiglu_back threads 128: 40 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_120 train_q4 tf_q4_swiglu_back threads 128: 46 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_121 train_q4 tf_q4_swiglu_back threads 128: 46 registers, 128 static shared bytes, bound 128 (record current) +RECORD train_q4 tf_q4_swiglu_back threads 128: .usage = .{ .{ .regs = 40, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, .{ .regs = 46, .static = 128, .bound = 128 }, .{ .regs = 46, .static = 128, .bound = 128 }, } +PASS limits sm_80 train_q4 tf_q4_experts_nt_packed threads 64: 72 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_86 train_q4 tf_q4_experts_nt_packed threads 64: 83 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_89 train_q4 tf_q4_experts_nt_packed threads 64: 83 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_120 train_q4 tf_q4_experts_nt_packed threads 64: 74 registers, 10816 static shared bytes, bound 64 (record current) +PASS limits sm_121 train_q4 tf_q4_experts_nt_packed threads 64: 74 registers, 10816 static shared bytes, bound 64 (record current) +RECORD train_q4 tf_q4_experts_nt_packed threads 64: .usage = .{ .{ .regs = 72, .static = 10816, .bound = 64 }, .{ .regs = 83, .static = 10816, .bound = 64 }, .{ .regs = 83, .static = 10816, .bound = 64 }, .{ .regs = 74, .static = 10816, .bound = 64 }, .{ .regs = 74, .static = 10816, .bound = 64 }, } +PASS limits sm_80 train_q4 tf_q4_experts_mm_packed threads 64: 72 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_86 train_q4 tf_q4_experts_mm_packed threads 64: 81 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_89 train_q4 tf_q4_experts_mm_packed threads 64: 81 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_120 train_q4 tf_q4_experts_mm_packed threads 64: 80 registers, 10304 static shared bytes, bound 64 (record current) +PASS limits sm_121 train_q4 tf_q4_experts_mm_packed threads 64: 80 registers, 10304 static shared bytes, bound 64 (record current) +RECORD train_q4 tf_q4_experts_mm_packed threads 64: .usage = .{ .{ .regs = 72, .static = 10304, .bound = 64 }, .{ .regs = 81, .static = 10304, .bound = 64 }, .{ .regs = 81, .static = 10304, .bound = 64 }, .{ .regs = 80, .static = 10304, .bound = 64 }, .{ .regs = 80, .static = 10304, .bound = 64 }, } +PASS limits sm_80 train_q4 tf_q4_route_back threads 256: 32 registers, 136 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_q4 tf_q4_route_back threads 256: 40 registers, 136 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_q4 tf_q4_route_back threads 256: 40 registers, 136 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_q4 tf_q4_route_back threads 256: 40 registers, 136 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_q4 tf_q4_route_back threads 256: 40 registers, 136 static shared bytes, bound 256 (record current) +RECORD train_q4 tf_q4_route_back threads 256: .usage = .{ .{ .regs = 32, .static = 136, .bound = 256 }, .{ .regs = 40, .static = 136, .bound = 256 }, .{ .regs = 40, .static = 136, .bound = 256 }, .{ .regs = 40, .static = 136, .bound = 256 }, .{ .regs = 40, .static = 136, .bound = 256 }, } +PASS limits sm_80 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train_q4 tf_q4_axpy threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD train_q4 tf_q4_axpy threads 256: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_q threads 256: 32 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_q threads 256: 47 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_q threads 256: 47 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_q threads 256: 48 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_q threads 256: 48 registers, 18560 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_q threads 256: .usage = .{ .{ .regs = 32, .static = 18560, .bound = 256 }, .{ .regs = 47, .static = 18560, .bound = 256 }, .{ .regs = 47, .static = 18560, .bound = 256 }, .{ .regs = 48, .static = 18560, .bound = 256 }, .{ .regs = 48, .static = 18560, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train_attn tf_gattn_gate_back threads 256: 17 registers, 0 static shared bytes, bound 0 (record current) +RECORD train_attn tf_gattn_gate_back threads 256: .usage = .{ .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 32: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 32: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 32: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 32: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 32: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 32: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 32: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 32: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 32: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 32: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 32: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 32: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 32: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 32: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 32: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 32: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 32: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 32: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 64: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 64: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 64: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 64: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 64: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 64: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 64: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 64: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 64: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 64: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 64: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 64: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 64: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 64: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 64: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 64: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 64: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 64: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 96: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 96: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 96: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 96: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 96: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 96: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 96: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 96: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 96: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 96: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 96: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 96: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 96: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 96: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 96: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 96: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 96: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 96: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 128: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 128: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 128: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 128: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 128: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 128: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 128: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 128: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 128: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 128: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 128: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 128: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 128: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 128: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 128: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 128: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 128: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 128: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 160: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 160: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 160: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 160: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 160: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 160: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 160: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 160: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 160: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 160: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 160: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 160: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 160: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 160: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 160: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 160: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 160: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 160: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 192: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 192: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 192: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 192: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 192: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 192: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 192: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 192: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 192: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 192: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 192: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 192: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 192: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 192: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 192: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 192: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 192: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 192: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 224: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 224: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 224: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 224: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 224: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 224: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 224: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 224: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 224: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 224: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 224: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 224: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 224: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 224: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 224: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 224: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 224: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 224: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train_attn tf_gattn_prep threads 256: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep threads 256: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep threads 256: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep threads 256: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep threads 256: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep threads 256: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train_attn tf_gattn_prep_back threads 256: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 train_attn tf_gattn_prep_back threads 256: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 train_attn tf_gattn_prep_back threads 256: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 train_attn tf_gattn_prep_back threads 256: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 train_attn tf_gattn_prep_back threads 256: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD train_attn tf_gattn_prep_back threads 256: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 train tf_train_attn_kv threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_attn_kv threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_attn_kv threads 256: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_attn_kv threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_attn_kv threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_attn_kv threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_softmax threads 1024: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_86 train tf_train_softmax threads 1024: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_89 train tf_train_softmax threads 1024: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_120 train tf_train_softmax threads 1024: 41 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_121 train tf_train_softmax threads 1024: 41 registers, 128 static shared bytes, bound 1024 (record current) +RECORD train tf_train_softmax threads 1024: .usage = .{ .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 41, .static = 128, .bound = 1024 }, .{ .regs = 41, .static = 128, .bound = 1024 }, } +PASS limits sm_80 train tf_train_logits threads 256: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 train tf_train_logits threads 256: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 train tf_train_logits threads 256: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 train tf_train_logits threads 256: 33 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 train tf_train_logits threads 256: 33 registers, 0 static shared bytes, bound 256 (record current) +RECORD train tf_train_logits threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, } +PASS limits sm_80 train tf_train_sort_pairs threads 256: 32 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_86 train tf_train_sort_pairs threads 256: 40 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_89 train tf_train_sort_pairs threads 256: 40 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_120 train tf_train_sort_pairs threads 256: 40 registers, 4096 static shared bytes, bound 256 (record current) +PASS limits sm_121 train tf_train_sort_pairs threads 256: 40 registers, 4096 static shared bytes, bound 256 (record current) +RECORD train tf_train_sort_pairs threads 256: .usage = .{ .{ .regs = 32, .static = 4096, .bound = 256 }, .{ .regs = 40, .static = 4096, .bound = 256 }, .{ .regs = 40, .static = 4096, .bound = 256 }, .{ .regs = 40, .static = 4096, .bound = 256 }, .{ .regs = 40, .static = 4096, .bound = 256 }, } +PASS limits sm_80 train tf_train_experts_tiles threads 1: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_experts_tiles threads 1: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_experts_tiles threads 1: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_experts_tiles threads 1: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_experts_tiles threads 1: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_experts_tiles threads 1: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_pairs_out threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_pairs_out threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_pairs_out threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_pairs_out threads 256: 34 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_pairs_out threads 256: 34 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_pairs_out threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_zero threads 256: 8 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_zero threads 256: .usage = .{ .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 train tf_train_narrow threads 256: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD train tf_train_narrow threads 256: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 train tf_train_gemm threads 128: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 train tf_train_gemm threads 128: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 train tf_train_gemm threads 128: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 train tf_train_gemm threads 128: 64 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 train tf_train_gemm threads 128: 64 registers, 10240 static shared bytes, bound 128 (record current) +RECORD train tf_train_gemm threads 128: .usage = .{ .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 64, .static = 10240, .bound = 128 }, .{ .regs = 64, .static = 10240, .bound = 128 }, } +PASS limits sm_80 flashnext_train tf_fn_hc_branch_back threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_hc_branch_back threads 256: 48 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_hc_branch_back threads 256: 48 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_hc_branch_back threads 256: 48 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_hc_branch_back threads 256: 48 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_hc_branch_back threads 256: .usage = .{ .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 48, .static = 128, .bound = 256 }, .{ .regs = 48, .static = 128, .bound = 256 }, .{ .regs = 48, .static = 128, .bound = 256 }, .{ .regs = 48, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_hc_mix_back threads 256: 29 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_train tf_fn_hc_mix_back threads 256: 37 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_train tf_fn_hc_mix_back threads 256: 37 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_train tf_fn_hc_mix_back threads 256: 34 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_train tf_fn_hc_mix_back threads 256: 34 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_train tf_fn_hc_mix_back threads 256: .usage = .{ .{ .regs = 29, .static = 0, .bound = 0 }, .{ .regs = 37, .static = 0, .bound = 0 }, .{ .regs = 37, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_train tf_fn_hc_act_back threads 256: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_hc_act_back threads 256: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_hc_act_back threads 256: 30 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_hc_act_back threads 256: 28 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_hc_act_back threads 256: 28 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_hc_act_back threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 30, .static = 0, .bound = 256 }, .{ .regs = 28, .static = 0, .bound = 256 }, .{ .regs = 28, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_hc_norm_back threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_hc_norm_back threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_hc_norm_back threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_hc_norm_back threads 256: 38 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_hc_norm_back threads 256: 38 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_hc_norm_back threads 256: .usage = .{ .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 38, .static = 128, .bound = 256 }, .{ .regs = 38, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_in threads 128: 32 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_in threads 128: 33 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_in threads 128: 33 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_in threads 128: 31 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_in threads 128: 31 registers, 0 static shared bytes, bound 128 (record current) +RECORD flashnext_train tf_fn_pocket_in threads 128: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 33, .static = 0, .bound = 128 }, .{ .regs = 33, .static = 0, .bound = 128 }, .{ .regs = 31, .static = 0, .bound = 128 }, .{ .regs = 31, .static = 0, .bound = 128 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_in_rows threads 256: 64 registers, 0 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_pocket_in_rows threads 256: .usage = .{ .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_out threads 128: 48 registers, 512 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_out threads 128: 48 registers, 512 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_out threads 128: 48 registers, 512 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_out threads 128: 64 registers, 512 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_out threads 128: 64 registers, 512 static shared bytes, bound 128 (record current) +RECORD flashnext_train tf_fn_pocket_out threads 128: .usage = .{ .{ .regs = 48, .static = 512, .bound = 128 }, .{ .regs = 48, .static = 512, .bound = 128 }, .{ .regs = 48, .static = 512, .bound = 128 }, .{ .regs = 64, .static = 512, .bound = 128 }, .{ .regs = 64, .static = 512, .bound = 128 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_out_rows threads 128: 72 registers, 2048 static shared bytes, bound 128 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_out_rows threads 128: 72 registers, 2048 static shared bytes, bound 128 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_out_rows threads 128: 72 registers, 2048 static shared bytes, bound 128 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_out_rows threads 128: 72 registers, 2048 static shared bytes, bound 128 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_out_rows threads 128: 72 registers, 2048 static shared bytes, bound 128 (record current) +RECORD flashnext_train tf_fn_pocket_out_rows threads 128: .usage = .{ .{ .regs = 72, .static = 2048, .bound = 128 }, .{ .regs = 72, .static = 2048, .bound = 128 }, .{ .regs = 72, .static = 2048, .bound = 128 }, .{ .regs = 72, .static = 2048, .bound = 128 }, .{ .regs = 72, .static = 2048, .bound = 128 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_back threads 256: 42 registers, 1536 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_back threads 256: 40 registers, 1536 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_back threads 256: 40 registers, 1536 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_back threads 256: 40 registers, 1536 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_back threads 256: 40 registers, 1536 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_pocket_back threads 256: .usage = .{ .{ .regs = 42, .static = 1536, .bound = 256 }, .{ .regs = 40, .static = 1536, .bound = 256 }, .{ .regs = 40, .static = 1536, .bound = 256 }, .{ .regs = 40, .static = 1536, .bound = 256 }, .{ .regs = 40, .static = 1536, .bound = 256 }, } +PASS limits sm_80 flashnext_train tf_fn_pocket_grad threads 256: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_train tf_fn_pocket_grad threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_train tf_fn_pocket_grad threads 256: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_train tf_fn_pocket_grad threads 256: 37 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_train tf_fn_pocket_grad threads 256: 37 registers, 128 static shared bytes, bound 256 (record current) +RECORD flashnext_train tf_fn_pocket_grad threads 256: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 37, .static = 128, .bound = 256 }, .{ .regs = 37, .static = 128, .bound = 256 }, } +PASS limits sm_80 flashnext_batch fn_moe_sum threads 256: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 flashnext_batch fn_moe_sum threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 flashnext_batch fn_moe_sum threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 flashnext_batch fn_moe_sum threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 flashnext_batch fn_moe_sum threads 256: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD flashnext_batch fn_moe_sum threads 256: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 flashnext_hc fn_hc_upmix_up threads 256: 69 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_86 flashnext_hc fn_hc_upmix_up threads 256: 64 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_89 flashnext_hc fn_hc_upmix_up threads 256: 64 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_120 flashnext_hc fn_hc_upmix_up threads 256: 62 registers, 2048 static shared bytes, bound 256 (record current) +PASS limits sm_121 flashnext_hc fn_hc_upmix_up threads 256: 62 registers, 2048 static shared bytes, bound 256 (record current) +RECORD flashnext_hc fn_hc_upmix_up threads 256: .usage = .{ .{ .regs = 69, .static = 2048, .bound = 256 }, .{ .regs = 64, .static = 2048, .bound = 256 }, .{ .regs = 64, .static = 2048, .bound = 256 }, .{ .regs = 62, .static = 2048, .bound = 256 }, .{ .regs = 62, .static = 2048, .bound = 256 }, } +PASS limits sm_80 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_train_zero: 8 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_train_zero: .usage = .{ .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_gemm_q4w_bf: 128 registers, 37888 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_gemm_q4w_bf: .usage = .{ .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_gemm_q4w: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_gemm_q4w: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_gemm_q4w: 128 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_gemm_q4w: 126 registers, 37888 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_gemm_q4w: 126 registers, 37888 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_gemm_q4w: .usage = .{ .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 128, .static = 37888, .bound = 256 }, .{ .regs = 126, .static = 37888, .bound = 256 }, .{ .regs = 126, .static = 37888, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_gemm_q4_bf: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_q27_gemm_q4_bf: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_q27_gemm_q4_bf: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_q27_gemm_q4_bf: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_q27_gemm_q4_bf: 56 registers, 10240 static shared bytes, bound 128 (record current) +RECORD 27B tf_q27_gemm_q4_bf: .usage = .{ .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, } +PASS limits sm_80 27B tf_q27_gemm_q4: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_q27_gemm_q4: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_q27_gemm_q4: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_q27_gemm_q4: 56 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_q27_gemm_q4: 56 registers, 10240 static shared bytes, bound 128 (record current) +RECORD 27B tf_q27_gemm_q4: .usage = .{ .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, .{ .regs = 56, .static = 10240, .bound = 128 }, } +PASS limits sm_80 27B tf_sum_parts: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_sum_parts: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_sum_parts: 36 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_sum_parts: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_sum_parts: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_sum_parts: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_fit_step: 40 registers, 5120 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_fit_step: 40 registers, 5120 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_fit_step: 40 registers, 5120 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_fit_step: 39 registers, 5120 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_fit_step: 39 registers, 5120 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_fit_step: .usage = .{ .{ .regs = 40, .static = 5120, .bound = 256 }, .{ .regs = 40, .static = 5120, .bound = 256 }, .{ .regs = 40, .static = 5120, .bound = 256 }, .{ .regs = 39, .static = 5120, .bound = 256 }, .{ .regs = 39, .static = 5120, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_zero_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_q27_zero_rows: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_q27_zero_rows: 13 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_q27_zero_rows: 12 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_q27_zero_rows: 12 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_q27_zero_rows: .usage = .{ .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, .{ .regs = 12, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_train_softmax: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_86 27B tf_train_softmax: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_89 27B tf_train_softmax: 28 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_120 27B tf_train_softmax: 41 registers, 128 static shared bytes, bound 1024 (record current) +PASS limits sm_121 27B tf_train_softmax: 41 registers, 128 static shared bytes, bound 1024 (record current) +RECORD 27B tf_train_softmax: .usage = .{ .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 28, .static = 128, .bound = 1024 }, .{ .regs = 41, .static = 128, .bound = 1024 }, .{ .regs = 41, .static = 128, .bound = 1024 }, } +PASS limits sm_80 27B tf_train_rms_back: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_train_rms_back: 39 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_train_rms_back: 39 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_train_rms_back: 37 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_train_rms_back: 37 registers, 128 static shared bytes, bound 256 (record current) +RECORD 27B tf_train_rms_back: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 39, .static = 128, .bound = 256 }, .{ .regs = 39, .static = 128, .bound = 256 }, .{ .regs = 37, .static = 128, .bound = 256 }, .{ .regs = 37, .static = 128, .bound = 256 }, } +PASS limits sm_80 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_axpy: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_axpy: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_add_bf16: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_q27_add_bf16: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_q27_add_bf16: 8 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_q27_add_bf16: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_q27_add_bf16: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_q27_add_bf16: .usage = .{ .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 8, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_q27_cols: 16 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_q27_cols: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_q27_pocket_rows: 34 registers, 512 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_pocket_rows: 40 registers, 512 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_pocket_rows: 40 registers, 512 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_pocket_rows: 40 registers, 512 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_pocket_rows: 40 registers, 512 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_pocket_rows: .usage = .{ .{ .regs = 34, .static = 512, .bound = 256 }, .{ .regs = 40, .static = 512, .bound = 256 }, .{ .regs = 40, .static = 512, .bound = 256 }, .{ .regs = 40, .static = 512, .bound = 256 }, .{ .regs = 40, .static = 512, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_pocket_back: 44 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_pocket_back: 42 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_pocket_back: 42 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_pocket_back: 40 registers, 1024 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_pocket_back: 40 registers, 1024 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_pocket_back: .usage = .{ .{ .regs = 44, .static = 1024, .bound = 256 }, .{ .regs = 42, .static = 1024, .bound = 256 }, .{ .regs = 42, .static = 1024, .bound = 256 }, .{ .regs = 40, .static = 1024, .bound = 256 }, .{ .regs = 40, .static = 1024, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_pocket_grad: 32 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_pocket_grad: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_pocket_grad: 40 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_pocket_grad: 37 registers, 128 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_pocket_grad: 37 registers, 128 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_pocket_grad: .usage = .{ .{ .regs = 32, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 40, .static = 128, .bound = 256 }, .{ .regs = 37, .static = 128, .bound = 256 }, .{ .regs = 37, .static = 128, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_swiglu_back: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_q27_swiglu_back: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_q27_swiglu_back: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_q27_swiglu_back: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_q27_swiglu_back: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_q27_swiglu_back: .usage = .{ .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_attn_prep: 30 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_attn_prep: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_attn_prep: 29 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_attn_prep: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_attn_prep: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD 27B tf_attn_prep: .usage = .{ .{ .regs = 30, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 29, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_attn_gate_back: 17 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_attn_gate_back: .usage = .{ .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, .{ .regs = 17, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_attn_q: 32 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_attn_q: 47 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_attn_q: 47 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_attn_q: 48 registers, 18560 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_attn_q: 48 registers, 18560 static shared bytes, bound 256 (record current) +RECORD 27B tf_attn_q: .usage = .{ .{ .regs = 32, .static = 18560, .bound = 256 }, .{ .regs = 47, .static = 18560, .bound = 256 }, .{ .regs = 47, .static = 18560, .bound = 256 }, .{ .regs = 48, .static = 18560, .bound = 256 }, .{ .regs = 48, .static = 18560, .bound = 256 }, } +PASS limits sm_80 27B tf_train_attn_kv: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_train_attn_kv: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_train_attn_kv: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_train_attn_kv: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_train_attn_kv: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_train_attn_kv: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_attn_prep_back: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_attn_prep_back: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_attn_prep_back: 31 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_attn_prep_back: 32 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_attn_prep_back: 32 registers, 1152 static shared bytes, bound 256 (record current) +RECORD 27B tf_attn_prep_back: .usage = .{ .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 31, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, .{ .regs = 32, .static = 1152, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_fit_dots: 31 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_fit_dots: 39 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_fit_dots: 39 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_fit_dots: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_fit_dots: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_fit_dots: .usage = .{ .{ .regs = 31, .static = 0, .bound = 256 }, .{ .regs = 39, .static = 0, .bound = 256 }, .{ .regs = 39, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 27B tf_q27_fit_dots_tree: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_q27_fit_dots_tree: 39 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_q27_fit_dots_tree: 39 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_q27_fit_dots_tree: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_q27_fit_dots_tree: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD 27B tf_q27_fit_dots_tree: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 39, .static = 0, .bound = 256 }, .{ .regs = 39, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, } +PASS limits sm_80 27B tf_gdn_front: 32 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_gdn_front: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_gdn_front: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_gdn_front: 40 registers, 1024 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_gdn_front: 40 registers, 1024 static shared bytes, bound 128 (record current) +RECORD 27B tf_gdn_front: .usage = .{ .{ .regs = 32, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, .{ .regs = 40, .static = 1024, .bound = 128 }, } +PASS limits sm_80 27B tf_gdn_scan_fwd: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_gdn_scan_fwd: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_gdn_scan_fwd: 48 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_gdn_scan_fwd: 60 registers, 1152 static shared bytes, bound 256 (record current) +PASS limits sm_121 27B tf_gdn_scan_fwd: 60 registers, 1152 static shared bytes, bound 256 (record current) +RECORD 27B tf_gdn_scan_fwd: .usage = .{ .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 48, .static = 1152, .bound = 256 }, .{ .regs = 60, .static = 1152, .bound = 256 }, .{ .regs = 60, .static = 1152, .bound = 256 }, } +PASS limits sm_80 27B tf_gdn_norm_back: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_gdn_norm_back: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_gdn_norm_back: 26 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_gdn_norm_back: 24 registers, 644 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_gdn_norm_back: 24 registers, 644 static shared bytes, bound 128 (record current) +RECORD 27B tf_gdn_norm_back: .usage = .{ .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 26, .static = 644, .bound = 128 }, .{ .regs = 24, .static = 644, .bound = 128 }, .{ .regs = 24, .static = 644, .bound = 128 }, } +PASS limits sm_80 27B tf_gdn_scan_back: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_86 27B tf_gdn_scan_back: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_89 27B tf_gdn_scan_back: 99 registers, 9536 static shared bytes, bound 256 (record current) +PASS limits sm_120 27B tf_gdn_scan_back: 102 registers, 9536 static shared bytes, bound 256 (record stale) +PASS limits sm_121 27B tf_gdn_scan_back: 102 registers, 9536 static shared bytes, bound 256 (record stale) +RECORD 27B tf_gdn_scan_back: .usage = .{ .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 99, .static = 9536, .bound = 256 }, .{ .regs = 102, .static = 9536, .bound = 256 }, .{ .regs = 102, .static = 9536, .bound = 256 }, } +PASS limits sm_80 27B tf_gdn_qk_back: 32 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_gdn_qk_back: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_gdn_qk_back: 39 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_gdn_qk_back: 40 registers, 128 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_gdn_qk_back: 40 registers, 128 static shared bytes, bound 128 (record current) +RECORD 27B tf_gdn_qk_back: .usage = .{ .{ .regs = 32, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 39, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, .{ .regs = 40, .static = 128, .bound = 128 }, } +PASS limits sm_80 27B tf_gdn_gates_back: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_gdn_gates_back: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_gdn_gates_back: 30 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_gdn_gates_back: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_gdn_gates_back: 32 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_gdn_gates_back: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 32, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_gdn_conv_back: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_gdn_conv_back: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_gdn_conv_back: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_gdn_conv_back: 35 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_gdn_conv_back: 35 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_gdn_conv_back: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, .{ .regs = 35, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_gdn_dequant: 18 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_gdn_dequant: .usage = .{ .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 27B tf_train_narrow: 10 registers, 0 static shared bytes, bound 0 (record current) +RECORD 27B tf_train_narrow: .usage = .{ .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, .{ .regs = 10, .static = 0, .bound = 0 }, } +PASS limits sm_80 27B tf_train_gemm: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_86 27B tf_train_gemm: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_89 27B tf_train_gemm: 55 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_120 27B tf_train_gemm: 64 registers, 10240 static shared bytes, bound 128 (record current) +PASS limits sm_121 27B tf_train_gemm: 64 registers, 10240 static shared bytes, bound 128 (record current) +RECORD 27B tf_train_gemm: .usage = .{ .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 55, .static = 10240, .bound = 128 }, .{ .regs = 64, .static = 10240, .bound = 128 }, .{ .regs = 64, .static = 10240, .bound = 128 }, } +PASS limits sm_80 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 paging tf_nemo_kv_write: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD paging tf_nemo_kv_write: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 paging tf_nemo_kv_write_segs: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 paging tf_nemo_kv_write_segs: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 paging tf_nemo_kv_write_segs: 26 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 paging tf_nemo_kv_write_segs: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 paging tf_nemo_kv_write_segs: 24 registers, 0 static shared bytes, bound 0 (record current) +RECORD paging tf_nemo_kv_write_segs: .usage = .{ .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, } +PASS limits sm_80 paging tf_nemo_attn_chunk: 76 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_86 paging tf_nemo_attn_chunk: 91 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_89 paging tf_nemo_attn_chunk: 91 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_120 paging tf_nemo_attn_chunk: 118 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_121 paging tf_nemo_attn_chunk: 118 registers, 45568 static shared bytes, bound 256 (record current) +RECORD paging tf_nemo_attn_chunk: .usage = .{ .{ .regs = 76, .static = 45568, .bound = 256 }, .{ .regs = 91, .static = 45568, .bound = 256 }, .{ .regs = 91, .static = 45568, .bound = 256 }, .{ .regs = 118, .static = 45568, .bound = 256 }, .{ .regs = 118, .static = 45568, .bound = 256 }, } +PASS limits sm_80 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_86 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_89 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_120 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_121 paging tf_nemo_attn_chunk_pf: 128 registers, 45568 static shared bytes, bound 256 (record current) +RECORD paging tf_nemo_attn_chunk_pf: .usage = .{ .{ .regs = 128, .static = 45568, .bound = 256 }, .{ .regs = 128, .static = 45568, .bound = 256 }, .{ .regs = 128, .static = 45568, .bound = 256 }, .{ .regs = 128, .static = 45568, .bound = 256 }, .{ .regs = 128, .static = 45568, .bound = 256 }, } +PASS limits sm_80 paging tf_nemo_attn_chunk_segs: 78 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_86 paging tf_nemo_attn_chunk_segs: 97 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_89 paging tf_nemo_attn_chunk_segs: 97 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_120 paging tf_nemo_attn_chunk_segs: 118 registers, 45568 static shared bytes, bound 256 (record current) +PASS limits sm_121 paging tf_nemo_attn_chunk_segs: 118 registers, 45568 static shared bytes, bound 256 (record current) +RECORD paging tf_nemo_attn_chunk_segs: .usage = .{ .{ .regs = 78, .static = 45568, .bound = 256 }, .{ .regs = 97, .static = 45568, .bound = 256 }, .{ .regs = 97, .static = 45568, .bound = 256 }, .{ .regs = 118, .static = 45568, .bound = 256 }, .{ .regs = 118, .static = 45568, .bound = 256 }, } +PASS limits sm_80 paging pattn128: 206 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 paging pattn128: 206 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 paging pattn128: 206 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 paging pattn128: 231 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 paging pattn128: 231 registers, 0 static shared bytes, bound 256 (record current) +RECORD paging pattn128: .usage = .{ .{ .regs = 206, .static = 0, .bound = 256 }, .{ .regs = 206, .static = 0, .bound = 256 }, .{ .regs = 206, .static = 0, .bound = 256 }, .{ .regs = 231, .static = 0, .bound = 256 }, .{ .regs = 231, .static = 0, .bound = 256 }, } +PASS limits sm_80 tf_moe_router: 32 registers, 16512 static shared bytes, bound 256 (record current) +PASS limits sm_86 tf_moe_router: 48 registers, 16512 static shared bytes, bound 256 (record current) +PASS limits sm_89 tf_moe_router: 48 registers, 16512 static shared bytes, bound 256 (record current) +PASS limits sm_120 tf_moe_router: 48 registers, 16512 static shared bytes, bound 256 (record current) +PASS limits sm_121 tf_moe_router: 48 registers, 16512 static shared bytes, bound 256 (record current) +RECORD tf_moe_router: .usage = .{ .{ .regs = 32, .static = 16512, .bound = 256 }, .{ .regs = 48, .static = 16512, .bound = 256 }, .{ .regs = 48, .static = 16512, .bound = 256 }, .{ .regs = 48, .static = 16512, .bound = 256 }, .{ .regs = 48, .static = 16512, .bound = 256 }, } +PASS limits sm_80 tf_moe_topk: 35 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tf_moe_topk: 35 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tf_moe_topk: 35 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tf_moe_topk: 34 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tf_moe_topk: 34 registers, 0 static shared bytes, bound 32 (record current) +RECORD tf_moe_topk: .usage = .{ .{ .regs = 35, .static = 0, .bound = 32 }, .{ .regs = 35, .static = 0, .bound = 32 }, .{ .regs = 35, .static = 0, .bound = 32 }, .{ .regs = 34, .static = 0, .bound = 32 }, .{ .regs = 34, .static = 0, .bound = 32 }, } +PASS limits sm_80 tf_moe_combine: 32 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 tf_moe_combine: 40 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 tf_moe_combine: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 tf_moe_combine: 38 registers, 0 static shared bytes, bound 256 (record current) +RECORD tf_moe_combine: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 40, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, } +PASS limits sm_80 experts plan_rows: 32 registers, 12672 static shared bytes, bound 1024 (record current) +PASS limits sm_86 experts plan_rows: 38 registers, 12672 static shared bytes, bound 1024 (record current) +PASS limits sm_89 experts plan_rows: 38 registers, 12672 static shared bytes, bound 1024 (record current) +PASS limits sm_120 experts plan_rows: 34 registers, 12672 static shared bytes, bound 1024 (record stale) +PASS limits sm_121 experts plan_rows: 34 registers, 12672 static shared bytes, bound 1024 (record stale) +RECORD experts plan_rows: .usage = .{ .{ .regs = 32, .static = 12672, .bound = 1024 }, .{ .regs = 38, .static = 12672, .bound = 1024 }, .{ .regs = 38, .static = 12672, .bound = 1024 }, .{ .regs = 34, .static = 12672, .bound = 1024 }, .{ .regs = 34, .static = 12672, .bound = 1024 }, } +PASS limits sm_80 experts plan_rank_rows: 32 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_86 experts plan_rank_rows: 40 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_89 experts plan_rank_rows: 40 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_120 experts plan_rank_rows: 42 registers, 4096 static shared bytes, bound 1024 (record current) +PASS limits sm_121 experts plan_rank_rows: 42 registers, 4096 static shared bytes, bound 1024 (record current) +RECORD experts plan_rank_rows: .usage = .{ .{ .regs = 32, .static = 4096, .bound = 1024 }, .{ .regs = 40, .static = 4096, .bound = 1024 }, .{ .regs = 40, .static = 4096, .bound = 1024 }, .{ .regs = 42, .static = 4096, .bound = 1024 }, .{ .regs = 42, .static = 4096, .bound = 1024 }, } +PASS limits sm_80 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 experts plan_scatter_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +RECORD experts plan_scatter_rows: .usage = .{ .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 14, .static = 0, .bound = 0 }, } +PASS limits sm_80 tf_q36m_norm_rope: 36 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tf_q36m_norm_rope: 37 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tf_q36m_norm_rope: 39 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tf_q36m_norm_rope: 42 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tf_q36m_norm_rope: 42 registers, 0 static shared bytes, bound 32 (record current) +RECORD tf_q36m_norm_rope: .usage = .{ .{ .regs = 36, .static = 0, .bound = 32 }, .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 39, .static = 0, .bound = 32 }, .{ .regs = 42, .static = 0, .bound = 32 }, .{ .regs = 42, .static = 0, .bound = 32 }, } +PASS limits sm_80 tf_q36m_attn: 50 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tf_q36m_attn: 52 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tf_q36m_attn: 51 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tf_q36m_attn: 52 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tf_q36m_attn: 52 registers, 0 static shared bytes, bound 32 (record current) +RECORD tf_q36m_attn: .usage = .{ .{ .regs = 50, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 51, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, } +PASS limits sm_80 tf_q36m_attn_tree: 53 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tf_q36m_attn_tree: 54 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tf_q36m_attn_tree: 52 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tf_q36m_attn_tree: 57 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tf_q36m_attn_tree: 57 registers, 0 static shared bytes, bound 32 (record current) +RECORD tf_q36m_attn_tree: .usage = .{ .{ .regs = 53, .static = 0, .bound = 32 }, .{ .regs = 54, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 57, .static = 0, .bound = 32 }, .{ .regs = 57, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_1: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_1: 55 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_1: 55 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_1: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_1: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD qwen35 tf_q35_gdn_slots_1: .usage = .{ .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 55, .static = 0, .bound = 128 }, .{ .regs = 55, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_2: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_2: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_2: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_2: 48 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_2: 48 registers, 0 static shared bytes, bound 128 (record current) +RECORD qwen35 tf_q35_gdn_slots_2: .usage = .{ .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, .{ .regs = 48, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_4: 72 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_4: 72 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_4: 72 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_4: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_4: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qwen35 tf_q35_gdn_slots_4: .usage = .{ .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_8: 96 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_8: 96 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_8: 96 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_8: 72 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_8: 72 registers, 0 static shared bytes, bound 128 (record current) +RECORD qwen35 tf_q35_gdn_slots_8: .usage = .{ .{ .regs = 96, .static = 0, .bound = 128 }, .{ .regs = 96, .static = 0, .bound = 128 }, .{ .regs = 96, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_slots_16: 150 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qwen35 tf_q35_gdn_slots_16: 149 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qwen35 tf_q35_gdn_slots_16: 149 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qwen35 tf_q35_gdn_slots_16: 112 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qwen35 tf_q35_gdn_slots_16: 112 registers, 0 static shared bytes, bound 128 (record current) +RECORD qwen35 tf_q35_gdn_slots_16: .usage = .{ .{ .regs = 150, .static = 0, .bound = 128 }, .{ .regs = 149, .static = 0, .bound = 128 }, .{ .regs = 149, .static = 0, .bound = 128 }, .{ .regs = 112, .static = 0, .bound = 128 }, .{ .regs = 112, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_chain: 72 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qwen35 tf_q35_gdn_chain: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qwen35 tf_q35_gdn_chain: 64 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qwen35 tf_q35_gdn_chain: 56 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qwen35 tf_q35_gdn_chain: 56 registers, 0 static shared bytes, bound 128 (record current) +RECORD qwen35 tf_q35_gdn_chain: .usage = .{ .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 64, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, .{ .regs = 56, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35 tf_q35_rope: 32 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen35 tf_q35_rope: 47 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen35 tf_q35_rope: 47 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen35 tf_q35_rope: 34 registers, 16384 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen35 tf_q35_rope: 34 registers, 16384 static shared bytes, bound 256 (record current) +RECORD qwen35 tf_q35_rope: .usage = .{ .{ .regs = 32, .static = 16384, .bound = 256 }, .{ .regs = 47, .static = 16384, .bound = 256 }, .{ .regs = 47, .static = 16384, .bound = 256 }, .{ .regs = 34, .static = 16384, .bound = 256 }, .{ .regs = 34, .static = 16384, .bound = 256 }, } +PASS limits sm_80 qwen35 tf_q35_attn_partial: 48 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qwen35 tf_q35_attn_partial: 48 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qwen35 tf_q35_attn_partial: 48 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qwen35 tf_q35_attn_partial: 39 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qwen35 tf_q35_attn_partial: 39 registers, 0 static shared bytes, bound 0 (record current) +RECORD qwen35 tf_q35_attn_partial: .usage = .{ .{ .regs = 48, .static = 0, .bound = 0 }, .{ .regs = 48, .static = 0, .bound = 0 }, .{ .regs = 48, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, .{ .regs = 39, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35 tf_q35_attn_merge: 40 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 qwen35 tf_q35_attn_merge: 38 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 qwen35 tf_q35_attn_merge: 37 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 qwen35 tf_q35_attn_merge: 42 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 qwen35 tf_q35_attn_merge: 42 registers, 0 static shared bytes, bound 32 (record current) +RECORD qwen35 tf_q35_attn_merge: .usage = .{ .{ .regs = 40, .static = 0, .bound = 32 }, .{ .regs = 38, .static = 0, .bound = 32 }, .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 42, .static = 0, .bound = 32 }, .{ .regs = 42, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35_kv8 tf_q35_rope8: 40 registers, 16416 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen35_kv8 tf_q35_rope8: 41 registers, 16416 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen35_kv8 tf_q35_rope8: 42 registers, 16416 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen35_kv8 tf_q35_rope8: 41 registers, 16416 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen35_kv8 tf_q35_rope8: 41 registers, 16416 static shared bytes, bound 256 (record current) +RECORD qwen35_kv8 tf_q35_rope8: .usage = .{ .{ .regs = 40, .static = 16416, .bound = 256 }, .{ .regs = 41, .static = 16416, .bound = 256 }, .{ .regs = 42, .static = 16416, .bound = 256 }, .{ .regs = 41, .static = 16416, .bound = 256 }, .{ .regs = 41, .static = 16416, .bound = 256 }, } +PASS limits sm_80 qwen35_kv8 tf_q35_attn_partial8: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qwen35_kv8 tf_q35_attn_partial8: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qwen35_kv8 tf_q35_attn_partial8: 56 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qwen35_kv8 tf_q35_attn_partial8: 37 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qwen35_kv8 tf_q35_attn_partial8: 37 registers, 0 static shared bytes, bound 0 (record current) +RECORD qwen35_kv8 tf_q35_attn_partial8: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 56, .static = 0, .bound = 0 }, .{ .regs = 37, .static = 0, .bound = 0 }, .{ .regs = 37, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35_keep tf_q35k_apply: 32 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 qwen35_keep tf_q35k_apply: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 qwen35_keep tf_q35k_apply: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 qwen35_keep tf_q35k_apply: 40 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_121 qwen35_keep tf_q35k_apply: 40 registers, 0 static shared bytes, bound 128 (record current) +RECORD qwen35_keep tf_q35k_apply: .usage = .{ .{ .regs = 32, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 qwen35_keep tf_q35k_compact: 18 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen35_keep tf_q35k_compact: 17 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen35_keep tf_q35k_compact: 17 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen35_keep tf_q35k_compact: 34 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen35_keep tf_q35k_compact: 34 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen35_keep tf_q35k_compact: .usage = .{ .{ .regs = 18, .static = 0, .bound = 256 }, .{ .regs = 17, .static = 0, .bound = 256 }, .{ .regs = 17, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen35_kv8 tf_q35k_compact8: 18 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen35_kv8 tf_q35k_compact8: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen35_kv8 tf_q35k_compact8: 38 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen35_kv8 tf_q35k_compact8: 22 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen35_kv8 tf_q35k_compact8: 22 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen35_kv8 tf_q35k_compact8: .usage = .{ .{ .regs = 18, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 38, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, .{ .regs = 22, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qwen35_draft tf_q35d_gather_rows: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD qwen35_draft tf_q35d_gather_rows: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35 tf_q35_gdn_pre: 50 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 qwen35 tf_q35_gdn_pre: 52 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 qwen35 tf_q35_gdn_pre: 52 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 qwen35 tf_q35_gdn_pre: 40 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 qwen35 tf_q35_gdn_pre: 40 registers, 0 static shared bytes, bound 32 (record current) +RECORD qwen35 tf_q35_gdn_pre: .usage = .{ .{ .regs = 50, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 40, .static = 0, .bound = 32 }, .{ .regs = 40, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35 tf_q35_conv_commit: 32 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qwen35 tf_q35_conv_commit: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qwen35 tf_q35_conv_commit: 40 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qwen35 tf_q35_conv_commit: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qwen35 tf_q35_conv_commit: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD qwen35 tf_q35_conv_commit: .usage = .{ .{ .regs = 32, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 40, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35 tf_q35_tap: 24 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 qwen35 tf_q35_tap: 24 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 qwen35 tf_q35_tap: 24 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 qwen35 tf_q35_tap: 26 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 qwen35 tf_q35_tap: 26 registers, 0 static shared bytes, bound 256 (record current) +RECORD qwen35 tf_q35_tap: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 24, .static = 0, .bound = 256 }, .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 26, .static = 0, .bound = 256 }, } +PASS limits sm_80 qwen35_keep tf_q35k_conv: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qwen35_keep tf_q35k_conv: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qwen35_keep tf_q35k_conv: 20 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qwen35_keep tf_q35k_conv: 24 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qwen35_keep tf_q35k_conv: 24 registers, 0 static shared bytes, bound 0 (record current) +RECORD qwen35_keep tf_q35k_conv: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35_draft tf_q35d_dyn_conv_batch: 29 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qwen35_draft tf_q35d_dyn_conv_batch: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qwen35_draft tf_q35d_dyn_conv_batch: 28 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qwen35_draft tf_q35d_dyn_conv_batch: 38 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qwen35_draft tf_q35d_dyn_conv_batch: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD qwen35_draft tf_q35d_dyn_conv_batch: .usage = .{ .{ .regs = 29, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 28, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35_draft tf_q35d_norm_rope_batch: 30 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 qwen35_draft tf_q35d_norm_rope_batch: 32 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 qwen35_draft tf_q35d_norm_rope_batch: 32 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 qwen35_draft tf_q35d_norm_rope_batch: 36 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 qwen35_draft tf_q35d_norm_rope_batch: 36 registers, 0 static shared bytes, bound 32 (record current) +RECORD qwen35_draft tf_q35d_norm_rope_batch: .usage = .{ .{ .regs = 30, .static = 0, .bound = 32 }, .{ .regs = 32, .static = 0, .bound = 32 }, .{ .regs = 32, .static = 0, .bound = 32 }, .{ .regs = 36, .static = 0, .bound = 32 }, .{ .regs = 36, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35_draft tf_q35d_mask_rows: 14 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_86 qwen35_draft tf_q35d_mask_rows: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_89 qwen35_draft tf_q35d_mask_rows: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_120 qwen35_draft tf_q35d_mask_rows: 16 registers, 0 static shared bytes, bound 0 (record current) +PASS limits sm_121 qwen35_draft tf_q35d_mask_rows: 16 registers, 0 static shared bytes, bound 0 (record current) +RECORD qwen35_draft tf_q35d_mask_rows: .usage = .{ .{ .regs = 14, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, } +PASS limits sm_80 qwen35_draft tf_q35d_attn_partial: 52 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 qwen35_draft tf_q35d_attn_partial: 52 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 qwen35_draft tf_q35d_attn_partial: 50 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 qwen35_draft tf_q35d_attn_partial: 50 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 qwen35_draft tf_q35d_attn_partial: 50 registers, 0 static shared bytes, bound 32 (record current) +RECORD qwen35_draft tf_q35d_attn_partial: .usage = .{ .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 52, .static = 0, .bound = 32 }, .{ .regs = 50, .static = 0, .bound = 32 }, .{ .regs = 50, .static = 0, .bound = 32 }, .{ .regs = 50, .static = 0, .bound = 32 }, } +PASS limits sm_80 qwen35_draft tf_q35d_attn_merge: 37 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 qwen35_draft tf_q35d_attn_merge: 37 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 qwen35_draft tf_q35d_attn_merge: 37 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 qwen35_draft tf_q35d_attn_merge: 34 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 qwen35_draft tf_q35d_attn_merge: 34 registers, 0 static shared bytes, bound 32 (record current) +RECORD qwen35_draft tf_q35d_attn_merge: .usage = .{ .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 37, .static = 0, .bound = 32 }, .{ .regs = 34, .static = 0, .bound = 32 }, .{ .regs = 34, .static = 0, .bound = 32 }, } +PASS limits sm_80 tf_topk_slices: 24 registers, 16448 static shared bytes, bound 256 (record current) +PASS limits sm_86 tf_topk_slices: 20 registers, 16448 static shared bytes, bound 256 (record current) +PASS limits sm_89 tf_topk_slices: 20 registers, 16448 static shared bytes, bound 256 (record current) +PASS limits sm_120 tf_topk_slices: 28 registers, 16448 static shared bytes, bound 256 (record current) +PASS limits sm_121 tf_topk_slices: 28 registers, 16448 static shared bytes, bound 256 (record current) +RECORD tf_topk_slices: .usage = .{ .{ .regs = 24, .static = 16448, .bound = 256 }, .{ .regs = 20, .static = 16448, .bound = 256 }, .{ .regs = 20, .static = 16448, .bound = 256 }, .{ .regs = 28, .static = 16448, .bound = 256 }, .{ .regs = 28, .static = 16448, .bound = 256 }, } +PASS limits sm_80 tf_topk_merge: 24 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_86 tf_topk_merge: 22 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_89 tf_topk_merge: 22 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_120 tf_topk_merge: 38 registers, 64 static shared bytes, bound 256 (record stale) +PASS limits sm_121 tf_topk_merge: 38 registers, 64 static shared bytes, bound 256 (record stale) +RECORD tf_topk_merge: .usage = .{ .{ .regs = 24, .static = 64, .bound = 256 }, .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 38, .static = 64, .bound = 256 }, .{ .regs = 38, .static = 64, .bound = 256 }, } +PASS limits sm_80 tf_q35_norm: 57 registers, 16416 static shared bytes, bound 256 (record current) +PASS limits sm_86 tf_q35_norm: 48 registers, 16416 static shared bytes, bound 256 (record current) +PASS limits sm_89 tf_q35_norm: 48 registers, 16416 static shared bytes, bound 256 (record current) +PASS limits sm_120 tf_q35_norm: 48 registers, 16416 static shared bytes, bound 256 (record current) +PASS limits sm_121 tf_q35_norm: 48 registers, 16416 static shared bytes, bound 256 (record current) +RECORD tf_q35_norm: .usage = .{ .{ .regs = 57, .static = 16416, .bound = 256 }, .{ .regs = 48, .static = 16416, .bound = 256 }, .{ .regs = 48, .static = 16416, .bound = 256 }, .{ .regs = 48, .static = 16416, .bound = 256 }, .{ .regs = 48, .static = 16416, .bound = 256 }, } +PASS limits sm_80 tf_q35_sums: 32 registers, 0 static shared bytes, bound 64 (record current) +PASS limits sm_86 tf_q35_sums: 42 registers, 0 static shared bytes, bound 64 (record current) +PASS limits sm_89 tf_q35_sums: 40 registers, 0 static shared bytes, bound 64 (record current) +PASS limits sm_120 tf_q35_sums: 40 registers, 0 static shared bytes, bound 64 (record current) +PASS limits sm_121 tf_q35_sums: 40 registers, 0 static shared bytes, bound 64 (record current) +RECORD tf_q35_sums: .usage = .{ .{ .regs = 32, .static = 0, .bound = 64 }, .{ .regs = 42, .static = 0, .bound = 64 }, .{ .regs = 40, .static = 0, .bound = 64 }, .{ .regs = 40, .static = 0, .bound = 64 }, .{ .regs = 40, .static = 0, .bound = 64 }, } +PASS limits sm_80 tf_q35_argmax_slices: 22 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_86 tf_q35_argmax_slices: 22 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_89 tf_q35_argmax_slices: 22 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_120 tf_q35_argmax_slices: 27 registers, 64 static shared bytes, bound 256 (record current) +PASS limits sm_121 tf_q35_argmax_slices: 27 registers, 64 static shared bytes, bound 256 (record current) +RECORD tf_q35_argmax_slices: .usage = .{ .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 22, .static = 64, .bound = 256 }, .{ .regs = 27, .static = 64, .bound = 256 }, .{ .regs = 27, .static = 64, .bound = 256 }, } +PASS limits sm_80 tf_q35_argmax_merge: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_86 tf_q35_argmax_merge: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_89 tf_q35_argmax_merge: 28 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_120 tf_q35_argmax_merge: 30 registers, 0 static shared bytes, bound 32 (record current) +PASS limits sm_121 tf_q35_argmax_merge: 30 registers, 0 static shared bytes, bound 32 (record current) +RECORD tf_q35_argmax_merge: .usage = .{ .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 28, .static = 0, .bound = 32 }, .{ .regs = 30, .static = 0, .bound = 32 }, .{ .regs = 30, .static = 0, .bound = 32 }, } +PASS limits sm_80 sample tf_draw: 52 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_86 sample tf_draw: 64 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_89 sample tf_draw: 64 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_120 sample tf_draw: 64 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_121 sample tf_draw: 64 registers, 9752 static shared bytes, bound 1024 (record current) +RECORD sample tf_draw: .usage = .{ .{ .regs = 52, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, } +PASS limits sm_80 sample tf_logprob_rows: 27 registers, 520 static shared bytes, bound 1024 (record current) +PASS limits sm_86 sample tf_logprob_rows: 28 registers, 520 static shared bytes, bound 1024 (record current) +PASS limits sm_89 sample tf_logprob_rows: 28 registers, 520 static shared bytes, bound 1024 (record current) +PASS limits sm_120 sample tf_logprob_rows: 40 registers, 520 static shared bytes, bound 1024 (record current) +PASS limits sm_121 sample tf_logprob_rows: 40 registers, 520 static shared bytes, bound 1024 (record current) +RECORD sample tf_logprob_rows: .usage = .{ .{ .regs = 27, .static = 520, .bound = 1024 }, .{ .regs = 28, .static = 520, .bound = 1024 }, .{ .regs = 28, .static = 520, .bound = 1024 }, .{ .regs = 40, .static = 520, .bound = 1024 }, .{ .regs = 40, .static = 520, .bound = 1024 }, } +PASS limits sm_80 sample tf_grammar_mask: 28 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_86 sample tf_grammar_mask: 26 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_89 sample tf_grammar_mask: 26 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_120 sample tf_grammar_mask: 17 registers, 0 static shared bytes, bound 256 (record current) +PASS limits sm_121 sample tf_grammar_mask: 17 registers, 0 static shared bytes, bound 256 (record current) +RECORD sample tf_grammar_mask: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 26, .static = 0, .bound = 256 }, .{ .regs = 17, .static = 0, .bound = 256 }, .{ .regs = 17, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_glue tf_glm_top1: 29 registers, 8192 static shared bytes, bound 1024 (record current) +PASS limits sm_86 glm_glue tf_glm_top1: 30 registers, 8192 static shared bytes, bound 1024 (record current) +PASS limits sm_89 glm_glue tf_glm_top1: 30 registers, 8192 static shared bytes, bound 1024 (record current) +PASS limits sm_120 glm_glue tf_glm_top1: 36 registers, 8192 static shared bytes, bound 1024 (record current) +PASS limits sm_121 glm_glue tf_glm_top1: 36 registers, 8192 static shared bytes, bound 1024 (record current) +RECORD glm_glue tf_glm_top1: .usage = .{ .{ .regs = 29, .static = 8192, .bound = 1024 }, .{ .regs = 30, .static = 8192, .bound = 1024 }, .{ .regs = 30, .static = 8192, .bound = 1024 }, .{ .regs = 36, .static = 8192, .bound = 1024 }, .{ .regs = 36, .static = 8192, .bound = 1024 }, } +PASS limits sm_80 glm_glue tf_glm_lat_write_fp8: 26 registers, 64 static shared bytes, bound 512 (record stale) +PASS limits sm_86 glm_glue tf_glm_lat_write_fp8: 28 registers, 64 static shared bytes, bound 512 (record stale) +PASS limits sm_89 glm_glue tf_glm_lat_write_fp8: 28 registers, 64 static shared bytes, bound 512 (record stale) +PASS limits sm_120 glm_glue tf_glm_lat_write_fp8: 25 registers, 64 static shared bytes, bound 512 (record stale) +PASS limits sm_121 glm_glue tf_glm_lat_write_fp8: 25 registers, 64 static shared bytes, bound 512 (record stale) +RECORD glm_glue tf_glm_lat_write_fp8: .usage = .{ .{ .regs = 26, .static = 64, .bound = 512 }, .{ .regs = 28, .static = 64, .bound = 512 }, .{ .regs = 28, .static = 64, .bound = 512 }, .{ .regs = 25, .static = 64, .bound = 512 }, .{ .regs = 25, .static = 64, .bound = 512 }, } +PASS limits sm_80 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 glm_glue tf_glm_chunks_16_fp8: 64 registers, 0 static shared bytes, bound 256 (record stale) +RECORD glm_glue tf_glm_chunks_16_fp8: .usage = .{ .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, } +PASS limits sm_80 glm_glue tf_glm_chunks_32_fp8: 128 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 glm_glue tf_glm_chunks_32_fp8: 128 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 glm_glue tf_glm_chunks_32_fp8: 127 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 glm_glue tf_glm_chunks_32_fp8: 130 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 glm_glue tf_glm_chunks_32_fp8: 130 registers, 0 static shared bytes, bound 256 (record stale) +RECORD glm_glue tf_glm_chunks_32_fp8: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 127, .static = 0, .bound = 256 }, .{ .regs = 130, .static = 0, .bound = 256 }, .{ .regs = 130, .static = 0, .bound = 256 }, } +PASS limits sm_80 sample tf_draw_pitched: 57 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_86 sample tf_draw_pitched: 64 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_89 sample tf_draw_pitched: 64 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_120 sample tf_draw_pitched: 64 registers, 9752 static shared bytes, bound 1024 (record current) +PASS limits sm_121 sample tf_draw_pitched: 64 registers, 9752 static shared bytes, bound 1024 (record current) +RECORD sample tf_draw_pitched: .usage = .{ .{ .regs = 57, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, .{ .regs = 64, .static = 9752, .bound = 1024 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 121 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 121, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 72 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 72, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 167 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 167, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 72 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 72, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 168 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 168, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 72 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 72, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 142 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 142, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 80 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 80, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 164 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 164, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 99 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 99, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 166 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 166, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 99 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 99, .static = 32832, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 96 registers, 16448 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 96, .static = 16448, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 124 registers, 16448 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 124, .static = 16448, .bound = 128 }, } +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 126 registers, 16448 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 126, .static = 16448, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 186 registers, 16640 static shared bytes, bound 128 (record current) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 186, .static = 16640, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 238 registers, 16640 static shared bytes, bound 128 (record current) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 238, .static = 16640, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 240 registers, 16640 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 240, .static = 16640, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 185 registers, 32896 static shared bytes, bound 128 (record current) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 185, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 250 registers, 32896 static shared bytes, bound 128 (record current) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 250, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 250 registers, 32896 static shared bytes, bound 128 (record current) +RECORD dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 250, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8: 125 registers, 32848 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8: .usage = .{ .{ .regs = 125, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10: 167 registers, 32848 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10: .usage = .{ .{ .regs = 167, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12: 167 registers, 32848 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12: .usage = .{ .{ .regs = 167, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8: 146 registers, 32848 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8: .usage = .{ .{ .regs = 146, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10: 163 registers, 32848 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10: .usage = .{ .{ .regs = 163, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12: 168 registers, 32848 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12: .usage = .{ .{ .regs = 168, .static = 32848, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8: 121 registers, 32896 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8: .usage = .{ .{ .regs = 121, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8: 140 registers, 32896 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8: .usage = .{ .{ .regs = 140, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8: 121 registers, 32896 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8: .usage = .{ .{ .regs = 121, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8: 140 registers, 32896 static shared bytes, bound 128 (record current) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8: .usage = .{ .{ .regs = 140, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 sm_80: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 sm_86: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 sm_89: below_minimum +N/A limits dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 sm_120: below_minimum +PASS limits sm_121 dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12: 168 registers, 32896 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12: .usage = .{ .{ .regs = 168, .static = 32896, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 118 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 118, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 167 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 167, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 168 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 168, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 134 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 134, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 168 registers, 32832 static shared bytes, bound 128 (record stale) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 168, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 168 registers, 32832 static shared bytes, bound 128 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 168, .static = 32832, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 89 registers, 16448 static shared bytes, bound 128 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 89, .static = 16448, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 96 registers, 16448 static shared bytes, bound 128 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 96, .static = 16448, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 registers, 16448 static shared bytes, bound 128 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: .usage = .{ .{ .regs = 128, .static = 16448, .bound = 128 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii: 23 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii: .usage = .{ .{ .regs = 23, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii: 23 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii: .usage = .{ .{ .regs = 23, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii: 30 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii: .usage = .{ .{ .regs = 30, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii: 30 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii: .usage = .{ .{ .regs = 30, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii: 28 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii: .usage = .{ .{ .regs = 28, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii: 28 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii: .usage = .{ .{ .regs = 28, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii: 42 registers, 128 static shared bytes, bound 1024 (record current) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii: .usage = .{ .{ .regs = 42, .static = 128, .bound = 1024 }, } +N/A limits dsv41_router_glue router_group_rot sm_80: below_minimum +N/A limits dsv41_router_glue router_group_rot sm_86: below_minimum +N/A limits dsv41_router_glue router_group_rot sm_89: below_minimum +N/A limits dsv41_router_glue router_group_rot sm_120: below_minimum +PASS limits sm_121 dsv41_router_glue router_group_rot: 25 registers, 6528 static shared bytes, bound 1024 (record current) +RECORD dsv41_router_glue router_group_rot: .usage = .{ .{ .regs = 25, .static = 6528, .bound = 1024 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 20 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii: 40 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii: 38 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii: .usage = .{ .{ .regs = 38, .static = 0, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii: 40 registers, 16384 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii: .usage = .{ .{ .regs = 40, .static = 16384, .bound = 0 }, } +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii sm_80: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii sm_86: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii sm_89: below_minimum +N/A limits dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii sm_120: below_minimum +PASS limits sm_121 dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii: 40 registers, 16384 static shared bytes, bound 0 (record current) +RECORD dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii: .usage = .{ .{ .regs = 40, .static = 16384, .bound = 0 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 126 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 162 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 162, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 238 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 238, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 168 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 168, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 161 registers, 272 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 161, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 124 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 124, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 166 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 166, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 166 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 166, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 243 registers, 528 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 243, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi: 26 registers, 48 static shared bytes, bound 32 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi: .usage = .{ .{ .regs = 26, .static = 48, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 162 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 162, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 168 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 168, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 159 registers, 272 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 159, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 124 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 124, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 164 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 164, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 164 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 164, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi: 20 registers, 64 static shared bytes, bound 32 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi: .usage = .{ .{ .regs = 20, .static = 64, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 126 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 151 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 151, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 174 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 174, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 143 registers, 272 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 143, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 168 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 168, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 167 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 167, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi: 26 registers, 80 static shared bytes, bound 32 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi: .usage = .{ .{ .regs = 26, .static = 80, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 126 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 151 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 151, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 174 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 174, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 143 registers, 272 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 143, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 168 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 168, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 167 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 167, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi: 26 registers, 96 static shared bytes, bound 32 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi: .usage = .{ .{ .regs = 26, .static = 96, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 126 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 155 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 155, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 125 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 182 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 182, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 149 registers, 272 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 149, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 123 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 123, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 168 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 168, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 160 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 160, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi: 22 registers, 112 static shared bytes, bound 32 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi: .usage = .{ .{ .regs = 22, .static = 112, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 162 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 162, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 126 registers, 528 static shared bytes, bound 512 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 238 registers, 528 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 238, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 168 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 168, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 149 registers, 272 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 149, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 125 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 164 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 164, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 167 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 167, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 242 registers, 528 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi: 20 registers, 128 static shared bytes, bound 32 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi: .usage = .{ .{ .regs = 20, .static = 128, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 157 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 157, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 126 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 155 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 155, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 155 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 155, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 158 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 158, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 127 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 127, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: no applicable target +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 238 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 238, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 158 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 158, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 149 registers, 272 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 149, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 119 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 119, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 152 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 152, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 160 registers, 528 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 160, .static = 528, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 240 registers, 528 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 528, .bound = 128 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi: 21 registers, 160 static shared bytes, bound 32 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi: .usage = .{ .{ .regs = 21, .static = 160, .bound = 32 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 252 registers, 272 static shared bytes, bound 256 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 252, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 272 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 272, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 253 registers, 272 static shared bytes, bound 128 (record stale) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 253, .static = 272, .bound = 128 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 128 registers, 528 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 528, .bound = 512 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 512 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 512 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 512 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: 250 registers, 544 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 250, .static = 544, .bound = 128 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: 255 registers, 544 static shared bytes, bound 128 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 255, .static = 544, .bound = 128 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE: 128 registers, 544 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 544, .bound = 256 }, } +N/A limits dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ sm_80: below_minimum +N/A limits dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ sm_86: below_minimum +N/A limits dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ sm_89: below_minimum +N/A limits dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_: 60 registers, 38980 static shared bytes, bound 512 (record current) +RECORD dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_: .usage = .{ .{ .regs = 60, .static = 38980, .bound = 512 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 28 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_80: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_86: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_89: below_minimum +N/A limits dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii sm_120: below_minimum +PASS limits sm_121 dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 28 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i: 22 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i: .usage = .{ .{ .regs = 22, .static = 0, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 127 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 127, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 115 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 115, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 122 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 122, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 161 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 161, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 161 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 161, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 153 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 153, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 126 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 126, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 117 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 117, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 126 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 126, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 162 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 162, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 162 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 162, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 154 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 154, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 165 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 165, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 165 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 165, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 154 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 154, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 228 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 228, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 156 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 156, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 154 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 154, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 220 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 220, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill: 24 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 24, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 218 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 218, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill: 23 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 23, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 127 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 127, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 127 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 127, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill: 21 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 21, .static = 0, .bound = 32 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 126 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 126, .static = 16, .bound = 64 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 122 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 122, .static = 16, .bound = 128 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 127 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 127, .static = 16, .bound = 256 }, } +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill: 22 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 22, .static = 0, .bound = 32 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE: 38 registers, 32 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE: .usage = .{ .{ .regs = 38, .static = 32, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii: 183 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 183, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii: 178 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 178, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii: 238 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 238, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii: 233 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 233, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii: 205 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 205, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii: 201 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 201, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii: 255 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 255, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii: 255 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 255, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii: 255 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 255, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii: 255 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 255, .static = 16, .bound = 256 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii: 172 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 172, .static = 16, .bound = 128 }, } +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii sm_80: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii sm_86: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii sm_89: below_minimum +N/A limits dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii: 168 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 192 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 192, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill: 32 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 32, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 196 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 196, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill: 31 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 31, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 166 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 166, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 206 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 206, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill: 29 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 29, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 212 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 212, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill: 37 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 37, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 128 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 222 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 222, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 128, .static = 16, .bound = 256 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill: 40 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill: .usage = .{ .{ .regs = 40, .static = 0, .bound = 32 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii: 36 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii: .usage = .{ .{ .regs = 36, .static = 0, .bound = 0 }, } +N/A limits dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii sm_80: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii sm_86: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii sm_89: below_minimum +N/A limits dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii sm_120: below_minimum +PASS limits sm_121 dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii: 30 registers, 0 static shared bytes, bound 0 (record current) +RECORD dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii: .usage = .{ .{ .regs = 30, .static = 0, .bound = 0 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 246 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 246, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 239 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 239, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 244 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 244, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 245 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 245, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 240 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 240, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 242 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 242, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 248 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 248, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 248 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 248, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 243 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 243, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 242 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 242, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 192 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 192, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 226 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 226, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 226 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 226, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 226 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 226, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 194 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 194, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 232 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 232, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 232 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 232, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 232 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 232, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 230 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 230, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 230 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 230, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 230 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 230, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 243 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 243, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 243 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 243, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 243 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 243, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 212 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 212, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 212 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 212, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 212 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 212, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 243 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 243, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 243 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 243, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 243 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 243, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 243 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 243, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 243 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 243, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 243 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 243, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 173 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 173, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 136 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 136, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 136 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 136, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 241 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 241, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 241 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 241, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 147 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 147, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 147 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 147, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 241 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 241, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 241 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 241, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 145 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 145, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 145 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 145, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 239 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 239, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 239 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 239, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 159 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 159, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 164 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 164, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 248 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 248, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 248 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 248, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 156 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 156, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 160 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 160, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 245 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 245, .static = 0, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 245 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: .usage = .{ .{ .regs = 245, .static = 0, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 173 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 173, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 171 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 171, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 173 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 173, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 167 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 167, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 174 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 174, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 166 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 166, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 174 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 174, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 174 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 174, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 173 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 173, .static = 16, .bound = 256 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 64 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 64 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 168 registers, 16 static shared bytes, bound 128 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 168, .static = 16, .bound = 128 }, } +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_80: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_86: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_89: below_minimum +N/A limits dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi sm_120: below_minimum +PASS limits sm_121 dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 170 registers, 16 static shared bytes, bound 256 (record current) +RECORD dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: .usage = .{ .{ .regs = 170, .static = 16, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE: 114 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 114, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE: 114 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 114, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE: 114 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 114, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE: 114 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 114, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE: 111 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 111, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE: 111 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 111, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE: 111 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 111, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE: 111 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 111, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE: 224 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 224, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE: 224 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 224, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE: 224 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 224, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE: 224 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 224, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE: 224 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 224, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE: 224 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 224, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE: 224 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 224, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE: 224 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 224, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE: 86 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE: 86 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE: 86 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE: 86 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE: 86 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE: 86 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE: 86 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 0, .bound = 256 }, } +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE: 86 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 0, .bound = 256 }, } +N/A limits dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE sm_80: below_minimum +N/A limits dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE sm_86: below_minimum +N/A limits dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE sm_89: below_minimum +N/A limits dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE: 55 registers, 3232 static shared bytes, bound 512 (record stale) +RECORD dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE: .usage = .{ .{ .regs = 55, .static = 3232, .bound = 512 }, } +N/A limits dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE sm_80: below_minimum +N/A limits dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE sm_86: below_minimum +N/A limits dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE sm_89: below_minimum +N/A limits dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE: 55 registers, 3232 static shared bytes, bound 512 (record current) +RECORD dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE: .usage = .{ .{ .regs = 55, .static = 3232, .bound = 512 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 113 registers, 20496 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 113, .static = 20496, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 125 registers, 21392 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 125, .static = 21392, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 80 registers, 20496 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 20496, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 62 registers, 21392 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 62, .static = 21392, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 80 registers, 20496 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 20496, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 89 registers, 21392 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 89, .static = 21392, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE: 96 registers, 20496 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 96, .static = 20496, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 113 registers, 20656 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 113, .static = 20656, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 128 registers, 21552 static shared bytes, bound 128 (record stale) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 21552, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 80 registers, 20656 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 20656, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 64 registers, 21552 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 64, .static = 21552, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 80 registers, 20656 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 20656, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 91 registers, 21552 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 91, .static = 21552, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE: 94 registers, 20656 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 94, .static = 20656, .bound = 128 }, } +N/A limits dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE: 22 registers, 18048 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE: .usage = .{ .{ .regs = 22, .static = 18048, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE: 141 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 141, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE: 143 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 143, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE: 141 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 141, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE: 143 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 143, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE: 150 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 150, .static = 0, .bound = 128 }, } +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE: 148 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE: .usage = .{ .{ .regs = 148, .static = 0, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE: 84 registers, 2052 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 84, .static = 2052, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE: 84 registers, 2052 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 84, .static = 2052, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE: 79 registers, 2052 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 79, .static = 2052, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE: 79 registers, 2052 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 79, .static = 2052, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE: 89 registers, 10244 static shared bytes, bound 32 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 89, .static = 10244, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE: 100 registers, 20484 static shared bytes, bound 64 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 100, .static = 20484, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE: 100 registers, 40964 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 100, .static = 40964, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE: 101 registers, 40964 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 101, .static = 40964, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE: 80 registers, 4100 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 4100, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE: 80 registers, 4100 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 4100, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE: 86 registers, 4100 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 4100, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE: 86 registers, 4100 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 86, .static = 4100, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE: 110 registers, 10244 static shared bytes, bound 32 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 110, .static = 10244, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE: 111 registers, 20484 static shared bytes, bound 64 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 111, .static = 20484, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE: 111 registers, 40964 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 111, .static = 40964, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE: 118 registers, 40964 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 118, .static = 40964, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE: 84 registers, 8196 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 84, .static = 8196, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE: 88 registers, 8196 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 8196, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE: 80 registers, 8196 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 80, .static = 8196, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE: 90 registers, 8196 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 90, .static = 8196, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE: 191 registers, 10244 static shared bytes, bound 32 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 191, .static = 10244, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE: 192 registers, 20484 static shared bytes, bound 64 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 192, .static = 20484, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE: 205 registers, 40964 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 205, .static = 40964, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE: 128 registers, 40964 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 40964, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE: 104 registers, 16388 static shared bytes, bound 256 (record stale) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 104, .static = 16388, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE: 109 registers, 16388 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 109, .static = 16388, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE: 105 registers, 16388 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 105, .static = 16388, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE: 116 registers, 16388 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 116, .static = 16388, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE: 142 registers, 5636 static shared bytes, bound 32 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 142, .static = 5636, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE: 140 registers, 10244 static shared bytes, bound 64 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 140, .static = 10244, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE: 140 registers, 20484 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 140, .static = 20484, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE: 142 registers, 40964 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 142, .static = 40964, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE: 122 registers, 32772 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 122, .static = 32772, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE: 199 registers, 32772 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 199, .static = 32772, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE: 177 registers, 32772 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 177, .static = 32772, .bound = 256 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE: 216 registers, 32772 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 216, .static = 32772, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE: 210 registers, 5636 static shared bytes, bound 32 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE: .usage = .{ .{ .regs = 210, .static = 5636, .bound = 32 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE: 158 registers, 10244 static shared bytes, bound 64 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE: .usage = .{ .{ .regs = 158, .static = 10244, .bound = 64 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE: 158 registers, 20484 static shared bytes, bound 128 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE: .usage = .{ .{ .regs = 158, .static = 20484, .bound = 128 }, } +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE: 170 registers, 40964 static shared bytes, bound 256 (record current) +RECORD dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE: .usage = .{ .{ .regs = 170, .static = 40964, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 242 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi: 26 registers, 10240 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 26, .static = 10240, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 134 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 134, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 240 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi: 30 registers, 10240 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 30, .static = 10240, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 133 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 133, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 87 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 87, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 244 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 244, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi: 31 registers, 10752 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 31, .static = 10752, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 136 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 136, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 133 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 133, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 91 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 91, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 242 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi: 33 registers, 10752 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 33, .static = 10752, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 133 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 133, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 87 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 87, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 244 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 244, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi: 31 registers, 11264 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 31, .static = 11264, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 134 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 134, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 133 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 133, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 91 registers, 0 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 91, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 242 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 242, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi: 31 registers, 11264 static shared bytes, bound 128 (record stale) +RECORD dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 31, .static = 11264, .bound = 128 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 132 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 132, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 128, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 126 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 126, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 88 registers, 0 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 88, .static = 0, .bound = 128 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 240 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: .usage = .{ .{ .regs = 240, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi sm_80: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi sm_86: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi sm_89: below_minimum +N/A limits dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi sm_120: below_minimum +PASS limits sm_121 dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi: 27 registers, 12288 static shared bytes, bound 128 (record current) +RECORD dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi: .usage = .{ .{ .regs = 27, .static = 12288, .bound = 128 }, } +N/A limits dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi sm_80: below_minimum +N/A limits dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi sm_86: below_minimum +N/A limits dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi sm_89: below_minimum +N/A limits dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi sm_120: below_minimum +PASS limits sm_121 dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi: 48 registers, 8 static shared bytes, bound 0 (record current) +RECORD dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi: .usage = .{ .{ .regs = 48, .static = 8, .bound = 0 }, } +N/A limits dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm sm_80: below_minimum +N/A limits dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm sm_86: below_minimum +N/A limits dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm sm_89: below_minimum +N/A limits dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm sm_120: below_minimum +PASS limits sm_121 dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm: 20 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm: .usage = .{ .{ .regs = 20, .static = 0, .bound = 32 }, } +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij sm_80: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij sm_86: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij sm_89: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij sm_120: below_minimum +PASS limits sm_121 dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij: 26 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij: .usage = .{ .{ .regs = 26, .static = 0, .bound = 256 }, } +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij sm_80: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij sm_86: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij sm_89: below_minimum +N/A limits dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij sm_120: below_minimum +PASS limits sm_121 dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij: 28 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx sm_80: below_minimum +N/A limits dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx sm_86: below_minimum +N/A limits dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx sm_89: below_minimum +N/A limits dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx sm_120: below_minimum +PASS limits sm_121 dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx: 64 registers, 9232 static shared bytes, bound 1024 (record current) +RECORD dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx: .usage = .{ .{ .regs = 64, .static = 9232, .bound = 1024 }, } +N/A limits dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif sm_80: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif sm_86: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif sm_89: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif sm_120: below_minimum +PASS limits sm_121 dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif: 28 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx sm_80: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx sm_86: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx sm_89: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx sm_120: below_minimum +PASS limits sm_121 dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx: 30 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, } +N/A limits dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x sm_80: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x sm_86: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x sm_89: below_minimum +N/A limits dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x sm_120: below_minimum +PASS limits sm_121 dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x: 30 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE: 34 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE: .usage = .{ .{ .regs = 34, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh: 30 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh: .usage = .{ .{ .regs = 30, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE: 36 registers, 2336 static shared bytes, bound 256 (record current) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE: .usage = .{ .{ .regs = 36, .static = 2336, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE: 36 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE: .usage = .{ .{ .regs = 36, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE: 24 registers, 32 static shared bytes, bound 256 (record current) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE: .usage = .{ .{ .regs = 24, .static = 32, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE: 37 registers, 2336 static shared bytes, bound 256 (record current) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE: .usage = .{ .{ .regs = 37, .static = 2336, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE: 35 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE: .usage = .{ .{ .regs = 35, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE: 38 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE: .usage = .{ .{ .regs = 38, .static = 0, .bound = 256 }, } +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE sm_80: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE sm_86: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE sm_89: below_minimum +N/A limits dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE sm_120: below_minimum +PASS limits sm_121 dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt: 32 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt: 64 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt: .usage = .{ .{ .regs = 64, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx: 23 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx: .usage = .{ .{ .regs = 23, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx: 26 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx: .usage = .{ .{ .regs = 26, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx: 24 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx: .usage = .{ .{ .regs = 24, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_: 12 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf: 16 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf: .usage = .{ .{ .regs = 16, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_: 12 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx: 28 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf: 42 registers, 2080 static shared bytes, bound 1024 (record current) +RECORD dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf: .usage = .{ .{ .regs = 42, .static = 2080, .bound = 1024 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd: 14 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd: .usage = .{ .{ .regs = 14, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx: 26 registers, 0 static shared bytes, bound 1024 (record current) +RECORD dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx: .usage = .{ .{ .regs = 26, .static = 0, .bound = 1024 }, } +N/A limits dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx: 20 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx: .usage = .{ .{ .regs = 20, .static = 0, .bound = 32 }, } +N/A limits dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx sm_80: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx sm_86: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx sm_89: below_minimum +N/A limits dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx sm_120: below_minimum +PASS limits sm_121 dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx: 60 registers, 24576 static shared bytes, bound 256 (record current) +RECORD dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx: .usage = .{ .{ .regs = 60, .static = 24576, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi: 48 registers, 9244 static shared bytes, bound 1024 (record current) +RECORD dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi: .usage = .{ .{ .regs = 48, .static = 9244, .bound = 1024 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf: 32 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi: 27 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi: .usage = .{ .{ .regs = 27, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx: 28 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx: .usage = .{ .{ .regs = 28, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf: 40 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf: .usage = .{ .{ .regs = 40, .static = 0, .bound = 256 }, } +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx sm_80: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx sm_86: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx sm_89: below_minimum +N/A limits dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx sm_120: below_minimum +PASS limits sm_121 dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx: 36 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx: .usage = .{ .{ .regs = 36, .static = 0, .bound = 256 }, } +N/A limits dsv41_kv_glue kv_norm_store sm_80: below_minimum +N/A limits dsv41_kv_glue kv_norm_store sm_86: below_minimum +N/A limits dsv41_kv_glue kv_norm_store sm_89: below_minimum +N/A limits dsv41_kv_glue kv_norm_store sm_120: below_minimum +PASS limits sm_121 dsv41_kv_glue kv_norm_store: 26 registers, 1056 static shared bytes, bound 128 (record current) +RECORD dsv41_kv_glue kv_norm_store: .usage = .{ .{ .regs = 26, .static = 1056, .bound = 128 }, } +N/A limits dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd sm_80: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd sm_86: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd sm_89: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd sm_120: below_minimum +PASS limits sm_121 dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd: 43 registers, 264 static shared bytes, bound 1024 (record current) +RECORD dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd: .usage = .{ .{ .regs = 43, .static = 264, .bound = 1024 }, } +N/A limits dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf sm_80: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf sm_86: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf sm_89: below_minimum +N/A limits dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf sm_120: below_minimum +PASS limits sm_121 dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf: 18 registers, 0 static shared bytes, bound 256 (record current) +RECORD dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf: .usage = .{ .{ .regs = 18, .static = 0, .bound = 256 }, } +N/A limits dsv41_sampling dsv41_sampling_stats_max sm_80: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_max sm_86: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_max sm_89: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_max sm_120: below_minimum +PASS limits sm_121 dsv41_sampling dsv41_sampling_stats_max: 39 registers, 32 static shared bytes, bound 256 (record current) +RECORD dsv41_sampling dsv41_sampling_stats_max: .usage = .{ .{ .regs = 39, .static = 32, .bound = 256 }, } +N/A limits dsv41_sampling dsv41_sampling_stats_sum sm_80: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_sum sm_86: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_sum sm_89: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_sum sm_120: below_minimum +PASS limits sm_121 dsv41_sampling dsv41_sampling_stats_sum: 38 registers, 64 static shared bytes, bound 256 (record current) +RECORD dsv41_sampling dsv41_sampling_stats_sum: .usage = .{ .{ .regs = 38, .static = 64, .bound = 256 }, } +N/A limits dsv41_sampling dsv41_sampling_stats_fin sm_80: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_fin sm_86: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_fin sm_89: below_minimum +N/A limits dsv41_sampling dsv41_sampling_stats_fin sm_120: below_minimum +PASS limits sm_121 dsv41_sampling dsv41_sampling_stats_fin: 40 registers, 0 static shared bytes, bound 32 (record current) +RECORD dsv41_sampling dsv41_sampling_stats_fin: .usage = .{ .{ .regs = 40, .static = 0, .bound = 32 }, } +PASS limits sm_80 tree_round tf_round_bf16_argmax: 25 registers, 3072 static shared bytes, bound 256 (record stale) +PASS limits sm_86 tree_round tf_round_bf16_argmax: 25 registers, 3072 static shared bytes, bound 256 (record stale) +PASS limits sm_89 tree_round tf_round_bf16_argmax: 25 registers, 3072 static shared bytes, bound 256 (record stale) +PASS limits sm_120 tree_round tf_round_bf16_argmax: 23 registers, 3072 static shared bytes, bound 256 (record stale) +PASS limits sm_121 tree_round tf_round_bf16_argmax: 23 registers, 3072 static shared bytes, bound 256 (record stale) +RECORD tree_round tf_round_bf16_argmax: .usage = .{ .{ .regs = 25, .static = 3072, .bound = 256 }, .{ .regs = 25, .static = 3072, .bound = 256 }, .{ .regs = 25, .static = 3072, .bound = 256 }, .{ .regs = 23, .static = 3072, .bound = 256 }, .{ .regs = 23, .static = 3072, .bound = 256 }, } +PASS limits sm_80 tree_round tf_tree_round_match: 25 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 tree_round tf_tree_round_match: 26 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 tree_round tf_tree_round_match: 26 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 tree_round tf_tree_round_match: 34 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 tree_round tf_tree_round_match: 34 registers, 0 static shared bytes, bound 0 (record stale) +RECORD tree_round tf_tree_round_match: .usage = .{ .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 26, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree_round tf_round_keep_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 tree_round tf_round_keep_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 tree_round tf_round_keep_rows: 16 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 tree_round tf_round_keep_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 tree_round tf_round_keep_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +RECORD tree_round tf_round_keep_rows: .usage = .{ .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 16, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 tree_round tf_round_gather_rows: 18 registers, 0 static shared bytes, bound 0 (record stale) +RECORD tree_round tf_round_gather_rows: .usage = .{ .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, .{ .regs = 18, .static = 0, .bound = 0 }, } +PASS limits sm_80 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_86 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_89 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_120 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +PASS limits sm_121 tree_round tf_round_keep_head: 20 registers, 0 static shared bytes, bound 0 (record stale) +RECORD tree_round tf_round_keep_head: .usage = .{ .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, .{ .regs = 20, .static = 0, .bound = 0 }, } +PASS limits sm_80 act_rows tf_swiglu_rows: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_swiglu_rows: 35 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_swiglu_rows: 35 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_swiglu_rows: 34 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_swiglu_rows: 34 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_swiglu_rows: .usage = .{ .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 35, .static = 0, .bound = 256 }, .{ .regs = 35, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_swiglu_rows, 16 rows or fewer: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_swiglu_rows, 16 rows or fewer: 35 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_swiglu_rows, 16 rows or fewer: 35 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_swiglu_rows, 16 rows or fewer: 34 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_swiglu_rows, 16 rows or fewer: 34 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_swiglu_rows, 16 rows or fewer: .usage = .{ .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 35, .static = 0, .bound = 256 }, .{ .regs = 35, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_gate_rows: 32 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_gate_rows: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_gate_rows: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_gate_rows: 34 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_gate_rows: 34 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_gate_rows: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_gate_rows, 16 rows or fewer: 32 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_gate_rows, 16 rows or fewer: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_gate_rows, 16 rows or fewer: 33 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_gate_rows, 16 rows or fewer: 34 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_gate_rows, 16 rows or fewer: 34 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_gate_rows, 16 rows or fewer: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 33, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, .{ .regs = 34, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_group_sums_rows: 12 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_group_sums_rows: 12 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_group_sums_rows: 12 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_group_sums_rows: 16 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_group_sums_rows: 16 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_group_sums_rows: .usage = .{ .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 12, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, .{ .regs = 16, .static = 0, .bound = 256 }, } +PASS limits sm_80 act_rows tf_dconv_rows: 32 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_86 act_rows tf_dconv_rows: 37 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_89 act_rows tf_dconv_rows: 37 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_120 act_rows tf_dconv_rows: 37 registers, 0 static shared bytes, bound 256 (record stale) +PASS limits sm_121 act_rows tf_dconv_rows: 37 registers, 0 static shared bytes, bound 256 (record stale) +RECORD act_rows tf_dconv_rows: .usage = .{ .{ .regs = 32, .static = 0, .bound = 256 }, .{ .regs = 37, .static = 0, .bound = 256 }, .{ .regs = 37, .static = 0, .bound = 256 }, .{ .regs = 37, .static = 0, .bound = 256 }, .{ .regs = 37, .static = 0, .bound = 256 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_64: 44 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_64: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_64: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_64: 46 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_64: 46 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_64: .usage = .{ .{ .regs = 44, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 46, .static = 0, .bound = 128 }, .{ .regs = 46, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 44 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 46 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_64, 16 rows or fewer: 46 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_64, 16 rows or fewer: .usage = .{ .{ .regs = 44, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 46, .static = 0, .bound = 128 }, .{ .regs = 46, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_128: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_128: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_128: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_128: 77 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_128: 77 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_128: .usage = .{ .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 77, .static = 0, .bound = 128 }, .{ .regs = 77, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 77 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_128, 16 rows or fewer: 77 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_128, 16 rows or fewer: .usage = .{ .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 77, .static = 0, .bound = 128 }, .{ .regs = 77, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_256: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_256: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_256: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_256: 145 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_256: 145 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_256: .usage = .{ .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 145, .static = 0, .bound = 128 }, .{ .regs = 145, .static = 0, .bound = 128 }, } +PASS limits sm_80 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 146 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 145 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 gdn_prep tf_gdn_prep_256, 16 rows or fewer: 145 registers, 0 static shared bytes, bound 128 (record stale) +RECORD gdn_prep tf_gdn_prep_256, 16 rows or fewer: .usage = .{ .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 146, .static = 0, .bound = 128 }, .{ .regs = 145, .static = 0, .bound = 128 }, .{ .regs = 145, .static = 0, .bound = 128 }, } +PASS limits sm_80 qk_rows tf_qk_rows at dim 64: 30 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_86 qk_rows tf_qk_rows at dim 64: 36 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_89 qk_rows tf_qk_rows at dim 64: 36 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_120 qk_rows tf_qk_rows at dim 64: 29 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_121 qk_rows tf_qk_rows at dim 64: 29 registers, 32 static shared bytes, bound 256 (record stale) +RECORD qk_rows tf_qk_rows at dim 64: .usage = .{ .{ .regs = 30, .static = 32, .bound = 256 }, .{ .regs = 36, .static = 32, .bound = 256 }, .{ .regs = 36, .static = 32, .bound = 256 }, .{ .regs = 29, .static = 32, .bound = 256 }, .{ .regs = 29, .static = 32, .bound = 256 }, } +PASS limits sm_80 qk_rows tf_qk_rows at dim 256: 30 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_86 qk_rows tf_qk_rows at dim 256: 36 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_89 qk_rows tf_qk_rows at dim 256: 36 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_120 qk_rows tf_qk_rows at dim 256: 29 registers, 32 static shared bytes, bound 256 (record stale) +PASS limits sm_121 qk_rows tf_qk_rows at dim 256: 29 registers, 32 static shared bytes, bound 256 (record stale) +RECORD qk_rows tf_qk_rows at dim 256: .usage = .{ .{ .regs = 30, .static = 32, .bound = 256 }, .{ .regs = 36, .static = 32, .bound = 256 }, .{ .regs = 36, .static = 32, .bound = 256 }, .{ .regs = 29, .static = 32, .bound = 256 }, .{ .regs = 29, .static = 32, .bound = 256 }, } +PASS limits sm_80 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 qk_rows tf_qk_rows_w: 40 registers, 0 static shared bytes, bound 128 (record stale) +RECORD qk_rows tf_qk_rows_w: .usage = .{ .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, .{ .regs = 40, .static = 0, .bound = 128 }, } +PASS limits sm_80 affine4 verify32 one CTA a tile: 79 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 affine4 verify32 one CTA a tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 affine4 verify32 one CTA a tile: 80 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 affine4 verify32 one CTA a tile: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 affine4 verify32 one CTA a tile: 72 registers, 0 static shared bytes, bound 128 (record stale) +RECORD affine4 verify32 one CTA a tile: .usage = .{ .{ .regs = 79, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 80, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, } +N/A limits affine4 verify32 cluster of 8 sm_80: below_minimum +N/A limits affine4 verify32 cluster of 8 sm_86: below_minimum +N/A limits affine4 verify32 cluster of 8 sm_89: below_minimum +PASS limits sm_120 affine4 verify32 cluster of 8: 72 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 affine4 verify32 cluster of 8: 72 registers, 0 static shared bytes, bound 128 (record stale) +RECORD affine4 verify32 cluster of 8: .usage = .{ .{ .regs = 72, .static = 0, .bound = 128 }, .{ .regs = 72, .static = 0, .bound = 128 }, } +PASS limits sm_80 affine4 verify64 one CTA a tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_86 affine4 verify64 one CTA a tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_89 affine4 verify64 one CTA a tile: 124 registers, 0 static shared bytes, bound 128 (record current) +PASS limits sm_120 affine4 verify64 one CTA a tile: 122 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 affine4 verify64 one CTA a tile: 122 registers, 0 static shared bytes, bound 128 (record stale) +RECORD affine4 verify64 one CTA a tile: .usage = .{ .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 124, .static = 0, .bound = 128 }, .{ .regs = 122, .static = 0, .bound = 128 }, .{ .regs = 122, .static = 0, .bound = 128 }, } +N/A limits affine4 verify64 cluster of 8 sm_80: below_minimum +N/A limits affine4 verify64 cluster of 8 sm_86: below_minimum +N/A limits affine4 verify64 cluster of 8 sm_89: below_minimum +PASS limits sm_120 affine4 verify64 cluster of 8: 122 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 affine4 verify64 cluster of 8: 122 registers, 0 static shared bytes, bound 128 (record stale) +RECORD affine4 verify64 cluster of 8: .usage = .{ .{ .regs = 122, .static = 0, .bound = 128 }, .{ .regs = 122, .static = 0, .bound = 128 }, } +PASS limits sm_80 affine4 verify128 one CTA a tile: 242 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_86 affine4 verify128 one CTA a tile: 242 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_89 affine4 verify128 one CTA a tile: 242 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_120 affine4 verify128 one CTA a tile: 252 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 affine4 verify128 one CTA a tile: 252 registers, 0 static shared bytes, bound 128 (record stale) +RECORD affine4 verify128 one CTA a tile: .usage = .{ .{ .regs = 242, .static = 0, .bound = 128 }, .{ .regs = 242, .static = 0, .bound = 128 }, .{ .regs = 242, .static = 0, .bound = 128 }, .{ .regs = 252, .static = 0, .bound = 128 }, .{ .regs = 252, .static = 0, .bound = 128 }, } +N/A limits affine4 verify128 cluster of 8 sm_80: below_minimum +N/A limits affine4 verify128 cluster of 8 sm_86: below_minimum +N/A limits affine4 verify128 cluster of 8 sm_89: below_minimum +PASS limits sm_120 affine4 verify128 cluster of 8: 252 registers, 0 static shared bytes, bound 128 (record stale) +PASS limits sm_121 affine4 verify128 cluster of 8: 252 registers, 0 static shared bytes, bound 128 (record stale) +RECORD affine4 verify128 cluster of 8: .usage = .{ .{ .regs = 252, .static = 0, .bound = 128 }, .{ .regs = 252, .static = 0, .bound = 128 }, } +RED limits sm_80 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_80: launched at 512 threads past its __launch_bounds__ 256 +RED limits sm_86 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_86: launched at 512 threads past its __launch_bounds__ 256 +RED limits sm_89 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_89: launched at 512 threads past its __launch_bounds__ 256 +RED limits sm_120 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_120: launched at 512 threads past its __launch_bounds__ 256 +RED limits sm_121 control: past its bounds: 10 registers, 0 static shared bytes, bound 256 (record current): control: past its bounds on sm_121: launched at 512 threads past its __launch_bounds__ 256 +RECORD control: past its bounds: .usage = .{ .{ .regs = 10, .static = 0, .bound = 256 }, .{ .regs = 10, .static = 0, .bound = 256 }, .{ .regs = 10, .static = 0, .bound = 256 }, .{ .regs = 10, .static = 0, .bound = 256 }, .{ .regs = 10, .static = 0, .bound = 256 }, } +RED limits sm_80 control: heavy at 1,024 threads: 168 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_80: 168 registers x 1024 threads need 172032, a block has 65536 +RED limits sm_86 control: heavy at 1,024 threads: 166 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_86: 166 registers x 1024 threads need 172032, a block has 65536 +RED limits sm_89 control: heavy at 1,024 threads: 166 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_89: 166 registers x 1024 threads need 172032, a block has 65536 +RED limits sm_120 control: heavy at 1,024 threads: 166 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_120: 166 registers x 1024 threads need 172032, a block has 65536 +RED limits sm_121 control: heavy at 1,024 threads: 166 registers, 0 static shared bytes, bound 0 (record current): control: heavy at 1,024 threads on sm_121: 166 registers x 1024 threads need 172032, a block has 65536 +RECORD control: heavy at 1,024 threads: .usage = .{ .{ .regs = 168, .static = 0, .bound = 0 }, .{ .regs = 166, .static = 0, .bound = 0 }, .{ .regs = 166, .static = 0, .bound = 0 }, .{ .regs = 166, .static = 0, .bound = 0 }, .{ .regs = 166, .static = 0, .bound = 0 }, } +FAIL kernel-contract-limits: 328 problems +FAIL kernel-contract-limits: TestFailed diff --git a/docs/receipts/glm-kv-pages-0b094b74/records-sha256.txt b/docs/receipts/glm-kv-pages-0b094b74/records-sha256.txt new file mode 100644 index 0000000000..06836d46ae --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/records-sha256.txt @@ -0,0 +1,3 @@ +30bf72eab801335e2014cc0aac73a786b258234a74aa13594193bfd4db675c34 3121337 tp2/serve/r2all/rows-without-bodies.jsonl +b20e699f3c18693d522d3aa62ff5a8d4f2960a3c9d2abdf8db2f870a124449be 3121342 tp2/serve/r2on/rows-without-bodies.jsonl +ceccdebd6013c42c9562fe33cbae8c3b05c215ffddd6224d76c6b0a50a560d98 3054313 tp3/serve/r6auto/rows-without-bodies.jsonl diff --git a/docs/receipts/glm-kv-pages-0b094b74/redaction-report.json b/docs/receipts/glm-kv-pages-0b094b74/redaction-report.json new file mode 100644 index 0000000000..242b1dfbab --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/redaction-report.json @@ -0,0 +1,162 @@ +{ + "policy": "CRLF line ends made LF and trailing blanks dropped; machine names, internal addresses, home and local paths, and the work's session, plan, records, container and image names replaced with placeholders (host-a, host-b and host-c run TP=3's ranks 0, 1 and 2, host-d and host-e TP=2's ranks 0 and 1; and name the direct links' and the ring's addresses); result values, token ids and token hashes kept.", + "replacements_by_file": { + "README.md": 0, + "binaries.txt": 0, + "ci/flashnext-served.log": 0, + "ci/fuzz-cortheon.log": 0, + "ci/glm-host.log": 2, + "ci/lean.txt": 0, + "ci/mutations/auto-at-two.log": 5, + "ci/mutations/bind-before-pages.log": 3, + "ci/mutations/bytes-on-paged-seq.log": 3, + "ci/mutations/close-keeps-pages.log": 3, + "ci/mutations/fp8-kernel-table.log": 4, + "ci/mutations/fp8-on-pages.log": 2, + "ci/mutations/head.log": 2, + "ci/mutations/kept-window-cache.log": 5, + "ci/mutations/map-any-id.log": 2, + "ci/mutations/no-poison.log": 3, + "ci/mutations/page-bytes-unchecked.log": 16, + "ci/mutations/paged-first-at-cache.log": 4, + "ci/mutations/paged-table-kept.log": 2, + "ci/mutations/piece-first-page.log": 7, + "ci/mutations/piece-no-offset.log": 3, + "ci/mutations/split-no-slots.log": 22, + "ci/security.log": 0, + "ci/test-golden.log": 0, + "ci/test-kernel-contract.log": 0, + "ci/test.log": 2, + "commit.txt": 0, + "gate5/build-nvcc13.0.log": 0, + "gate5/build-nvcc13.3.log": 0, + "gate5/limits-nvcc13.0.log": 0, + "gate5/limits-nvcc13.3.log": 0, + "records-sha256.txt": 0, + "runs/tp2-chain.txt": 40, + "single-main/kernel-contract.log": 0, + "single/glm-paged-check.log": 0, + "single/kernel-contract.log": 0, + "single/reg/glm-lanes-concurrent.log": 0, + "single/reg/glm-lanes-cover.log": 0, + "single/reg/glm-shared-check.log": 0, + "single/reg/paged.log": 0, + "single/reg/ref.log": 0, + "single/reg/summary.log": 0, + "single/reg/synth.log": 0, + "tools/kvp_admit.py": 0, + "tools/kvp_sum.py": 0, + "tools/media_http.py": 0, + "tools/media_resume_http.py": 0, + "tp2/check/glm-paged-check-rank0.log": 0, + "tp2/check/glm-paged-check-rank1.log": 0, + "tp2/compare/auto-against-all.log": 0, + "tp2/compare/auto-against-on.log": 0, + "tp2/compare/main-against-all.log": 0, + "tp2/compare/main-against-on.log": 0, + "tp2/serve/r2all/client.log": 0, + "tp2/serve/r2all/conc.json": 0, + "tp2/serve/r2all/rows-without-bodies.jsonl": 0, + "tp2/serve/r2all/server-rank0.log": 121, + "tp2/serve/r2all/server-rank1.log": 114, + "tp2/serve/r2auto/client.log": 0, + "tp2/serve/r2auto/fresh.jsonl": 0, + "tp2/serve/r2auto/server-rank0.log": 121, + "tp2/serve/r2auto/server-rank1.log": 114, + "tp2/serve/r2main/client.log": 0, + "tp2/serve/r2main/fresh.jsonl": 0, + "tp2/serve/r2main/server-rank0.log": 121, + "tp2/serve/r2main/server-rank1.log": 114, + "tp2/serve/r2on/client.log": 0, + "tp2/serve/r2on/conc.json": 0, + "tp2/serve/r2on/rows-without-bodies.jsonl": 0, + "tp2/serve/r2on/server-rank0.log": 121, + "tp2/serve/r2on/server-rank1.log": 114, + "tp2/serve/r2vis/client.log": 0, + "tp2/serve/r2vis/media-resume.json": 0, + "tp2/serve/r2vis/media.json": 0, + "tp2/serve/r2vis/server-rank0.log": 121, + "tp2/serve/r2vis/server-rank1.log": 114, + "tp2/speed/r3k2a/cache.json": 0, + "tp2/speed/r3k2a/client.log": 0, + "tp2/speed/r3k2a/cold.json": 0, + "tp2/speed/r3k2a/decode.json": 0, + "tp2/speed/r3k2a/server-rank0.log": 121, + "tp2/speed/r3k2a/server-rank1.log": 114, + "tp2/speed/r3k2all/cache.json": 0, + "tp2/speed/r3k2all/client.log": 0, + "tp2/speed/r3k2all/cold.json": 0, + "tp2/speed/r3k2all/decode.json": 0, + "tp2/speed/r3k2all/server-rank0.log": 121, + "tp2/speed/r3k2all/server-rank1.log": 114, + "tp2/speed/r3k2b/cache.json": 0, + "tp2/speed/r3k2b/client.log": 0, + "tp2/speed/r3k2b/cold.json": 0, + "tp2/speed/r3k2b/decode.json": 0, + "tp2/speed/r3k2b/server-rank0.log": 121, + "tp2/speed/r3k2b/server-rank1.log": 114, + "tp2/speed/r3maink2/cache.json": 0, + "tp2/speed/r3maink2/client.log": 0, + "tp2/speed/r3maink2/cold.json": 0, + "tp2/speed/r3maink2/decode.json": 0, + "tp2/speed/r3maink2/server-rank0.log": 121, + "tp2/speed/r3maink2/server-rank1.log": 114, + "tp2/speed/r3maink22/cache.json": 0, + "tp2/speed/r3maink22/client.log": 0, + "tp2/speed/r3maink22/cold.json": 0, + "tp2/speed/r3maink22/decode.json": 0, + "tp2/speed/r3maink22/server-rank0.log": 121, + "tp2/speed/r3maink22/server-rank1.log": 114, + "tp3/build.log": 0, + "tp3/chain.log": 10, + "tp3/check/glm-paged-check-rank0.log": 10850, + "tp3/check/glm-paged-check-rank1.log": 2499, + "tp3/check/glm-paged-check-rank2.log": 14550, + "tp3/found-b7fecb3d/r6all/client.log": 0, + "tp3/found-b7fecb3d/r6all/conc.json": 0, + "tp3/found-b7fecb3d/r6all/hostwatch-rank0.log": 0, + "tp3/found-b7fecb3d/r6all/hostwatch-rank1.log": 0, + "tp3/found-b7fecb3d/r6all/hostwatch-rank2.log": 0, + "tp3/found-b7fecb3d/r6all/server-rank0.log": 31199, + "tp3/found-b7fecb3d/r6all/server-rank1.log": 5489, + "tp3/found-b7fecb3d/r6all/server-rank2.log": 3016, + "tp3/ring.patch": 2, + "tp3/serve/r6admit/admit.json": 0, + "tp3/serve/r6admit/client.log": 0, + "tp3/serve/r6admit/hostwatch-rank0.log": 0, + "tp3/serve/r6admit/hostwatch-rank1.log": 0, + "tp3/serve/r6admit/hostwatch-rank2.log": 0, + "tp3/serve/r6admit/server-rank0.log": 9541, + "tp3/serve/r6admit/server-rank1.log": 8029, + "tp3/serve/r6admit/server-rank2.log": 7449, + "tp3/serve/r6all/client.log": 0, + "tp3/serve/r6all/conc.json": 0, + "tp3/serve/r6all/hostwatch-rank0.log": 0, + "tp3/serve/r6all/hostwatch-rank1.log": 0, + "tp3/serve/r6all/hostwatch-rank2.log": 0, + "tp3/serve/r6all/server-rank0.log": 26972, + "tp3/serve/r6all/server-rank1.log": 6645, + "tp3/serve/r6all/server-rank2.log": 1346, + "tp3/serve/r6auto/client.log": 0, + "tp3/serve/r6auto/cold.json": 0, + "tp3/serve/r6auto/conc.json": 0, + "tp3/serve/r6auto/hostwatch-rank0.log": 0, + "tp3/serve/r6auto/hostwatch-rank1.log": 0, + "tp3/serve/r6auto/hostwatch-rank2.log": 0, + "tp3/serve/r6auto/long-beside-short.json": 0, + "tp3/serve/r6auto/rows-without-bodies.jsonl": 0, + "tp3/serve/r6auto/server-rank0.log": 13553, + "tp3/serve/r6auto/server-rank1.log": 4890, + "tp3/serve/r6auto/server-rank2.log": 6518, + "tp3/serve/r6off/client.log": 0, + "tp3/serve/r6off/compare-auto.log": 0, + "tp3/serve/r6off/fresh.jsonl": 0, + "tp3/serve/r6off/hostwatch-rank0.log": 0, + "tp3/serve/r6off/hostwatch-rank1.log": 0, + "tp3/serve/r6off/hostwatch-rank2.log": 0, + "tp3/serve/r6off/server-rank0.log": 23562, + "tp3/serve/r6off/server-rank1.log": 5054, + "tp3/serve/r6off/server-rank2.log": 1047 + }, + "undecodable_bytes_replaced_by_file": {} +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/runs/tp2-chain.txt b/docs/receipts/glm-kv-pages-0b094b74/runs/tp2-chain.txt new file mode 100644 index 0000000000..df4b036ad5 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/runs/tp2-chain.txt @@ -0,0 +1,90 @@ +kvp-k3 start 23:39:24 +04: 60 logs aside +05: 11 logs aside +2026-10-22 23:39:29 fmt rc=0 +2026-10-22 23:44:03 build rc=0 274s: +2026-10-22 23:46:34 native rc=0 151s: +2026-10-22 23:47:01 host rc=0 27s: failed command: ./.zig-cache/o/84343b39e4dcdabee01ced98805076de/test --cache-dir=./.zig-cache --seed=0x551fd05b --listen=- failed command: ./.zig-cache/o/ac05dba1db5 +2026-10-22 23:47:06 kvpool rc=0 5s: Build Summary: 5/5 steps succeeded; 16/16 tests passed +2026-10-22 23:47:48 test rc=0 42s: failed command: ./.zig-cache/o/7250ba077cad6049f6c43b354768d2f0/test --cache-dir=./.zig-cache --seed=0x14b06dc --listen=- Build Summary: 276/276 steps succeeded; 150 +2026-10-22 23:47:49 lean rc=1 1s: +d82043de5d8b1f04 3d28137d172f1584 05: d82043de5d8b1f04 +kvp-k2 start 23:51:07 +=== kvp-check pg2k2 rc 0 0 +started rank 1 (r2all) on host-e +started rank 0 (r2all) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2all rc 0 +started rank 1 (r2on) on host-e +started rank 0 (r2on) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2on rc 0 +started rank 1 (r2auto) on host-e +started rank 0 (r2auto) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2auto rc 0 +started rank 1 (r2main) on host-e +started rank 0 (r2main) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2main rc 0 +compare auto-on rc=0 +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh +compare auto-all rc=0 +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh +compare main-on rc=0 +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh +compare main-all rc=0 +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh +=== kvp-r2m done +2026-10-23 01:22:17 === kvp-reg k2 done 0 failed +=== kvp-check pg1k2 rc 0 +2026-10-23 01:29:09 === kvp-base tp3 done +2026-10-23 01:24:46 build rc=0 306s: +2026-10-23 01:27:17 native rc=0 151s: +2026-10-23 01:28:27 host rc=0 69s: failed command: ./.zig-cache/o/5b21a6c856bcc580edb3128b41ac4e50/test --cache-dir=./.zig-cache --seed=0xaeb51753 --listen=- failed command: ./.zig-cache/o/50bc9f88b9a +2026-10-23 01:28:33 kvpool rc=0 6s: Build Summary: 5/5 steps succeeded; 6/6 tests passed +2026-10-23 01:29:09 test rc=0 36s: failed command: ./.zig-cache/o/7250ba077cad6049f6c43b354768d2f0/test --cache-dir=./.zig-cache --seed=0xe551a39b --listen=- Build Summary: 276/276 steps succeeded; 91 +2026-10-23 01:29:09 lean rc=1 0s: +2026-10-23 01:29:09 === kvp-base tp3 done +62649bbd8c2ce0d4 +=== kvp-post3 done +started rank 1 (r2vis) on host-e +started rank 0 (r2vis) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2vis rc 0 +=== kvp-k2 done 01:38:32 +started rank 1 (r3k2a) on host-e +started rank 0 (r3k2a) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3k2a rc 0 +started rank 1 (r3maink2) on host-e +started rank 0 (r3maink2) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3maink2 rc 0 +started rank 1 (r3k2b) on host-e +started rank 0 (r3k2b) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3k2b rc 0 +started rank 1 (r3maink22) on host-e +started rank 0 (r3maink22) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3maink22 rc 0 +started rank 1 (r3k2all) on host-e +started rank 0 (r3k2all) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3k2all rc 0 +=== kvp-k3 done 03:00:30 diff --git a/docs/receipts/glm-kv-pages-0b094b74/single-main/kernel-contract.log b/docs/receipts/glm-kv-pages-0b094b74/single-main/kernel-contract.log new file mode 100644 index 0000000000..1a301b2614 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single-main/kernel-contract.log @@ -0,0 +1,3112 @@ +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (flashnext_fact.fatbin) w +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (flashnext_attn.fatbin) w +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (prefill_attention.fatbin) w +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (fp8_experts.fatbin) w +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (drama3_fp8.fatbin) w +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (glm_mm.fatbin) w +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + + +PASS nonfinite cuda RMS norm tf_rms_rows_bf16 (16-byte form): 2692 non-finite and 18812 finite outputs match the float64 classes +PASS repeat cuda RMS norm tf_rms_rows_bf16 (16-byte form): five runs byte-identical (91392 bytes), inputs and guards untouched, no overlap +PASS shapes cuda RMS norm tf_rms_rows_bf16 (16-byte form): 2 shapes within the float64 bound (worst 0.995 of the bound), 5 edges refused by the kernel +PASS sums cuda RMS norm tf_rms_rows_bf16 (16-byte form): within its bound at k 7168; fp16 sums would miss 256x, bf16 0.6x (a bf16 output follows the sum of squares as ss^-1/2: a bf16 sum's error is a few output ulps (the fp32-out form proves it)) +PASS nonfinite cuda RMS norm tf_rms_rows (f16 in, fp32 out): 2692 non-finite and 18812 finite outputs match the float64 classes +PASS repeat cuda RMS norm tf_rms_rows (f16 in, fp32 out): five runs byte-identical (182784 bytes), inputs and guards untouched, no overlap +PASS shapes cuda RMS norm tf_rms_rows (f16 in, fp32 out): 2 shapes within the float64 bound (worst 0.186 of the bound), 5 edges refused by the kernel +PASS sums cuda RMS norm tf_rms_rows (f16 in, fp32 out): exact sums at k 9216, where fp16 sums would miss by 1864135x and bf16 by 3137x +FAIL nonfinite control: fmax row max: 1 of 8 outputs differ in class; first at 0: finite where float64 gives nan +CONTROL control: fmax row max: red at nonfinite, as planted +FAIL repeat control: atomic-add row sum: 4 of 4 repeats differ from the first run's bytes, first at byte 8 +CONTROL control: atomic-add row sum: red at repeat, as planted +FAIL repeat control: row sum writing its input: 4 of 4 repeats differ from the first run's bytes, first at byte 0 +CONTROL control: row sum writing its input: red at repeat, as planted +FAIL shapes control: row sum losing a tail input: at 8x1x1 output 0 is 0.000000e0 where float64 gives -5.117188e-1: 2396744.29 of its bound +CONTROL control: row sum losing a tail input: red at shapes, as planted +FAIL sums control: dot summed in half: 79310475.6x over its exact-sum bound at k 4096: not fp32 sums (fp16 would miss 79310476x, bf16 16777216x) +CONTROL control: dot summed in half: red at sums, as planted +PASS nonfinite cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), output y: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), output y: five runs byte-identical (265200 bytes), inputs and guards untouched, no overlap +PASS shapes cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), output y: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), output y: within its bound at k 5376; fp16 sums would miss 51x, bf16 0.7x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), residual h: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), residual h: five runs byte-identical (265200 bytes), inputs and guards untouched, no overlap +PASS shapes cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), residual h: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), residual h: within its bound at k 5376; fp16 sums would miss 51x, bf16 0.5x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), output y: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), output y: five runs byte-identical (265200 bytes), inputs and guards untouched, no overlap +PASS shapes cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), output y: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), output y: within its bound at k 5376; fp16 sums would miss 51x, bf16 0.4x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), residual h: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), residual h: five runs byte-identical (265200 bytes), inputs and guards untouched, no overlap +PASS shapes cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), residual h: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), residual h: within its bound at k 5376; fp16 sums would miss 51x, bf16 0.4x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +FAIL shapes control: MLX tail without the branch's normalized-row rounding: at 17x3840x3840: row 0 h[6] = 4.1015625e-1 outside float64's [4.12109375e-1, 4.12109375e-1] +CONTROL control: MLX tail without the branch's normalized-row rounding: red at shapes, as planted +FAIL shapes control: MLX tail without the branch output's rounding: at 17x3840x3840: row 0 h[2] = -1.484375e0 outside float64's [-1.4921875e0, -1.4921875e0] +CONTROL control: MLX tail without the branch output's rounding: red at shapes, as planted +FAIL shapes control: MLX tail without the sum's rounding before the scalar: at 17x3840x3840: row 0 h[0] = -1.7480469e-1 outside float64's [-1.7578125e-1, -1.7578125e-1] +CONTROL control: MLX tail without the sum's rounding before the scalar: red at shapes, as planted +FAIL shapes control: MLX tail without the output's normalized-row rounding: at 17x3840x3840: row 0 y[10] = 1.0703125e0 outside float64's [1.078125e0, 1.078125e0] +CONTROL control: MLX tail without the output's normalized-row rounding: red at shapes, as planted +PASS nonfinite cuda nemotron router logits tf_nemo_router: 640 non-finite and 5504 finite outputs match the float64 classes +PASS repeat cuda nemotron router logits tf_nemo_router: five runs byte-identical (49152 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron router logits tf_nemo_router: 9 shapes within the float64 bound (worst 0.003 of the bound), 0 edges refused by the kernel +PASS sums cuda nemotron router logits tf_nemo_router: exact sums at k 2688, where fp16 sums would miss by 15316x and bf16 by 107354x +PASS nonfinite cuda nemotron router logits tf_nemo_router_tiles: 640 non-finite and 5504 finite outputs match the float64 classes +PASS repeat cuda nemotron router logits tf_nemo_router_tiles: five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron router logits tf_nemo_router_tiles: 12 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda nemotron router logits tf_nemo_router_tiles: exact sums at k 4096, where fp16 sums would miss by 20826x and bf16 by 177109x +PASS nonfinite cuda nemotron top-k tf_nemo_topk: the stated rule holds: a NaN logit is never picked ahead of a number; ties go to the lower expert; two shared slots follow the top-k +PASS repeat cuda nemotron top-k tf_nemo_topk: five runs byte-identical (1024 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron top-k tf_nemo_topk: 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda nemotron top-k tf_nemo_topk: declares none sums +PASS nonfinite cuda nemotron latent top-k tf_super_topk_few: the stated rule holds: tf_super_topk's bytes, and its host twin's picks: a NaN never ahead of a number, ties to the lower expert, every pick past the numbers expert 0 +PASS repeat cuda nemotron latent top-k tf_super_topk_few: five runs byte-identical (264 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron latent top-k tf_super_topk_few: 7 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda nemotron latent top-k tf_super_topk_few: declares none sums +PASS nonfinite cuda nemotron mamba scan tf_nemo_scan: the stated rule holds: glue_ref.scan's bytes (y, the window's dt, the state after the kept rows), NaN wherever it gives NaN +PASS repeat cuda nemotron mamba scan tf_nemo_scan: five runs byte-identical (4489216 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron mamba scan tf_nemo_scan: 11 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda nemotron mamba scan tf_nemo_scan: declares none sums +PASS nonfinite cuda nemotron latent combine tf_super_combine_sums (bf16 share sums): the stated rule holds: as the kernel it replaces gives them from the fp32 shares: NaN and infinite shares and weights +PASS repeat cuda nemotron latent combine tf_super_combine_sums (bf16 share sums): five runs byte-identical (131072 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron latent combine tf_super_combine_sums (bf16 share sums): 4 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda nemotron latent combine tf_super_combine_sums (bf16 share sums): exact sums at k 22, where fp16 sums would miss by infx and bf16 by 3840x +PASS nonfinite cuda nemotron latent shared add tf_super_moe_out_sums (bf16 share sums): the stated rule holds: as the kernel it replaces gives them from the fp32 shares: NaN and infinite shares and weights +PASS repeat cuda nemotron latent shared add tf_super_moe_out_sums (bf16 share sums): five runs byte-identical (524288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron latent shared add tf_super_moe_out_sums (bf16 share sums): 6 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda nemotron latent shared add tf_super_moe_out_sums (bf16 share sums): within its bound at k 1; fp16 sums would miss infx, bf16 1.0x (one fp32 add of two bf16 values a output) +PASS nonfinite cuda qmm_group (cluster tile): 41216 non-finite and 41216 finite outputs match the float64 classes +PASS repeat cuda qmm_group (cluster tile): five runs byte-identical (329728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qmm_group (cluster tile): 12 shapes within the float64 bound (worst 0.981 of the bound), 0 edges refused by the kernel +PASS sums cuda qmm_group (cluster tile): exact sums at k 4096, where fp16 sums would miss by 8671x and bf16 by 52173x +PASS nonfinite cuda lane_gemv: 41216 non-finite and 41216 finite outputs match the float64 classes +PASS repeat cuda lane_gemv: five runs byte-identical (329728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda lane_gemv: 12 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda lane_gemv: exact sums at k 4096, where fp16 sums would miss by 11313x and bf16 by 59663x +PASS nonfinite cuda lane_gemv split-K: 41216 non-finite and 41216 finite outputs match the float64 classes +PASS repeat cuda lane_gemv split-K: five runs byte-identical (329728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda lane_gemv split-K: 8 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 4 edges refused by the kernel +PASS sums cuda lane_gemv split-K: exact sums at k 4096, where fp16 sums would miss by 9203x and bf16 by 73828x +PASS nonfinite cuda prompt16 FP4 tile 4: 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda prompt16 FP4 tile 4: five runs byte-identical (65536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda prompt16 FP4 tile 4: 13 shapes within the float64 bound (worst 0.994 of the bound), 0 edges refused by the kernel +PASS sums cuda prompt16 FP4 tile 4: exact sums at k 2560, where fp16 sums would miss by 416x and bf16 by 5985x +PASS nonfinite cuda prompt16 FP4 tile 4, fp32 out: 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda prompt16 FP4 tile 4, fp32 out: five runs byte-identical (131072 bytes), inputs and guards untouched, no overlap +PASS shapes cuda prompt16 FP4 tile 4, fp32 out: 15 shapes within the float64 bound (worst 0.099 of the bound), 0 edges refused by the kernel +PASS sums cuda prompt16 FP4 tile 4, fp32 out: exact sums at k 3072, where fp16 sums would miss by 19387004x and bf16 by 195734179x +PASS nonfinite cuda qmmf FP4 fp32 out (slices in a cluster): 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda qmmf FP4 fp32 out (slices in a cluster): five runs byte-identical (32768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qmmf FP4 fp32 out (slices in a cluster): 18 shapes within the float64 bound (worst 0.034 of the bound), 0 edges refused by the kernel +PASS sums cuda qmmf FP4 fp32 out (slices in a cluster): exact sums at k 6144, where fp16 sums would miss by 49982121x and bf16 by 512104525x +PASS nonfinite cuda qmmf FP4 fp32 out (slices by reduce_kernel): 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda qmmf FP4 fp32 out (slices by reduce_kernel): five runs byte-identical (32768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qmmf FP4 fp32 out (slices by reduce_kernel): 18 shapes within the float64 bound (worst 0.038 of the bound), 0 edges refused by the kernel +PASS sums cuda qmmf FP4 fp32 out (slices by reduce_kernel): exact sums at k 6144, where fp16 sums would miss by 71303165x and bf16 by 609572157x +PASS nonfinite cuda affine4 verify32 one-row bits: 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda affine4 verify32 one-row bits: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify32 one-row bits: 39 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify32 one-row bits: exact sums at k 25600, where fp16 sums would miss by 10555x and bf16 by 83630x +PASS nonfinite cuda affine4 verify32 FP64: 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda affine4 verify32 FP64: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify32 FP64: 37 shapes within the float64 bound (worst 0.971 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify32 FP64: exact sums at k 25600, where fp16 sums would miss by 9007x and bf16 by 72060x +PASS nonfinite cuda affine4 verify64 one-row bits: 260 non-finite and 260 finite outputs match the float64 classes +PASS repeat cuda affine4 verify64 one-row bits: five runs byte-identical (4290 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify64 one-row bits: 35 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify64 one-row bits: exact sums at k 25600, where fp16 sums would miss by 8939x and bf16 by 87125x +PASS nonfinite cuda affine4 verify64 FP64: 260 non-finite and 260 finite outputs match the float64 classes +PASS repeat cuda affine4 verify64 FP64: five runs byte-identical (4290 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify64 FP64: 33 shapes within the float64 bound (worst 0.967 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify64 FP64: exact sums at k 25600, where fp16 sums would miss by 10594x and bf16 by 110120x +PASS nonfinite cuda affine4 verify128 one-row bits: 260 non-finite and 260 finite outputs match the float64 classes +PASS repeat cuda affine4 verify128 one-row bits: five runs byte-identical (8320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify128 one-row bits: 39 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify128 one-row bits: exact sums at k 25600, where fp16 sums would miss by 7225x and bf16 by 256x +PASS nonfinite cuda affine4 verify128 FP64: 260 non-finite and 260 finite outputs match the float64 classes +PASS repeat cuda affine4 verify128 FP64: five runs byte-identical (8320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify128 FP64: 37 shapes within the float64 bound (worst 0.976 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify128 FP64: exact sums at k 25600, where fp16 sums would miss by 7657x and bf16 by 52173x +FAIL shapes control: verify32 truncated one-row bits: at 17x65x512 its bytes differ from the original one-row affine4 decode's from byte 2080 +CONTROL control: verify32 truncated one-row bits: red at shapes, as planted +FAIL shapes control: verify32 truncated FP64: at 17x65x512 output 1040 is -nan where float64 gives 2.149148e0: nan of its bound +CONTROL control: verify32 truncated FP64: red at shapes, as planted +FAIL shapes control: verify64 truncated one-row bits: at 33x65x512 its bytes differ from the original one-row affine4 decode's from byte 2080 +CONTROL control: verify64 truncated one-row bits: red at shapes, as planted +FAIL shapes control: verify64 truncated FP64: at 33x65x512 output 1040 is -nan where float64 gives -1.232138e0: nan of its bound +CONTROL control: verify64 truncated FP64: red at shapes, as planted +FAIL shapes control: verify128 truncated one-row bits: at 65x65x512 its bytes differ from the original one-row affine4 decode's from byte 2080 +CONTROL control: verify128 truncated one-row bits: red at shapes, as planted +FAIL shapes control: verify128 truncated FP64: at 65x65x512 output 1040 is -nan where float64 gives 4.566685e0: nan of its bound +CONTROL control: verify128 truncated FP64: red at shapes, as planted +PASS nonfinite cuda MXFP8 lane clustered slices: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda MXFP8 lane clustered slices: five runs byte-identical (140 bytes), inputs and guards untouched, no overlap +PASS shapes cuda MXFP8 lane clustered slices: 19 shapes within the float64 bound (worst 0.992 of the bound), 0 edges refused by the kernel +PASS sums cuda MXFP8 lane clustered slices: exact sums at k 2560, where fp16 sums would miss by 593x and bf16 by 5030x +PASS nonfinite cuda MXFP8 lane explicit slice reduction: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda MXFP8 lane explicit slice reduction: five runs byte-identical (140 bytes), inputs and guards untouched, no overlap +PASS shapes cuda MXFP8 lane explicit slice reduction: 19 shapes within the float64 bound (worst 0.985 of the bound), 0 edges refused by the kernel +PASS sums cuda MXFP8 lane explicit slice reduction: exact sums at k 2560, where fp16 sums would miss by 515x and bf16 by 5059x +PASS nonfinite cuda BF16 fp32 clustered: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda BF16 fp32 clustered: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda BF16 fp32 clustered: 20 shapes within the float64 bound (worst 0.050 of the bound), 0 edges refused by the kernel +PASS sums cuda BF16 fp32 clustered: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 1791630067x +PASS nonfinite cuda BF16 fp32 reduce: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda BF16 fp32 reduce: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda BF16 fp32 reduce: 20 shapes within the float64 bound (worst 0.041 of the bound), 0 edges refused by the kernel +PASS sums cuda BF16 fp32 reduce: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2273872009x +PASS nonfinite cuda FP8 fp32 clustered: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda FP8 fp32 clustered: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda FP8 fp32 clustered: 20 shapes within the float64 bound (worst 0.111 of the bound), 0 edges refused by the kernel +PASS sums cuda FP8 fp32 clustered: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2482112847x +PASS nonfinite cuda FP8 fp32 reduce: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda FP8 fp32 reduce: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda FP8 fp32 reduce: 20 shapes within the float64 bound (worst 0.128 of the bound), 0 edges refused by the kernel +PASS sums cuda FP8 fp32 reduce: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2651940327x +PASS nonfinite cuda block FP8 fp32 clustered: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda block FP8 fp32 clustered: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda block FP8 fp32 clustered: 20 shapes within the float64 bound (worst 0.034 of the bound), 0 edges refused by the kernel +PASS sums cuda block FP8 fp32 clustered: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2332606604x +PASS nonfinite cuda block FP8 fp32 reduce: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda block FP8 fp32 reduce: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda block FP8 fp32 reduce: 20 shapes within the float64 bound (worst 0.042 of the bound), 0 edges refused by the kernel +PASS sums cuda block FP8 fp32 reduce: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2236320382x +PASS nonfinite cuda MXFP8 fp32 clustered: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda MXFP8 fp32 clustered: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda MXFP8 fp32 clustered: 20 shapes within the float64 bound (worst 0.039 of the bound), 0 edges refused by the kernel +PASS sums cuda MXFP8 fp32 clustered: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2097668222x +PASS nonfinite cuda MXFP8 fp32 reduce: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda MXFP8 fp32 reduce: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda MXFP8 fp32 reduce: 20 shapes within the float64 bound (worst 0.047 of the bound), 0 edges refused by the kernel +PASS sums cuda MXFP8 fp32 reduce: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2505038444x +PASS nonfinite cuda tf_rank_sum (a reduce-scatter's rank-order sum): 4 non-finite and 32764 finite outputs match the float64 classes +PASS repeat cuda tf_rank_sum (a reduce-scatter's rank-order sum): five runs byte-identical (262144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tf_rank_sum (a reduce-scatter's rank-order sum): 7 shapes within the float64 bound (worst 0.550 of the bound), 0 edges refused by the kernel +PASS sums cuda tf_rank_sum (a reduce-scatter's rank-order sum): exact sums at k 5, where fp16 sums would miss by infx and bf16 by 19617358994x +PASS nonfinite cuda Flash Next fn_hc_writeback0: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda Flash Next fn_hc_writeback0: five runs byte-identical (2720 bytes), inputs and guards untouched, no overlap +PASS shapes cuda Flash Next fn_hc_writeback0: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda Flash Next fn_hc_writeback0: exact sums at k 8, where fp16 sums would miss by infx and bf16 by 96849x +PASS nonfinite cuda Flash Next fn_hc_writeback_moe: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda Flash Next fn_hc_writeback_moe: five runs byte-identical (350880 bytes), inputs and guards untouched, no overlap +PASS shapes cuda Flash Next fn_hc_writeback_moe: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda Flash Next fn_hc_writeback_moe: exact sums at k 8, where fp16 sums would miss by infx and bf16 by 88794x +PASS nonfinite cuda Flash Next fn_hc_writeback_slices: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda Flash Next fn_hc_writeback_slices: five runs byte-identical (350880 bytes), inputs and guards untouched, no overlap +PASS shapes cuda Flash Next fn_hc_writeback_slices: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda Flash Next fn_hc_writeback_slices: exact sums at k 8, where fp16 sums would miss by infx and bf16 by 80093x +PASS nonfinite cuda fn_sum_slices f32: the stated rule holds: propagates NaN and infinities; finite outputs use ordered f32 and one final RNE +PASS repeat cuda fn_sum_slices f32: five runs byte-identical (696320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_sum_slices f32: 13 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda fn_sum_slices f32: exact sums at k 8, where fp16 sums would miss by 12582912x and bf16 by 12582912x +PASS nonfinite cuda fn_sum_slices bf16: the stated rule holds: propagates NaN and infinities; finite outputs use ordered f32 and one final RNE +PASS repeat cuda fn_sum_slices bf16: five runs byte-identical (348160 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_sum_slices bf16: 13 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda fn_sum_slices bf16: exact sums at k 8, where fp16 sums would miss by 192x and bf16 by 192x +PASS nonfinite cuda fn_shift_window contiguous: the stated rule holds: copies all bf16 bits, including NaN payloads, infinities and signed zero +PASS repeat cuda fn_shift_window contiguous: five runs byte-identical (184320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_shift_window contiguous: 18 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda fn_shift_window contiguous: declares none sums +PASS nonfinite cuda fn_shift_window strided: the stated rule holds: copies all bf16 bits, including NaN payloads, infinities and signed zero +PASS repeat cuda fn_shift_window strided: five runs byte-identical (184320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_shift_window strided: 18 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda fn_shift_window strided: declares none sums +PASS nonfinite cuda Flash Next fn_row_best: the stated rule holds: NaNs do not win; all NaN/-Inf retain -Inf and INT_MAX; a NaN or infinite maximum makes LSE NaN +PASS repeat cuda Flash Next fn_row_best: five runs byte-identical (192 bytes), inputs and guards untouched, no overlap +PASS shapes cuda Flash Next fn_row_best: 14 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda Flash Next fn_row_best: exact sums at k 248320, where fp16 sums would miss by 2159920x and bf16 by 929x +PASS nonfinite cuda fn_qsa_select packed scores: the stated rule holds: float total order includes signed NaN payloads and signed zero; ties choose earlier blocks +PASS repeat cuda fn_qsa_select packed scores: five runs byte-identical (65728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_qsa_select packed scores: 23 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +N/A sums cuda fn_qsa_select packed scores: declares none sums +PASS nonfinite cuda fn_qsa_select padded scores: the stated rule holds: float total order includes signed NaN payloads and signed zero; ties choose earlier blocks +PASS repeat cuda fn_qsa_select padded scores: five runs byte-identical (65728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_qsa_select padded scores: 23 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +N/A sums cuda fn_qsa_select padded scores: declares none sums +PASS nonfinite cuda h3 modulated norm h3_norm: 5381 non-finite and 80635 finite outputs match the float64 classes +PASS repeat cuda h3 modulated norm h3_norm: five runs byte-identical (344064 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 modulated norm h3_norm: 1 shapes within the float64 bound (worst 0.579 of the bound), 4 edges refused by the kernel +PASS sums cuda h3 modulated norm h3_norm: within its bound at k 5376; fp16 sums would miss 43x, bf16 0.1x (a bf16 output follows the sum of squares as ss^-1/2: a bf16 sum's error is a few output ulps) +PASS nonfinite cuda h3 gated add h3_gate_add: 3 non-finite and 43005 finite outputs match the float64 classes +PASS repeat cuda h3 gated add h3_gate_add: five runs byte-identical (172032 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 gated add h3_gate_add: 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +N/A sums cuda h3 gated add h3_gate_add: declares none sums +PASS nonfinite cuda h3 pooled branch mix h3_gate_mix: 3 non-finite and 2045 finite outputs match the float64 classes +PASS repeat cuda h3 pooled branch mix h3_gate_mix: five runs byte-identical (32768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 pooled branch mix h3_gate_mix: 5 shapes within the float64 bound (worst 0.876 of the bound), 0 edges refused by the kernel +N/A sums cuda h3 pooled branch mix h3_gate_mix: declares none sums +PASS nonfinite cuda h3 SwiGLU h3_swiglu: 3 non-finite and 114685 finite outputs match the float64 classes +PASS repeat cuda h3 SwiGLU h3_swiglu: five runs byte-identical (458752 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 SwiGLU h3_swiglu: 5 shapes within the float64 bound (worst 0.927 of the bound), 0 edges refused by the kernel +N/A sums cuda h3 SwiGLU h3_swiglu: declares none sums +PASS nonfinite cuda h3 head layout h3_heads: 131 non-finite and 786301 finite outputs match the float64 classes +PASS repeat cuda h3 head layout h3_heads: five runs byte-identical (1572864 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 head layout h3_heads: 1 shapes within the float64 bound (worst 0.555 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 head layout h3_heads: within its bound at k 96; fp16 sums would miss 0x, bf16 0.3x (a bf16 output follows the sum of squares as ss^-1/2: a bf16 sum's error is a few output ulps) +PASS nonfinite cuda h3 tile means h3_pool: 1 non-finite and 6143 finite outputs match the float64 classes +PASS repeat cuda h3 tile means h3_pool: five runs byte-identical (49152 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 tile means h3_pool: 1 shapes within the float64 bound (worst 0.009 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 tile means h3_pool: within its bound at k 64; fp16 sums would miss 117x, bf16 865.4x (a mean of at most 64 bf16 values: no sum reaches past fp16's range) +PASS nonfinite cuda h3 tile scores h3_tile_scores: 24 non-finite and 104 finite outputs match the float64 classes +PASS repeat cuda h3 tile scores h3_tile_scores: five runs byte-identical (2048 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 tile scores h3_tile_scores: 5 shapes within the float64 bound (worst 0.131 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 tile scores h3_tile_scores: exact sums at k 128, where fp16 sums would miss by 335544x and bf16 by 11072962x +PASS nonfinite cuda h3 pooled branch h3_coarse: 256 non-finite and 1792 finite outputs match the float64 classes +PASS repeat cuda h3 pooled branch h3_coarse: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 pooled branch h3_coarse: 2 shapes within the float64 bound (worst 0.213 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 pooled branch h3_coarse: within its bound at k 16; fp16 sums would miss 1139x, bf16 8240.4x (softmax weights in [0, 1], rounded to bf16 before the value product: the bf16 output hides the sums' error) +PASS nonfinite cuda h3 routed attention h3_attention: 128 non-finite and 99712 finite outputs match the float64 classes +PASS repeat cuda h3 routed attention h3_attention: five runs byte-identical (432640 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 routed attention h3_attention: 6 shapes within the float64 bound (worst 0.767 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 routed attention h3_attention: within its bound at k 1024; fp16 sums would miss 6x, bf16 61.4x (softmax weights in [0, 1], rounded to bf16 before the value product: the bf16 output hides the sums' error) +PASS nonfinite cuda h3 tile top-k h3_topk: the stated rule holds: a NaN score ranks as -inf, ties go to the lower tile, so every row keeps exactly n tiles +PASS repeat cuda h3 tile top-k h3_topk: five runs byte-identical (143360 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 tile top-k h3_topk: 2 shapes exact by its own check (worst 0.000 of the bound), 4 edges refused by the kernel +N/A sums cuda h3 tile top-k h3_topk: declares none sums +PASS nonfinite cuda drama3 d3_rmsnorm (bf16 rows, plain gamma): 2052 non-finite and 8188 finite outputs match the float64 classes +PASS repeat cuda drama3 d3_rmsnorm (bf16 rows, plain gamma): five runs byte-identical (4096 bytes), inputs and guards untouched, no overlap +PASS shapes cuda drama3 d3_rmsnorm (bf16 rows, plain gamma): 2 shapes within the float64 bound (worst 0.995 of the bound), 4 edges refused by the kernel +PASS sums cuda drama3 d3_rmsnorm (bf16 rows, plain gamma): within its bound at k 2048; fp16 sums would miss 256x, bf16 0.0x (a bf16 output follows the sum of squares as ss^-1/2: a bf16 sum's error is a few output ulps) +PASS nonfinite cuda ck experts up tf_ck4_up_relu2_few: the stated rule holds: as the prompt kernel it replaces gives them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts up tf_ck4_up_relu2_few: five runs byte-identical (10752 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts up tf_ck4_up_relu2_few: 15 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts up tf_ck4_up_relu2_few: exact sums at k 4096, where fp16 sums would miss by 128x and bf16 by 128x +PASS nonfinite cuda ck experts down tf_ck4_down_f32_few: the stated rule holds: as the prompt kernel it replaces gives them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts down tf_ck4_down_f32_few: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts down tf_ck4_down_f32_few: 15 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts down tf_ck4_down_f32_few: exact sums at k 2688, where fp16 sums would miss by infx and bf16 by 14620145x +PASS nonfinite cuda ck experts prompt up tf_ck4_up_nvfp4_prompt (NVFP4 rows out): the stated rule holds: as the kernels it replaces give them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts prompt up tf_ck4_up_nvfp4_prompt (NVFP4 rows out): five runs byte-identical (193536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts prompt up tf_ck4_up_nvfp4_prompt (NVFP4 rows out): 16 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts prompt up tf_ck4_up_nvfp4_prompt (NVFP4 rows out): exact sums at k 4096, where fp16 sums would miss by infx and bf16 by infx +PASS nonfinite cuda ck experts prompt down tf_ck4_down_f32_prompt: the stated rule holds: as the kernels it replaces give them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts prompt down tf_ck4_down_f32_prompt: five runs byte-identical (1048576 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts prompt down tf_ck4_down_f32_prompt: 16 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts prompt down tf_ck4_down_f32_prompt: exact sums at k 2688, where fp16 sums would miss by infx and bf16 by 25165823x +PASS nonfinite cuda ck experts prompt down tf_ck4_down_sum_prompt (two shares summed, bf16): the stated rule holds: as the kernels it replaces give them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts prompt down tf_ck4_down_sum_prompt (two shares summed, bf16): five runs byte-identical (262144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts prompt down tf_ck4_down_sum_prompt (two shares summed, bf16): 16 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts prompt down tf_ck4_down_sum_prompt (two shares summed, bf16): exact sums at k 2688, where fp16 sums would miss by infx and bf16 by 337x +PASS nonfinite cuda ck FP8 prompt tile tf_ck8_128c (slices in a cluster): the stated rule holds: as the 64-row tile it replaces gives them: a NaN input and a NaN weight code +PASS repeat cuda ck FP8 prompt tile tf_ck8_128c (slices in a cluster): five runs byte-identical (524288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck FP8 prompt tile tf_ck8_128c (slices in a cluster): 10 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck FP8 prompt tile tf_ck8_128c (slices in a cluster): exact sums at k 8192, where fp16 sums would miss by 2116x and bf16 by 13187x +PASS nonfinite cuda ck FP8 prompt tile tf_ck8_128 (slices by tf_ck8_reduce): the stated rule holds: as the 64-row tile it replaces gives them: a NaN input and a NaN weight code +PASS repeat cuda ck FP8 prompt tile tf_ck8_128 (slices by tf_ck8_reduce): five runs byte-identical (524288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck FP8 prompt tile tf_ck8_128 (slices by tf_ck8_reduce): 10 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck FP8 prompt tile tf_ck8_128 (slices by tf_ck8_reduce): exact sums at k 8192, where fp16 sums would miss by 2523x and bf16 by 19162x +PASS nonfinite cuda tree_round tf_round_bf16_argmax: the stated rule holds: a row holding NaN or an infinity picks no token and flags it (1 NaN, 2 +inf, 4 -inf); ties go to the lower token +PASS repeat cuda tree_round tf_round_bf16_argmax: five runs byte-identical (128 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_round_bf16_argmax: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_round_bf16_argmax: declares none sums +PASS nonfinite cuda tree_round tf_tree_round_match: the stated rule holds: a flagged pick or a malformed tree gives core/tree_round.oracle's invalid result, byte for byte +PASS repeat cuda tree_round tf_tree_round_match: five runs byte-identical (1344 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_tree_round_match: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_tree_round_match: declares none sums +PASS nonfinite cuda tree_round tf_round_keep_rows: the stated rule holds: a tree whose result is not ok moves nothing; kept rows move bit for bit, NaN and infinities included +PASS repeat cuda tree_round tf_round_keep_rows: five runs byte-identical (544 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_round_keep_rows: 2 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_round_keep_rows: declares none sums +PASS nonfinite cuda tree_round tf_round_gather_rows: the stated rule holds: a tree whose result is not ok moves nothing; kept rows move bit for bit, NaN and infinities included +PASS repeat cuda tree_round tf_round_gather_rows: five runs byte-identical (1310720 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_round_gather_rows: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_round_gather_rows: declares none sums +PASS nonfinite cuda tree_round tf_round_keep_head: the stated rule holds: a tree whose result is not ok moves nothing; kept rows move bit for bit, NaN and infinities included +PASS repeat cuda tree_round tf_round_keep_head: five runs byte-identical (3973120 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_round_keep_head: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_round_keep_head: declares none sums +PASS nonfinite cuda act_rows tf_swiglu_rows: 5 non-finite and 87035 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_swiglu_rows: five runs byte-identical (34816 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_swiglu_rows: 9 shapes within the float64 bound (worst 0.995 of the bound), 0 edges refused by the kernel +N/A sums cuda act_rows tf_swiglu_rows: declares none sums +PASS nonfinite cuda act_rows tf_swiglu_rows group sums (of its stored bf16 outputs): 5 non-finite and 1355 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_swiglu_rows group sums (of its stored bf16 outputs): five runs byte-identical (1088 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_swiglu_rows group sums (of its stored bf16 outputs): 9 shapes within the float64 bound (worst 0.204 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_swiglu_rows group sums (of its stored bf16 outputs): exact sums at k 1, where fp16 sums would miss by 246724x and bf16 by 2960685x +PASS nonfinite cuda act_rows tf_gate_rows: 4 non-finite and 30716 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_gate_rows: five runs byte-identical (12288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_gate_rows: 7 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +N/A sums cuda act_rows tf_gate_rows: declares none sums +PASS nonfinite cuda act_rows tf_gate_rows group sums (of its stored bf16 outputs): 4 non-finite and 476 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_gate_rows group sums (of its stored bf16 outputs): five runs byte-identical (384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_gate_rows group sums (of its stored bf16 outputs): 7 shapes within the float64 bound (worst 0.076 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_gate_rows group sums (of its stored bf16 outputs): exact sums at k 256, where fp16 sums would miss by 123362x and bf16 by 2960685x +PASS nonfinite cuda act_rows tf_group_sums_rows: 3 non-finite and 397 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_group_sums_rows: five runs byte-identical (320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_group_sums_rows: 7 shapes within the float64 bound (worst 0.034 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_group_sums_rows: exact sums at k 17408, where fp16 sums would miss by 2097152x and bf16 by 67108864x +PASS nonfinite cuda act_rows tf_dconv_rows: 9 non-finite and 40951 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_dconv_rows: five runs byte-identical (163840 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_dconv_rows: 9 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_dconv_rows: exact sums at k 16, where fp16 sums would miss by 256x and bf16 by 256x +PASS nonfinite cuda act_rows tf_dconv_rows with residual: the stated rule holds: own check, core/act_ref's bound: the conv is rounded to bf16 before the residual add, an error of the conv's size +PASS repeat cuda act_rows tf_dconv_rows with residual: five runs byte-identical (163840 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_dconv_rows with residual: 9 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_dconv_rows with residual: exact sums at k 16, where fp16 sums would miss by 205x and bf16 by 205x +PASS nonfinite cuda act tf_geglu_rows (MLX's bf16 steps): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda act tf_geglu_rows (MLX's bf16 steps): five runs byte-identical (538560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act tf_geglu_rows (MLX's bf16 steps): 12 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda act tf_geglu_rows (MLX's bf16 steps): declares none sums +PASS nonfinite cuda act tf_swiglu_rows_round (MLX's bf16 steps): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda act tf_swiglu_rows_round (MLX's bf16 steps): five runs byte-identical (538560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act tf_swiglu_rows_round (MLX's bf16 steps): 12 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda act tf_swiglu_rows_round (MLX's bf16 steps): declares none sums +FAIL shapes control: GeGLU without x^3's rounding: at 17x15360x1: row 0 out[283] = 4.4189453125e-2 outside float64's [4.39453125e-2, 4.39453125e-2] +CONTROL control: GeGLU without x^3's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without c1 x^3's rounding: at 17x15360x1: row 0 out[14] = 2.546875e0 outside float64's [2.53125e0, 2.53125e0] +CONTROL control: GeGLU without c1 x^3's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without the inner sum's rounding: at 17x15360x1: row 0 out[15] = -3.453125e0 outside float64's [-3.4375e0, -3.4375e0] +CONTROL control: GeGLU without the inner sum's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without c0 inner's rounding: at 17x15360x1: row 0 out[10] = 1.259765625e-1 outside float64's [1.279296875e-1, 1.279296875e-1] +CONTROL control: GeGLU without c0 inner's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without tanh's rounding: at 17x15360x1: row 0 out[0] = 7.8582763671875e-4 outside float64's [0e0, 0e0] +CONTROL control: GeGLU without tanh's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without 1 + tanh's rounding: at 17x15360x1: row 0 out[20] = -7.1875e-1 outside float64's [-7.1484375e-1, -7.1484375e-1] +CONTROL control: GeGLU without 1 + tanh's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without the gelu's rounding: at 17x15360x1: row 0 out[5] = -7.080078125e-2 outside float64's [-7.03125e-2, -7.03125e-2] +CONTROL control: GeGLU without the gelu's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without exp's rounding: at 17x15360x1: row 0 out[11] = -3.92578125e-1 outside float64's [-3.90625e-1, -3.90625e-1] +CONTROL control: SwiGLU steps without exp's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without 1 + e's rounding: at 17x15360x1: row 0 out[12] = -2.5390625e-1 outside float64's [-2.51953125e-1, -2.51953125e-1] +CONTROL control: SwiGLU steps without 1 + e's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without the reciprocal's rounding: at 17x15360x1: row 0 out[0] = -4.40625e0 outside float64's [-4.4375e0, -4.4375e0] +CONTROL control: SwiGLU steps without the reciprocal's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without 1 - the reciprocal's rounding: at 17x15360x1: row 0 out[10] = -1.640625e-1 outside float64's [-1.630859375e-1, -1.630859375e-1] +CONTROL control: SwiGLU steps without 1 - the reciprocal's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without silu's rounding: at 17x15360x1: row 0 out[3] = 3.984375e0 outside float64's [3.96875e0, 3.96875e0] +CONTROL control: SwiGLU steps without silu's rounding: red at shapes, as planted +PASS nonfinite cuda gdn_prep tf_gdn_prep_64 qk: 256 non-finite and 1024 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_64 qk: five runs byte-identical (512 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_64 qk: 7 shapes within the float64 bound (worst 0.687 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_64 qk: within its bound at k 4; fp16 sums would miss infx, bf16 0.2x (these stress inputs do not separate bf16 conv/square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_64 v: 5 non-finite and 1915 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_64 v: five runs byte-identical (768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_64 v: 7 shapes within the float64 bound (worst 0.995 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_64 v: within its bound at k 4; fp16 sums would miss infx, bf16 0.8x (these stress inputs do not separate bf16 tap sums beyond the rounded output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_64 gb: the stated rule holds: own check, the PR's float64 bound: g = exp(-y) carries y times y's relative error, beyond a fixed op count +PASS repeat cuda gdn_prep tf_gdn_prep_64 gb: five runs byte-identical (48 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_64 gb: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda gdn_prep tf_gdn_prep_64 gb: declares none sums +PASS nonfinite cuda gdn_prep tf_gdn_prep_128 qk: 385 non-finite and 20095 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_128 qk: five runs byte-identical (8192 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_128 qk: 10 shapes within the float64 bound (worst 0.689 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_128 qk: within its bound at k 4; fp16 sums would miss infx, bf16 0.2x (these stress inputs do not separate bf16 conv/square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_128 v: 7 non-finite and 30713 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_128 v: five runs byte-identical (12288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_128 v: 10 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_128 v: within its bound at k 4; fp16 sums would miss infx, bf16 0.8x (these stress inputs do not separate bf16 tap sums beyond the rounded output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_128 gb: the stated rule holds: own check, the PR's float64 bound: g = exp(-y) carries y times y's relative error, beyond a fixed op count +PASS repeat cuda gdn_prep tf_gdn_prep_128 gb: five runs byte-identical (384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_128 gb: 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda gdn_prep tf_gdn_prep_128 gb: declares none sums +PASS nonfinite cuda gdn_prep tf_gdn_prep_256 qk: 513 non-finite and 2047 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_256 qk: five runs byte-identical (1024 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_256 qk: 5 shapes within the float64 bound (worst 0.644 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_256 qk: within its bound at k 2; fp16 sums would miss infx, bf16 0.3x (these stress inputs do not separate bf16 conv/square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_256 v: 3 non-finite and 3837 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_256 v: five runs byte-identical (1536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_256 v: 5 shapes within the float64 bound (worst 0.995 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_256 v: within its bound at k 2; fp16 sums would miss infx, bf16 0.5x (these stress inputs do not separate bf16 tap sums beyond the rounded output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_256 gb: the stated rule holds: own check, the PR's float64 bound: g = exp(-y) carries y times y's relative error, beyond a fixed op count +PASS repeat cuda gdn_prep tf_gdn_prep_256 gb: five runs byte-identical (24 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_256 gb: 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda gdn_prep tf_gdn_prep_256 gb: declares none sums +PASS nonfinite cuda qk_rows tf_qk_rows_w (norm, rotation from a table): 135 non-finite and 35705 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows_w (norm, rotation from a table): five runs byte-identical (14336 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_w (norm, rotation from a table): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_w (norm, rotation from a table): within its bound at k 128; fp16 sums would miss 256x, bf16 1.9x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows_w (norm, no rotation): 132 non-finite and 35708 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows_w (norm, no rotation): five runs byte-identical (14336 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_w (norm, no rotation): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_w (norm, no rotation): within its bound at k 128; fp16 sums would miss 256x, bf16 0.9x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows_w (bf16 norm, rotation from inv_freq): 135 non-finite and 35705 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows_w (bf16 norm, rotation from inv_freq): five runs byte-identical (14336 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_w (bf16 norm, rotation from inv_freq): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_w (bf16 norm, rotation from inv_freq): within its bound at k 128; fp16 sums would miss 256x, bf16 1.8x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows_w (no norm, rotation from a table): 9 non-finite and 12791 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows_w (no norm, rotation from a table): five runs byte-identical (5120 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_w (no norm, rotation from a table): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_w (no norm, rotation from a table): exact sums at k 128, where fp16 sums would miss by infx and bf16 by 256x +PASS nonfinite cuda qk_rows tf_qk_rows (dim 256, norm, rotary 64 from a table): 262 non-finite and 40698 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows (dim 256, norm, rotary 64 from a table): five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows (dim 256, norm, rotary 64 from a table): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows (dim 256, norm, rotary 64 from a table): within its bound at k 256; fp16 sums would miss 256x, bf16 1.6x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows (dim 256, bf16 norm, rotary 64 from inv_freq): 262 non-finite and 40698 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows (dim 256, bf16 norm, rotary 64 from inv_freq): five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows (dim 256, bf16 norm, rotary 64 from inv_freq): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows (dim 256, bf16 norm, rotary 64 from inv_freq): within its bound at k 256; fp16 sums would miss 256x, bf16 1.5x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows (dim 64, no norm, rotary 32 from a table): 8 non-finite and 3832 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows (dim 64, no norm, rotary 32 from a table): five runs byte-identical (1536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows (dim 64, no norm, rotary 32 from a table): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows (dim 64, no norm, rotary 32 from a table): exact sums at k 64, where fp16 sums would miss by infx and bf16 by 256x +PASS nonfinite cuda tree attention tf_tree_paths: the stated rule holds: integers only: each row's depth (at most 128) and its nearest 128 ancestors root first; other slots untouched +PASS repeat cuda tree attention tf_tree_paths: five runs byte-identical (8256 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention tf_tree_paths: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree attention tf_tree_paths: declares none sums +PASS nonfinite cuda tree attention dim 128, four-warp blocks (tf_tree_shared, tf_tree_part, tf_tree_merge): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 128, four-warp blocks (tf_tree_shared, tf_tree_part, tf_tree_merge): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 128, four-warp blocks (tf_tree_shared, tf_tree_part, tf_tree_merge): 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda tree attention dim 128, four-warp blocks (tf_tree_shared, tf_tree_part, tf_tree_merge): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1445465x +PASS nonfinite cuda tree attention dim 128, one-warp blocks (tf_tree_shared_w1, tf_tree_part_w1, sliced merge): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 128, one-warp blocks (tf_tree_shared_w1, tf_tree_part_w1, sliced merge): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 128, one-warp blocks (tf_tree_shared_w1, tf_tree_part_w1, sliced merge): 9 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +PASS sums cuda tree attention dim 128, one-warp blocks (tf_tree_shared_w1, tf_tree_part_w1, sliced merge): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1839101x +PASS nonfinite cuda tree attention dim 128, one-warp blocks, 3 stages (tf_tree_shared_w1s3, tf_tree_part_w1s3): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 128, one-warp blocks, 3 stages (tf_tree_shared_w1s3, tf_tree_part_w1s3): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 128, one-warp blocks, 3 stages (tf_tree_shared_w1s3, tf_tree_part_w1s3): 9 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +PASS sums cuda tree attention dim 128, one-warp blocks, 3 stages (tf_tree_shared_w1s3, tf_tree_part_w1s3): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1954573x +PASS nonfinite cuda tree attention dim 256, four-warp blocks (tf_tree_shared_256, tf_tree_part_256, tf_tree_merge): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 256, four-warp blocks (tf_tree_shared_256, tf_tree_part_256, tf_tree_merge): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 256, four-warp blocks (tf_tree_shared_256, tf_tree_part_256, tf_tree_merge): 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda tree attention dim 256, four-warp blocks (tf_tree_shared_256, tf_tree_part_256, tf_tree_merge): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1773922x +PASS nonfinite cuda tree attention dim 256, one-warp blocks (tf_tree_shared_256_w1, tf_tree_part_256_w1): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 256, one-warp blocks (tf_tree_shared_256_w1, tf_tree_part_256_w1): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 256, one-warp blocks (tf_tree_shared_256_w1, tf_tree_part_256_w1): 9 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +PASS sums cuda tree attention dim 256, one-warp blocks (tf_tree_shared_256_w1, tf_tree_part_256_w1): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 2003330x +PASS nonfinite cuda tree attention dim 256, one-warp blocks, 3 stages (tf_tree_shared_256_w1s3): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 256, one-warp blocks, 3 stages (tf_tree_shared_256_w1s3): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 256, one-warp blocks, 3 stages (tf_tree_shared_256_w1s3): 9 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +PASS sums cuda tree attention dim 256, one-warp blocks, 3 stages (tf_tree_shared_256_w1s3): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 2102379x +PASS nonfinite cuda block attention tf_block_attention (DFlash2 blocks of 16): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda block attention tf_block_attention (DFlash2 blocks of 16): five runs byte-identical (131072 bytes), inputs and guards untouched, no overlap +PASS shapes cuda block attention tf_block_attention (DFlash2 blocks of 16): 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda block attention tf_block_attention (DFlash2 blocks of 16): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1602240x +PASS nonfinite cuda block attention tf_block_attention, causal blocks of 7: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda block attention tf_block_attention, causal blocks of 7: five runs byte-identical (57344 bytes), inputs and guards untouched, no overlap +PASS shapes cuda block attention tf_block_attention, causal blocks of 7: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda block attention tf_block_attention, causal blocks of 7: exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1704486x +PASS nonfinite cuda window decode tf_window_decode_tc, 48/4 heads, window 513, ring 640: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window decode tf_window_decode_tc, 48/4 heads, window 513, ring 640: five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window decode tf_window_decode_tc, 48/4 heads, window 513, ring 640: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window decode tf_window_decode_tc, 48/4 heads, window 513, ring 640: exact sums at k 3000, where fp16 sums would miss by infx and bf16 by 204941x +PASS nonfinite cuda window decode tf_window_decode_tc, 16/1 heads, no window, ring 300: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window decode tf_window_decode_tc, 16/1 heads, no window, ring 300: five runs byte-identical (65536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window decode tf_window_decode_tc, 16/1 heads, no window, ring 300: 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window decode tf_window_decode_tc, 16/1 heads, no window, ring 300: exact sums at k 300, where fp16 sums would miss by infx and bf16 by 90367x +PASS nonfinite cuda window decode tf_window_part + tf_window_merge, 8/2 heads of 64, window 100, ring 128: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window decode tf_window_part + tf_window_merge, 8/2 heads of 64, window 100, ring 128: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window decode tf_window_part + tf_window_merge, 8/2 heads of 64, window 100, ring 128: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window decode tf_window_part + tf_window_merge, 8/2 heads of 64, window 100, ring 128: exact sums at k 3000, where fp16 sums would miss by infx and bf16 by 65279x +PASS nonfinite cuda window prompt wattn_kernel<128, 8, 4, 8>, 48/4 heads, window 513, ring 2624: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window prompt wattn_kernel<128, 8, 4, 8>, 48/4 heads, window 513, ring 2624: five runs byte-identical (786432 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window prompt wattn_kernel<128, 8, 4, 8>, 48/4 heads, window 513, ring 2624: 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window prompt wattn_kernel<128, 8, 4, 8>, 48/4 heads, window 513, ring 2624: exact sums at k 2600, where fp16 sums would miss by infx and bf16 by 902610x +PASS nonfinite cuda window prompt wattn_kernel<128, 8, 8, 8>, 16/2 heads, no window, a row a position: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window prompt wattn_kernel<128, 8, 8, 8>, 16/2 heads, no window, a row a position: five runs byte-identical (262144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window prompt wattn_kernel<128, 8, 8, 8>, 16/2 heads, no window, a row a position: 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window prompt wattn_kernel<128, 8, 8, 8>, 16/2 heads, no window, a row a position: exact sums at k 100, where fp16 sums would miss by infx and bf16 by 65791x +PASS nonfinite cuda moe top-k tf_moe_topk_sigmoid_warp (Kolibri: 384 experts, top 6 + shared): the stated rule holds: picks by fp32 logit + bias, larger first, ties to the lower id, a NaN never picked; weights sigmoid(logit) +PASS repeat cuda moe top-k tf_moe_topk_sigmoid_warp (Kolibri: 384 experts, top 6 + shared): five runs byte-identical (896 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe top-k tf_moe_topk_sigmoid_warp (Kolibri: 384 experts, top 6 + shared): 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda moe top-k tf_moe_topk_sigmoid_warp (Kolibri: 384 experts, top 6 + shared): declares none sums +PASS nonfinite cuda moe top-k tf_moe_topk_sigmoid_warp (256 experts, top 8 normalized x2.5): the stated rule holds: picks by fp32 logit + bias, larger first, ties to the lower id, a NaN never picked; weights sigmoid(logit) +PASS repeat cuda moe top-k tf_moe_topk_sigmoid_warp (256 experts, top 8 normalized x2.5): five runs byte-identical (1024 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe top-k tf_moe_topk_sigmoid_warp (256 experts, top 8 normalized x2.5): 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda moe top-k tf_moe_topk_sigmoid_warp (256 experts, top 8 normalized x2.5): declares none sums +PASS nonfinite cuda moe top-k tf_moe_topk_sigmoid (block form, 513 to 4,096 experts): the stated rule holds: picks by fp32 logit + bias, larger first, ties to the lower id, a NaN never picked; weights sigmoid(logit) +PASS repeat cuda moe top-k tf_moe_topk_sigmoid (block form, 513 to 4,096 experts): five runs byte-identical (8320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe top-k tf_moe_topk_sigmoid (block form, 513 to 4,096 experts): 3 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda moe top-k tf_moe_topk_sigmoid (block form, 513 to 4,096 experts): declares none sums +PASS nonfinite cuda moe combine tf_moe_combine (fp32 rows into bf16): 4 non-finite and 20476 finite outputs match the float64 classes +PASS repeat cuda moe combine tf_moe_combine (fp32 rows into bf16): five runs byte-identical (81920 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe combine tf_moe_combine (fp32 rows into bf16): 11 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda moe combine tf_moe_combine (fp32 rows into bf16): exact sums at k 9, where fp16 sums would miss by infx and bf16 by 1711x +PASS nonfinite cuda moe combine tf_moe_combine (bf16 rows into bf16): 4 non-finite and 20476 finite outputs match the float64 classes +PASS repeat cuda moe combine tf_moe_combine (bf16 rows into bf16): five runs byte-identical (81920 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe combine tf_moe_combine (bf16 rows into bf16): 11 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda moe combine tf_moe_combine (bf16 rows into bf16): exact sums at k 9, where fp16 sums would miss by infx and bf16 by 2033x +PASS nonfinite cuda moe combine tf_moe_combine (fp32 rows into fp32): 4 non-finite and 20476 finite outputs match the float64 classes +PASS repeat cuda moe combine tf_moe_combine (fp32 rows into fp32): five runs byte-identical (163840 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe combine tf_moe_combine (fp32 rows into fp32): 11 shapes within the float64 bound (worst 0.590 of the bound), 0 edges refused by the kernel +PASS sums cuda moe combine tf_moe_combine (fp32 rows into fp32): exact sums at k 9, where fp16 sums would miss by infx and bf16 by 294170643x +PASS nonfinite cuda moe combine tf_moe_combine (fp16 rows into fp16, the generic loop): 4 non-finite and 20476 finite outputs match the float64 classes +PASS repeat cuda moe combine tf_moe_combine (fp16 rows into fp16, the generic loop): five runs byte-identical (81920 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe combine tf_moe_combine (fp16 rows into fp16, the generic loop): 11 shapes within the float64 bound (worst 0.997 of the bound), 0 edges refused by the kernel +PASS sums cuda moe combine tf_moe_combine (fp16 rows into fp16, the generic loop): exact sums at k 9, where fp16 sums would miss by infx and bf16 by 15838x +PASS nonfinite cuda dense rows tf_dense_rows_1: 1536 non-finite and 384 finite outputs match the float64 classes +PASS repeat cuda dense rows tf_dense_rows_1: five runs byte-identical (1536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda dense rows tf_dense_rows_1: 12 shapes within the float64 bound (worst 0.117 of the bound), 0 edges refused by the kernel +PASS sums cuda dense rows tf_dense_rows_1: exact sums at k 7168, where fp16 sums would miss by 159112952x and bf16 by 537412113x +PASS nonfinite cuda dense rows tf_dense_rows_4: 1536 non-finite and 384 finite outputs match the float64 classes +PASS repeat cuda dense rows tf_dense_rows_4: five runs byte-identical (6144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda dense rows tf_dense_rows_4: 15 shapes within the float64 bound (worst 0.157 of the bound), 0 edges refused by the kernel +PASS sums cuda dense rows tf_dense_rows_4: exact sums at k 7168, where fp16 sums would miss by 194615706x and bf16 by 619514235x +PASS nonfinite cuda dense rows tf_dense_rows_16: 1536 non-finite and 1536 finite outputs match the float64 classes +PASS repeat cuda dense rows tf_dense_rows_16: five runs byte-identical (24576 bytes), inputs and guards untouched, no overlap +PASS shapes cuda dense rows tf_dense_rows_16: 15 shapes within the float64 bound (worst 0.156 of the bound), 0 edges refused by the kernel +PASS sums cuda dense rows tf_dense_rows_16: exact sums at k 7168, where fp16 sums would miss by 141315781x and bf16 by 764125929x +PASS nonfinite cuda dense rows tf_dense_rows_32: 1536 non-finite and 1536 finite outputs match the float64 classes +PASS repeat cuda dense rows tf_dense_rows_32: five runs byte-identical (49152 bytes), inputs and guards untouched, no overlap +PASS shapes cuda dense rows tf_dense_rows_32: 17 shapes within the float64 bound (worst 0.213 of the bound), 0 edges refused by the kernel +PASS sums cuda dense rows tf_dense_rows_32: exact sums at k 7168, where fp16 sums would miss by 130722475x and bf16 by 842216243x +PASS nonfinite cuda embed_rows tf_embed_rows_affine4 (MLX 4-bit rows): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine4 (MLX 4-bit rows): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine4 (MLX 4-bit rows): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine4 (MLX 4-bit rows): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (2-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (2-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (2-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (2-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (3-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (3-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (3-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (3-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (4-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (4-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (4-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (4-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (5-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (5-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (5-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (5-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (6-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (6-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (6-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (6-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (8-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (8-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (8-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (8-bit, bf16 scale): declares none sums +FAIL shapes control: scaled rows without the dequantized value's bf16 rounding: at 17x3840x1: row 0 out[4] = -2.0375e1 where float64 at the rounding points gives -2.05e1 +CONTROL control: scaled rows without the dequantized value's bf16 rounding: red at shapes, as planted +FAIL shapes control: scaled rows with the scale folded into the group's scale and bias: at 17x3840x1: row 0 out[2] = 1.37e2 where float64 at the rounding points gives 1.38e2 +CONTROL control: scaled rows with the scale folded into the group's scale and bias: red at shapes, as planted +PASS nonfinite cuda qk_rows tf_qk_rows_round (MLX's rows, dim 512, k's columns as v by a unit norm, 64 of 256 pairs rotated): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda qk_rows tf_qk_rows_round (MLX's rows, dim 512, k's columns as v by a unit norm, 64 of 256 pairs rotated): five runs byte-identical (40960 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_round (MLX's rows, dim 512, k's columns as v by a unit norm, 64 of 256 pairs rotated): 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_round (MLX's rows, dim 512, k's columns as v by a unit norm, 64 of 256 pairs rotated): within its bound at k 512; fp16 sums would miss 51x, bf16 0.7x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda qk_rows tf_qk_rows_round (MLX's rows, dim 256, unit v norm, rotation from inv_freq): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda qk_rows tf_qk_rows_round (MLX's rows, dim 256, unit v norm, rotation from inv_freq): five runs byte-identical (32768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_round (MLX's rows, dim 256, unit v norm, rotation from inv_freq): 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_round (MLX's rows, dim 256, unit v norm, rotation from inv_freq): within its bound at k 256; fp16 sums would miss 51x, bf16 0.7x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda qk_rows tf_qk_rows_once (dim 512, norm, rotation from a table, v as is): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda qk_rows tf_qk_rows_once (dim 512, norm, rotation from a table, v as is): five runs byte-identical (18432 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_once (dim 512, norm, rotation from a table, v as is): 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_once (dim 512, norm, rotation from a table, v as is): within its bound at k 512; fp16 sums would miss 51x, bf16 0.4x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +FAIL shapes control: MLX's qk rows without the normalized row's rounding: at 1x32x512: row 0 head 0 out[5] = 1.572265625e-1 outside float64's [1.5625e-1, 1.5625e-1] +CONTROL control: MLX's qk rows without the normalized row's rounding: red at shapes, as planted +FAIL shapes control: MLX's qk rows without the weighted row's rounding before RoPE: at 300x32x512: row 1 head 0 out[0] = -2.353515625e-1 outside float64's [-2.36328125e-1, -2.36328125e-1] +CONTROL control: MLX's qk rows without the weighted row's rounding before RoPE: red at shapes, as planted +FAIL shapes control: MLX's qk rows without v's unit norm: at 1x32x512: row 0 head 36 out[0] = -1.1875e0 outside float64's [-6.2109375e-1, -6.2109375e-1] +CONTROL control: MLX's qk rows without v's unit norm: red at shapes, as planted +PASS nonfinite cuda flashnext factored prompt matmul 64x128: 55808 non-finite and 55808 finite outputs match the float64 classes +PASS repeat cuda flashnext factored prompt matmul 64x128: five runs byte-identical (1785856 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext factored prompt matmul 64x128: 14 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda flashnext factored prompt matmul 64x128: exact sums at k 2560, where fp16 sums would miss by 6357x and bf16 by 37192x +PASS nonfinite cuda flashnext factored prompt matmul 64x128, K slices folded: 10240 non-finite and 10240 finite outputs match the float64 classes +PASS repeat cuda flashnext factored prompt matmul 64x128, K slices folded: five runs byte-identical (655360 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext factored prompt matmul 64x128, K slices folded: 12 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 3 edges refused by the kernel +PASS sums cuda flashnext factored prompt matmul 64x128, K slices folded: exact sums at k 6144, where fp16 sums would miss by 189258762x and bf16 by 1437248114x +PASS nonfinite cuda route_rows tf_route_logits: 2052 non-finite and 2052 finite outputs match the float64 classes +PASS repeat cuda route_rows tf_route_logits: five runs byte-identical (65664 bytes), inputs and guards untouched, no overlap +PASS shapes cuda route_rows tf_route_logits: 14 shapes within the float64 bound (worst 0.086 of the bound), 0 edges refused by the kernel +PASS sums cuda route_rows tf_route_logits: exact sums at k 2560, where fp16 sums would miss by 30905398x and bf16 by 376928119x +PASS nonfinite cuda route_rows tf_route_logits_few: 2052 non-finite and 2052 finite outputs match the float64 classes +PASS repeat cuda route_rows tf_route_logits_few: five runs byte-identical (32832 bytes), inputs and guards untouched, no overlap +PASS shapes cuda route_rows tf_route_logits_few: 11 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda route_rows tf_route_logits_few: exact sums at k 2560, where fp16 sums would miss by 34225521x and bf16 by 366702007x +PASS nonfinite cuda route_rows tf_route_logits_wide: 2052 non-finite and 2052 finite outputs match the float64 classes +PASS repeat cuda route_rows tf_route_logits_wide: five runs byte-identical (131328 bytes), inputs and guards untouched, no overlap +PASS shapes cuda route_rows tf_route_logits_wide: 14 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda route_rows tf_route_logits_wide: exact sums at k 2560, where fp16 sums would miss by 27682406x and bf16 by 331897099x +PASS nonfinite cuda flashnext factored experts down 64 pairs, fp32 out: 10240 non-finite and 10240 finite outputs match the float64 classes +PASS repeat cuda flashnext factored experts down 64 pairs, fp32 out: five runs byte-identical (655360 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext factored experts down 64 pairs, fp32 out: 13 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda flashnext factored experts down 64 pairs, fp32 out: exact sums at k 640, where fp16 sums would miss by 15711995x and bf16 by 5592405x +PASS nonfinite cuda flashnext factored experts gate and up 64 pairs, SwiGLU: 2560 non-finite and 2560 finite outputs match the float64 classes +PASS repeat cuda flashnext factored experts gate and up 64 pairs, SwiGLU: five runs byte-identical (81920 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext factored experts gate and up 64 pairs, SwiGLU: 13 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda flashnext factored experts gate and up 64 pairs, SwiGLU: exact sums at k 2560, where fp16 sums would miss by 428203617113232950000000x and bf16 by 128x +PASS nonfinite cuda flashnext DeltaNet prompt chain fn_gdn_chain_tpr: the stated rule holds: ys and the state bit-equal to fn_gdn_rows_to's, non-finite inputs included +PASS repeat cuda flashnext DeltaNet prompt chain fn_gdn_chain_tpr: five runs byte-identical (4718592 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext DeltaNet prompt chain fn_gdn_chain_tpr: 6 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda flashnext DeltaNet prompt chain fn_gdn_chain_tpr: declares none sums +PASS nonfinite cuda flashnext indexer scores qsa_tile::scores<8, 4>: the stated rule holds: every score byte equal to fn_qsa_scores', non-finite inputs included +PASS repeat cuda flashnext indexer scores qsa_tile::scores<8, 4>: five runs byte-identical (153600 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext indexer scores qsa_tile::scores<8, 4>: 8 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 4 edges refused by the kernel +N/A sums cuda flashnext indexer scores qsa_tile::scores<8, 4>: declares none sums +PASS nonfinite cuda flashnext hyper-connection read-out on prompt tiles (hcf down, reduce, hcf up, rinv, mix_h): the stated rule holds: act, xs, inj, mixed and xsm bit-equal to #952's chain, non-finite inputs included +PASS repeat cuda flashnext hyper-connection read-out on prompt tiles (hcf down, reduce, hcf up, rinv, mix_h): five runs byte-identical (392192 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext hyper-connection read-out on prompt tiles (hcf down, reduce, hcf up, rinv, mix_h): 5 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda flashnext hyper-connection read-out on prompt tiles (hcf down, reduce, hcf up, rinv, mix_h): declares none sums +RESULT gdn-back finite differences (6 rows, 2 key / 6 value heads): 120 coordinates of P, worst |fd - analytic| / max(|analytic|, 1e-3 max) 7.91e-7 +RESULT gdn-back finite differences (9 rows, 1 key / 3 value heads): 120 coordinates of P, worst |fd - analytic| / max(|analytic|, 1e-3 max) 1.43e-6 +--- gdn-back: 64 rows, after a prefix (conv history and state) +RESULT gdn-back time 64 rows: layer backward 3.653 ms (forward for training, chain and gates 1.064 ms, the rest the two projections' dequant and GEMM); scratch 167.1 MiB (checkpoints 48.0, replay 12.0, dequantized projection 80.5) +RESULT gdn-back steps 64 rows (ms): front 0.015, scan_fwd 0.264, norm_back 0.038, scan_back 0.688, qk_back 0.054, gates_back 0.006, conv_back 0.030 +RESULT gdn-back forward act 3 of 655360 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 3072 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 50 of 393216 bf16 values differ from float64's rounding, at most 2 ulp +RESULT gdn-back forward q worst column rel 1.98e-7 (col 1743), max abs 5.67e-9, ref scale 5.04e-2 +RESULT gdn-back forward k worst column rel 1.71e-7 (col 1920), max abs 6.66e-8, ref scale 6.00e-1 +RESULT gdn-back forward g worst column rel 9.24e-8 (col 27), max abs 8.79e-8, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 1.35e-3 (col 2595), max abs 7.63e-6, ref scale 6.30e-2 +RESULT gdn-back dout (o_proj back) worst column rel 5.54e-3 (col 13), max abs 8.45e-5, ref scale 3.49e-2 +RESULT gdn-back dout, bf16 operands worst column rel 4.93e-6 (col 1875), max abs 9.05e-8, ref scale 3.49e-2 +RESULT gdn-back dy (norm back) worst column rel 2.93e-7 (col 5351), max abs 4.79e-6, ref scale 3.17e1 +RESULT gdn-back dz worst column rel 8.23e-7 (col 5349), max abs 4.86e-9, ref scale 2.40e-2 +RESULT gdn-back dq (chain) worst column rel 6.10e-7 (col 1042), max abs 2.11e-6, ref scale 1.43e1 +RESULT gdn-back dk (chain) worst column rel 6.35e-7 (col 112), max abs 2.07e-7, ref scale 1.39e0 +RESULT gdn-back dv (chain) worst column rel 1.02e-6 (col 4340), max abs 2.93e-8, ref scale 1.55e-1 +RESULT gdn-back dg (chain) worst column rel 1.06e-6 (col 3), max abs 1.58e-7, ref scale 5.03e-1 +RESULT gdn-back dbeta (chain) worst column rel 8.08e-7 (col 44), max abs 1.72e-7, ref scale 5.41e-1 +RESULT gdn-back dact q (L2 back) worst column rel 6.51e-7 (col 986), max abs 7.69e-8, ref scale 3.46e-1 +RESULT gdn-back dact k (L2 back) worst column rel 7.57e-7 (col 515), max abs 7.45e-8, ref scale 3.63e-1 +RESULT gdn-back dproj q (conv back) worst column rel 1.06e-6 (col 542), max abs 4.73e-8, ref scale 1.23e-1 +RESULT gdn-back dproj k (conv back) worst column rel 7.68e-7 (col 515), max abs 2.84e-8, ref scale 1.51e-1 +RESULT gdn-back dproj v (conv back) worst column rel 1.03e-6 (col 4340), max abs 1.26e-8, ref scale 6.90e-2 +RESULT gdn-back dproj b (beta back) worst column rel 4.81e-7 (col 16), max abs 1.70e-8, ref scale 7.14e-2 +RESULT gdn-back dproj a (g back) worst column rel 1.13e-6 (col 3), max abs 2.09e-8, ref scale 6.49e-2 +RESULT gdn-back dx (in_proj back) worst column rel 4.95e-3 (col 1043), max abs 1.47e-4, ref scale 6.20e-2 +RESULT gdn-back dx, bf16 operands worst column rel 4.37e-5 (col 1415), max abs 1.43e-6, ref scale 6.20e-2 +--- gdn-back: 256 rows, from the sequence start +RESULT gdn-back time 256 rows: layer backward 6.864 ms (forward for training, chain and gates 4.080 ms, the rest the two projections' dequant and GEMM); scratch 391.1 MiB (checkpoints 192.0, replay 12.0, dequantized projection 80.5) +RESULT gdn-back steps 256 rows (ms): front 0.061, scan_fwd 1.054, norm_back 0.135, scan_back 2.470, qk_back 0.266, gates_back 0.006, conv_back 0.096 +RESULT gdn-back forward act 11 of 2621440 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 12288 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 177 of 1572864 bf16 values differ from float64's rounding, at most 10 ulp +RESULT gdn-back forward q worst column rel 2.00e-7 (col 593), max abs 7.63e-9, ref scale 5.58e-2 +RESULT gdn-back forward k worst column rel 1.86e-7 (col 62), max abs 7.25e-8, ref scale 6.21e-1 +RESULT gdn-back forward g worst column rel 1.08e-7 (col 30), max abs 9.63e-8, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 2.12e-3 (col 2828), max abs 3.05e-5, ref scale 6.18e-2 +RESULT gdn-back dout (o_proj back) worst column rel 6.33e-3 (col 4447), max abs 7.60e-5, ref scale 3.23e-2 +RESULT gdn-back dout, bf16 operands worst column rel 5.02e-6 (col 115), max abs 9.78e-8, ref scale 3.23e-2 +RESULT gdn-back dy (norm back) worst column rel 2.77e-7 (col 3309), max abs 5.91e-6, ref scale 3.62e1 +RESULT gdn-back dz worst column rel 8.01e-7 (col 5695), max abs 6.20e-9, ref scale 3.33e-2 +RESULT gdn-back dq (chain) worst column rel 5.15e-7 (col 1818), max abs 2.84e-6, ref scale 1.44e1 +RESULT gdn-back dk (chain) worst column rel 5.19e-7 (col 1198), max abs 2.23e-7, ref scale 1.39e0 +RESULT gdn-back dv (chain) worst column rel 7.24e-7 (col 4989), max abs 3.33e-8, ref scale 1.46e-1 +RESULT gdn-back dg (chain) worst column rel 6.25e-7 (col 35), max abs 1.22e-7, ref scale 6.07e-1 +RESULT gdn-back dbeta (chain) worst column rel 4.22e-7 (col 40), max abs 1.68e-7, ref scale 6.93e-1 +RESULT gdn-back dact q (L2 back) worst column rel 6.70e-7 (col 1682), max abs 9.95e-8, ref scale 3.88e-1 +RESULT gdn-back dact k (L2 back) worst column rel 5.90e-7 (col 1391), max abs 7.69e-8, ref scale 4.72e-1 +RESULT gdn-back dproj q (conv back) worst column rel 7.64e-7 (col 1682), max abs 4.43e-8, ref scale 1.56e-1 +RESULT gdn-back dproj k (conv back) worst column rel 7.01e-7 (col 977), max abs 2.84e-8, ref scale 1.58e-1 +RESULT gdn-back dproj v (conv back) worst column rel 6.77e-7 (col 1722), max abs 2.63e-8, ref scale 7.86e-2 +RESULT gdn-back dproj b (beta back) worst column rel 5.11e-7 (col 2), max abs 1.71e-8, ref scale 8.65e-2 +RESULT gdn-back dproj a (g back) worst column rel 4.91e-7 (col 12), max abs 2.09e-8, ref scale 6.99e-2 +RESULT gdn-back dx (in_proj back) worst column rel 5.62e-3 (col 2311), max abs 1.42e-4, ref scale 6.96e-2 +RESULT gdn-back dx, bf16 operands worst column rel 4.22e-5 (col 2107), max abs 2.18e-6, ref scale 6.97e-2 +--- gdn-back: 37 rows, after a prefix (conv history and state) +RESULT gdn-back time 37 rows: layer backward 3.166 ms (forward for training, chain and gates 0.631 ms, the rest the two projections' dequant and GEMM); scratch 137.9 MiB (checkpoints 30.0, replay 12.0, dequantized projection 80.5) +RESULT gdn-back steps 37 rows (ms): front 0.011, scan_fwd 0.164, norm_back 0.026, scan_back 0.416, qk_back 0.021, gates_back 0.004, conv_back 0.018 +RESULT gdn-back forward act 1 of 378880 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 1776 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 25 of 227328 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward q worst column rel 1.76e-7 (col 1327), max abs 6.19e-9, ref scale 5.26e-2 +RESULT gdn-back forward k worst column rel 1.51e-7 (col 362), max abs 6.43e-8, ref scale 5.82e-1 +RESULT gdn-back forward g worst column rel 9.50e-8 (col 10), max abs 9.33e-8, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 1.33e-3 (col 5959), max abs 1.53e-5, ref scale 5.32e-2 +RESULT gdn-back dout (o_proj back) worst column rel 1.32e-2 (col 5971), max abs 7.81e-5, ref scale 3.33e-2 +RESULT gdn-back dout, bf16 operands worst column rel 6.45e-6 (col 3966), max abs 8.90e-8, ref scale 3.33e-2 +RESULT gdn-back dy (norm back) worst column rel 2.67e-7 (col 5113), max abs 4.87e-6, ref scale 2.70e1 +RESULT gdn-back dz worst column rel 9.25e-7 (col 1425), max abs 4.35e-9, ref scale 2.46e-2 +RESULT gdn-back dq (chain) worst column rel 7.02e-7 (col 443), max abs 3.15e-6, ref scale 3.46e1 +RESULT gdn-back dk (chain) worst column rel 8.63e-7 (col 1350), max abs 2.49e-7, ref scale 1.29e0 +RESULT gdn-back dv (chain) worst column rel 9.06e-7 (col 1584), max abs 2.91e-8, ref scale 1.31e-1 +RESULT gdn-back dg (chain) worst column rel 5.21e-7 (col 45), max abs 2.70e-7, ref scale 1.11e0 +RESULT gdn-back dbeta (chain) worst column rel 6.01e-7 (col 0), max abs 1.20e-7, ref scale 4.72e-1 +RESULT gdn-back dact q (L2 back) worst column rel 6.79e-7 (col 723), max abs 1.34e-7, ref scale 8.70e-1 +RESULT gdn-back dact k (L2 back) worst column rel 1.24e-6 (col 1350), max abs 6.20e-8, ref scale 3.34e-1 +RESULT gdn-back dproj q (conv back) worst column rel 8.61e-7 (col 706), max abs 3.13e-8, ref scale 1.08e-1 +RESULT gdn-back dproj k (conv back) worst column rel 1.32e-6 (col 1210), max abs 1.89e-8, ref scale 1.07e-1 +RESULT gdn-back dproj v (conv back) worst column rel 1.04e-6 (col 5917), max abs 1.62e-8, ref scale 6.13e-2 +RESULT gdn-back dproj b (beta back) worst column rel 5.73e-7 (col 17), max abs 1.45e-8, ref scale 5.97e-2 +RESULT gdn-back dproj a (g back) worst column rel 5.49e-7 (col 46), max abs 3.68e-8, ref scale 7.86e-2 +RESULT gdn-back dx (in_proj back) worst column rel 1.53e-2 (col 2267), max abs 1.44e-4, ref scale 6.01e-2 +RESULT gdn-back dx, bf16 operands worst column rel 4.69e-5 (col 1115), max abs 1.66e-6, ref scale 6.01e-2 +PASS gdn-back: forward for training and every input gradient within tolerance of float64 +RESULT fd head cross-entropy dlogits: float64 reference against central differences at 4 coordinates, worst relative 7.96e-7 +RESULT head weights: 378.9 MiB (vocab 248320) +RESULT head 64 rows (loss, target probability): worst column 6.77e-8 (column 0 of 2), relative L2 3.32e-8, largest |ref| 1.16e1 +RESULT head 64 rows dlogits (bf16): largest difference 2.54e-3 of the largest |ref| 5.78e-3 +RESULT head 64 rows dmixed: worst column 6.55e-3 (column 435 of 2560), relative L2 2.56e-3, largest |ref| 3.10e-4 +RESULT head 64 rows: 4.960 ms softmax and head, 3.8 MiB of gradients and scratch (+ 30.3 MiB bf16 logits) +RESULT head 256 rows (loss, target probability): worst column 7.24e-8 (column 0 of 2), relative L2 3.18e-8, largest |ref| 1.16e1 +RESULT head 256 rows dlogits (bf16): largest difference 2.57e-3 of the largest |ref| 5.85e-3 +RESULT head 256 rows dmixed: worst column 6.03e-3 (column 1172 of 2560), relative L2 2.41e-3, largest |ref| 3.15e-4 +RESULT head 256 rows: 16.303 ms softmax and head, 7.5 MiB of gradients and scratch (+ 121.3 MiB bf16 logits) +PASS q4-back: 7 checks within tolerance (14.5 s with the float64 references) +RESULT fd moe dx: float64 reference against central differences at 8 coordinates, worst relative 1.18e-9 +RESULT moe weights: 5513.2 MiB (512 routed experts as MLX stacks and packed, the shared expert, the router, a pocket of 0) +RESULT moe 64 rows dx: worst column 2.74e-6 (column 437 of 2560), relative L2 1.15e-6, largest |ref| 7.33e0 +RESULT moe 64 rows slot scores a = dL/dw: worst column 1.74e-6 (column 3 of 11), relative L2 1.13e-6, largest |ref| 9.45e1 +RESULT moe 64 rows router logits' gradients: worst column 1.94e-6 (column 3 of 11), relative L2 1.21e-6, largest |ref| 1.96e1 +RESULT moe 64 rows: 17.939 ms on the GPU, 16.2 MiB of gradients and scratch +RESULT moe 64 rows packed dx: worst column 2.74e-6 (column 437 of 2560), relative L2 1.15e-6, largest |ref| 7.33e0 +RESULT moe 64 rows packed: 14.464 ms, dx equals the MLX stacks' bits +RESULT moe 256 rows dx: worst column 2.30e-6 (column 588 of 2560), relative L2 1.15e-6, largest |ref| 8.93e0 +RESULT moe 256 rows slot scores a = dL/dw: worst column 1.64e-6 (column 1 of 11), relative L2 1.18e-6, largest |ref| 1.47e2 +RESULT moe 256 rows router logits' gradients: worst column 1.73e-6 (column 9 of 11), relative L2 1.18e-6, largest |ref| 2.83e1 +RESULT moe 256 rows: 27.860 ms on the GPU, 64.9 MiB of gradients and scratch +RESULT moe 256 rows packed dx: worst column 2.30e-6 (column 588 of 2560), relative L2 1.15e-6, largest |ref| 8.93e0 +RESULT moe 256 rows packed: 21.455 ms, dx equals the MLX stacks' bits +PASS moe-back: 11 checks within tolerance (13.8 s with the float64 references) +RESULT fd attn dx: float64 reference against central differences at 6 coordinates, worst relative 7.62e-9 +RESULT attn weights: 29.7 MiB +RESULT attn 64 rows prepared heads: 6 of 458752 values differ from the host's bf16, the largest by 6.10e-3 of max(1, |value|) (row 40, column 4584) +RESULT attn 64 rows dpq (q and gate): worst column 1.29e-2 (column 12072 of 12288), relative L2 2.12e-3, largest |ref| 4.63e-1 +RESULT attn 64 rows dpk: worst column 7.32e-3 (column 399 of 512), relative L2 2.10e-3, largest |ref| 9.64e-1 +RESULT attn 64 rows dv: worst column 8.73e-3 (column 107 of 512), relative L2 2.09e-3, largest |ref| 4.16e0 +RESULT attn 64 rows dx: worst column 1.26e-2 (column 486 of 2560), relative L2 3.09e-3, largest |ref| 2.62e0 +RESULT attn 64 rows: 1.302 ms on the GPU, 10.1 MiB of gradients and scratch +RESULT attn 256 rows prepared heads: 26 of 1835008 values differ from the host's bf16, the largest by 6.25e-3 of max(1, |value|) (row 111, column 1607) +RESULT attn 256 rows dpq (q and gate): worst column 1.29e-2 (column 12072 of 12288), relative L2 2.12e-3, largest |ref| 4.63e-1 +RESULT attn 256 rows dpk: worst column 6.90e-3 (column 366 of 512), relative L2 2.11e-3, largest |ref| 1.02e0 +RESULT attn 256 rows dv: worst column 8.84e-3 (column 107 of 512), relative L2 2.06e-3, largest |ref| 4.29e0 +RESULT attn 256 rows dx: worst column 1.03e-2 (column 734 of 2560), relative L2 3.07e-3, largest |ref| 2.47e0 +RESULT attn 256 rows: 3.656 ms on the GPU, 49.5 MiB of gradients and scratch +PASS attn-back: 11 checks within tolerance (1.6 s with the float64 references) +RESULT fd hc block dh: float64 reference against central differences at 10 coordinates, worst relative 6.35e-8 +RESULT fd hc block dbranch: float64 reference against central differences at 6 coordinates, worst relative 2.55e-10 +RESULT fd hc final mixer dh: float64 reference against central differences at 10 coordinates, worst relative 9.80e-9 +RESULT hc weights: 4.0 MiB a read-out +RESULT hc block 64 rows dbranch: worst column 1.48e-7 (column 1547 of 2560), relative L2 4.02e-8, largest |ref| 5.27e0 +RESULT hc block 64 rows dinject: worst column 9.96e-8 (column 3 of 4), relative L2 8.78e-8, largest |ref| 5.45e1 +RESULT hc block 64 rows dh: worst column 5.82e-3 (column 429 of 10240), relative L2 1.12e-3, largest |ref| 4.82e0 +RESULT hc block 64 rows dh less the residual: worst column 7.39e-3 (column 9779 of 10240), relative L2 2.10e-3, largest |ref| 4.18e0 +RESULT hc block 64 rows: 0.670 ms on the GPU, 8.9 MiB of gradients and scratch +RESULT hc final mixer 64 rows dh: worst column 4.65e-3 (column 8916 of 10240), relative L2 1.68e-3, largest |ref| 2.02e0 +RESULT hc final mixer 64 rows: 0.747 ms on the GPU, 8.9 MiB of gradients and scratch +RESULT hc block 256 rows dbranch: worst column 1.21e-7 (column 2302 of 2560), relative L2 4.03e-8, largest |ref| 5.90e0 +RESULT hc block 256 rows dinject: worst column 1.30e-7 (column 3 of 4), relative L2 9.54e-8, largest |ref| 5.45e1 +RESULT hc block 256 rows dh: worst column 5.06e-3 (column 786 of 10240), relative L2 1.11e-3, largest |ref| 4.82e0 +RESULT hc block 256 rows dh less the residual: worst column 6.49e-3 (column 7869 of 10240), relative L2 2.06e-3, largest |ref| 4.19e0 +RESULT hc block 256 rows: 1.290 ms on the GPU, 35.6 MiB of gradients and scratch +RESULT hc final mixer 256 rows dh: worst column 3.82e-3 (column 2455 of 10240), relative L2 1.68e-3, largest |ref| 2.02e0 +RESULT hc final mixer 256 rows: 1.064 ms on the GPU, 35.6 MiB of gradients and scratch +PASS fn-hc-back: 13 checks within tolerance (0.7 s with the float64 references) +RESULT fd moe dx: float64 reference against central differences at 8 coordinates, worst relative 3.22e-9 +RESULT fd moe pocket output vectors: float64 reference against central differences at 4 coordinates, worst relative 2.31e-12 +RESULT moe weights: 5513.8 MiB (512 routed experts as MLX stacks and packed, the shared expert, the router, a pocket of 16) +RESULT moe 64 rows dx: worst column 3.29e-6 (column 2351 of 2560), relative L2 1.11e-6, largest |ref| 8.04e0 +RESULT moe 64 rows slot scores a = dL/dw: worst column 1.55e-6 (column 2 of 11), relative L2 1.12e-6, largest |ref| 1.22e2 +RESULT moe 64 rows router logits' gradients: worst column 1.51e-6 (column 2 of 11), relative L2 1.09e-6, largest |ref| 1.93e1 +RESULT moe 64 rows pocket forward: worst column 7.30e-7 (column 2325 of 2560), relative L2 2.69e-7, largest |ref| 1.04e0 +RESULT moe 64 rows pocket output vectors' gradients: worst column 1.36e-6 (column 791 of 2560), relative L2 2.97e-7, largest |ref| 1.00e0 +RESULT moe 64 rows: 17.562 ms on the GPU, 16.2 MiB of gradients and scratch +RESULT moe 64 rows packed dx: worst column 3.29e-6 (column 2351 of 2560), relative L2 1.11e-6, largest |ref| 8.04e0 +RESULT moe 64 rows packed: 14.167 ms, dx equals the MLX stacks' bits +RESULT moe 256 rows dx: worst column 2.53e-6 (column 443 of 2560), relative L2 1.13e-6, largest |ref| 1.02e1 +RESULT moe 256 rows slot scores a = dL/dw: worst column 1.83e-6 (column 9 of 11), relative L2 1.15e-6, largest |ref| 1.22e2 +RESULT moe 256 rows router logits' gradients: worst column 1.42e-6 (column 2 of 11), relative L2 1.14e-6, largest |ref| 2.27e1 +RESULT moe 256 rows pocket forward: worst column 5.27e-7 (column 892 of 2560), relative L2 2.68e-7, largest |ref| 1.06e0 +RESULT moe 256 rows pocket output vectors' gradients: worst column 1.14e-6 (column 1502 of 2560), relative L2 3.51e-7, largest |ref| 5.15e0 +RESULT moe 256 rows: 27.712 ms on the GPU, 64.9 MiB of gradients and scratch +RESULT moe 256 rows packed dx: worst column 2.53e-6 (column 443 of 2560), relative L2 1.13e-6, largest |ref| 1.02e1 +RESULT moe 256 rows packed: 21.458 ms, dx equals the MLX stacks' bits +PASS fn-moe-back: 16 checks within tolerance (13.9 s with the float64 references) +PASS fn-pocket: 444 cases byte-equal to tf_fn_pocket_rows (1 to 2048 rows, 1 to 128 neurons) +PASS fn-pocket controls: neurons in reverse change 12703 and 212652 values, inputs rotated by one 9633 and 153479 +PASS fn-pocket: no neuron in use launches nothing and keeps every output bit, -0.0 included +PASS gradient component suites (FP64 and finite-difference checks); resource receipts still required +PASS tf_moe_router: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_moe_topk: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_moe_combine: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_q36m_norm_rope: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_q36m_attn: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_q36m_attn_tree: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS plan_rows: buckets 1, 16, 32, 64, 114, 128 at every live count (wide past 1,024 pairs) equal the plan over the live pairs, five repeats, guards +RESULT fd mlp dh: float64 reference against central differences at 6 coordinates, worst relative 1.02e-7 +RESULT mlp 64 rows dy (MLP and pocket): worst column 6.93e-3 (column 5116 of 5120), relative L2 3.08e-3, largest |ref| 1.45e1 +RESULT mlp 64 rows dh (through the post norm): worst column 7.00e-3 (column 5116 of 5120), relative L2 3.08e-3, largest |ref| 1.38e1 +RESULT mlp 64 rows pocket output (bf16): worst column 3.88e-3 (column 187 of 5120), relative L2 1.66e-3, largest |ref| 1.96e0 +RESULT mlp 64 rows pocket output gradient: worst column 6.48e-7 (column 1289 of 5120), relative L2 1.98e-7, largest |ref| 2.79e0 +RESULT mlp 256 rows dy (MLP and pocket): worst column 6.00e-3 (column 1334 of 5120), relative L2 3.07e-3, largest |ref| 1.46e1 +RESULT mlp 256 rows dh (through the post norm): worst column 6.00e-3 (column 5116 of 5120), relative L2 3.07e-3, largest |ref| 1.60e1 +RESULT mlp 256 rows pocket output (bf16): worst column 3.86e-3 (column 2234 of 5120), relative L2 1.66e-3, largest |ref| 2.64e0 +RESULT mlp 256 rows pocket output gradient: worst column 5.18e-7 (column 4774 of 5120), relative L2 2.46e-7, largest |ref| 2.11e0 +RESULT fd attn dx: float64 reference against central differences at 6 coordinates, worst relative 4.46e-8 +RESULT attn 64 rows prepared heads: the largest difference 7.81e-3 of max(1, |value|) +RESULT attn 64 rows dpq (q and gate): worst column 1.46e-2 (column 2445 of 12288), relative L2 2.06e-3, largest |ref| 8.65e-1 +RESULT attn 64 rows dpk: worst column 6.95e-3 (column 1000 of 1024), relative L2 2.02e-3, largest |ref| 1.37e0 +RESULT attn 64 rows dv: worst column 1.11e-2 (column 1010 of 1024), relative L2 2.09e-3, largest |ref| 4.38e0 +RESULT attn 64 rows dx: worst column 1.67e-2 (column 2129 of 5120), relative L2 3.07e-3, largest |ref| 2.49e0 +RESULT attn 256 rows prepared heads: the largest difference 7.81e-3 of max(1, |value|) +RESULT attn 256 rows dpq (q and gate): worst column 1.46e-2 (column 2445 of 12288), relative L2 2.09e-3, largest |ref| 8.65e-1 +RESULT attn 256 rows dpk: worst column 6.56e-3 (column 674 of 1024), relative L2 2.07e-3, largest |ref| 1.37e0 +RESULT attn 256 rows dv: worst column 1.01e-2 (column 640 of 1024), relative L2 2.10e-3, largest |ref| 4.47e0 +RESULT attn 256 rows dx: worst column 1.73e-2 (column 4503 of 5120), relative L2 3.07e-3, largest |ref| 2.55e0 +RESULT fd final norm dh: float64 reference against central differences at 6 coordinates, worst relative 5.07e-8 +RESULT head 64 rows dlast (head back): worst column 3.96e-3 (column 4322 of 5120), relative L2 1.50e-3, largest |ref| 2.11e-4 +RESULT head 64 rows dh (final norm): worst column 3.97e-3 (column 4330 of 5120), relative L2 1.50e-3, largest |ref| 2.33e-4 +RESULT head 256 rows dlast (head back): worst column 3.94e-3 (column 270 of 5120), relative L2 1.51e-3, largest |ref| 2.26e-4 +RESULT head 256 rows dh (final norm): worst column 3.96e-3 (column 1571 of 5120), relative L2 1.51e-3, largest |ref| 2.47e-4 +PASS q27-back all: 25 checks within tolerance (19.8 s with the float64 references) +RESULT q27-gemm [984, 17408] x 4-bit [17408, 5120]: 64x64 11.33 ms (15.5 TFLOPS), wide 3.88 ms (45.3 TFLOPS) +RESULT q27-gemm [984, 5120] x 4-bit [5120, 17408]: 64x64 11.47 ms (15.3 TFLOPS), wide 3.77 ms (46.5 TFLOPS) +RESULT q27-gemm [984, 12288] x 4-bit [12288, 5120]: 64x64 7.79 ms (15.9 TFLOPS), wide 2.60 ms (47.6 TFLOPS) +RESULT q27-gemm [984, 1024] x 4-bit [1024, 5120]: 64x64 0.65 ms (15.9 TFLOPS), wide 0.25 ms (40.9 TFLOPS) +RESULT q27-gemm [984, 5120] x 4-bit [5120, 6144]: 64x64 3.86 ms (16.0 TFLOPS), wide 1.22 ms (50.9 TFLOPS) +PASS q27-gemm: 181 cases byte for byte (wide tiles against 64x64 and against the DeltaNet's dequantized GEMM, gathered rows against whole) +RESULT gdn-back finite differences (6 rows, 2 key / 6 value heads): 120 coordinates of P, worst |fd - analytic| / max(|analytic|, 1e-3 max) 7.91e-7 +RESULT gdn-back finite differences (9 rows, 1 key / 3 value heads): 120 coordinates of P, worst |fd - analytic| / max(|analytic|, 1e-3 max) 1.43e-6 +--- gdn-back: 64 rows, after a prefix (conv history and state) +RESULT gdn-back time 64 rows: layer backward 5.160 ms (forward for training, chain and gates 1.027 ms, the rest the two projections' dequant and GEMM); scratch 247.6 MiB (checkpoints 48.0, replay 12.0, dequantized projection 160.9) +RESULT gdn-back steps 64 rows (ms): front 0.015, scan_fwd 0.268, norm_back 0.038, scan_back 0.646, qk_back 0.056, gates_back 0.006, conv_back 0.029 +RESULT gdn-back forward act 3 of 655360 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 3072 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 50 of 393216 bf16 values differ from float64's rounding, at most 2 ulp +RESULT gdn-back forward q worst column rel 1.98e-7 (col 1743), max abs 5.67e-9, ref scale 5.04e-2 +RESULT gdn-back forward k worst column rel 1.71e-7 (col 1920), max abs 6.66e-8, ref scale 6.00e-1 +RESULT gdn-back forward g worst column rel 9.24e-8 (col 27), max abs 8.79e-8, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 1.35e-3 (col 2595), max abs 7.63e-6, ref scale 6.30e-2 +RESULT gdn-back dout (o_proj back) worst column rel 5.04e-3 (col 860), max abs 1.28e-4, ref scale 4.79e-2 +RESULT gdn-back dout, bf16 operands worst column rel 9.71e-6 (col 3003), max abs 2.89e-7, ref scale 4.79e-2 +RESULT gdn-back dy (norm back) worst column rel 2.72e-7 (col 3697), max abs 6.90e-6, ref scale 4.66e1 +RESULT gdn-back dz worst column rel 8.70e-7 (col 411), max abs 6.54e-9, ref scale 4.32e-2 +RESULT gdn-back dq (chain) worst column rel 7.07e-7 (col 1766), max abs 4.92e-6, ref scale 2.78e1 +RESULT gdn-back dk (chain) worst column rel 7.29e-7 (col 1544), max abs 3.90e-7, ref scale 3.28e0 +RESULT gdn-back dv (chain) worst column rel 9.06e-7 (col 2625), max abs 4.27e-8, ref scale 2.05e-1 +RESULT gdn-back dg (chain) worst column rel 9.23e-7 (col 16), max abs 2.29e-7, ref scale 6.83e-1 +RESULT gdn-back dbeta (chain) worst column rel 3.65e-7 (col 45), max abs 1.26e-7, ref scale 8.15e-1 +RESULT gdn-back dact q (L2 back) worst column rel 5.46e-7 (col 393), max abs 9.10e-8, ref scale 5.62e-1 +RESULT gdn-back dact k (L2 back) worst column rel 5.80e-7 (col 787), max abs 1.15e-7, ref scale 8.97e-1 +RESULT gdn-back dproj q (conv back) worst column rel 1.00e-6 (col 1986), max abs 3.65e-8, ref scale 1.94e-1 +RESULT gdn-back dproj k (conv back) worst column rel 9.57e-7 (col 1795), max abs 5.86e-8, ref scale 2.39e-1 +RESULT gdn-back dproj v (conv back) worst column rel 1.07e-6 (col 4468), max abs 2.04e-8, ref scale 8.02e-2 +RESULT gdn-back dproj b (beta back) worst column rel 5.17e-7 (col 31), max abs 2.30e-8, ref scale 8.67e-2 +RESULT gdn-back dproj a (g back) worst column rel 8.71e-7 (col 38), max abs 3.00e-8, ref scale 7.45e-2 +RESULT gdn-back dx (in_proj back) worst column rel 5.15e-3 (col 4114), max abs 2.34e-4, ref scale 1.10e-1 +RESULT gdn-back dx, bf16 operands worst column rel 3.99e-5 (col 168), max abs 2.84e-6, ref scale 1.10e-1 +--- gdn-back: 256 rows, from the sequence start +RESULT gdn-back time 256 rows: layer backward 8.565 ms (forward for training, chain and gates 3.972 ms, the rest the two projections' dequant and GEMM); scratch 471.5 MiB (checkpoints 192.0, replay 12.0, dequantized projection 160.9) +RESULT gdn-back steps 256 rows (ms): front 0.064, scan_fwd 1.066, norm_back 0.133, scan_back 2.561, qk_back 0.271, gates_back 0.006, conv_back 0.100 +RESULT gdn-back forward act 21 of 2621440 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 12288 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 192 of 1572864 bf16 values differ from float64's rounding, at most 5 ulp +RESULT gdn-back forward q worst column rel 1.94e-7 (col 1588), max abs 6.53e-9, ref scale 5.33e-2 +RESULT gdn-back forward k worst column rel 1.59e-7 (col 1784), max abs 8.37e-8, ref scale 6.51e-1 +RESULT gdn-back forward g worst column rel 1.15e-7 (col 46), max abs 1.11e-7, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 4.00e-3 (col 2186), max abs 6.10e-5, ref scale 5.64e-2 +RESULT gdn-back dout (o_proj back) worst column rel 5.91e-3 (col 3943), max abs 1.11e-4, ref scale 4.96e-2 +RESULT gdn-back dout, bf16 operands worst column rel 1.02e-5 (col 4059), max abs 3.01e-7, ref scale 4.96e-2 +RESULT gdn-back dy (norm back) worst column rel 3.20e-7 (col 4726), max abs 7.90e-6, ref scale 5.12e1 +RESULT gdn-back dz worst column rel 6.40e-7 (col 1694), max abs 8.72e-9, ref scale 4.92e-2 +RESULT gdn-back dq (chain) worst column rel 5.21e-7 (col 284), max abs 4.45e-6, ref scale 2.89e1 +RESULT gdn-back dk (chain) worst column rel 5.28e-7 (col 402), max abs 3.32e-7, ref scale 2.36e0 +RESULT gdn-back dv (chain) worst column rel 6.85e-7 (col 5917), max abs 6.13e-8, ref scale 2.33e-1 +RESULT gdn-back dg (chain) worst column rel 7.49e-7 (col 28), max abs 1.63e-7, ref scale 8.87e-1 +RESULT gdn-back dbeta (chain) worst column rel 4.28e-7 (col 39), max abs 1.79e-7, ref scale 9.67e-1 +RESULT gdn-back dact q (L2 back) worst column rel 7.44e-7 (col 1499), max abs 1.14e-7, ref scale 7.52e-1 +RESULT gdn-back dact k (L2 back) worst column rel 6.27e-7 (col 402), max abs 9.08e-8, ref scale 6.36e-1 +RESULT gdn-back dproj q (conv back) worst column rel 6.18e-7 (col 380), max abs 4.92e-8, ref scale 2.30e-1 +RESULT gdn-back dproj k (conv back) worst column rel 7.92e-7 (col 1801), max abs 4.46e-8, ref scale 2.12e-1 +RESULT gdn-back dproj v (conv back) worst column rel 9.56e-7 (col 6112), max abs 2.74e-8, ref scale 9.35e-2 +RESULT gdn-back dproj b (beta back) worst column rel 3.80e-7 (col 3), max abs 2.42e-8, ref scale 1.22e-1 +RESULT gdn-back dproj a (g back) worst column rel 6.33e-7 (col 28), max abs 3.23e-8, ref scale 8.40e-2 +RESULT gdn-back dx (in_proj back) worst column rel 6.78e-3 (col 3513), max abs 2.64e-4, ref scale 1.13e-1 +RESULT gdn-back dx, bf16 operands worst column rel 4.36e-5 (col 4893), max abs 2.75e-6, ref scale 1.13e-1 +--- gdn-back: 37 rows, after a prefix (conv history and state) +RESULT gdn-back time 37 rows: layer backward 4.636 ms (forward for training, chain and gates 0.623 ms, the rest the two projections' dequant and GEMM); scratch 218.3 MiB (checkpoints 30.0, replay 12.0, dequantized projection 160.9) +RESULT gdn-back steps 37 rows (ms): front 0.011, scan_fwd 0.165, norm_back 0.024, scan_back 0.404, qk_back 0.027, gates_back 0.004, conv_back 0.018 +RESULT gdn-back forward act 1 of 378880 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 1776 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 28 of 227328 bf16 values differ from float64's rounding, at most 26 ulp +RESULT gdn-back forward q worst column rel 1.68e-7 (col 1069), max abs 7.32e-9, ref scale 5.34e-2 +RESULT gdn-back forward k worst column rel 1.54e-7 (col 1790), max abs 5.85e-8, ref scale 6.13e-1 +RESULT gdn-back forward g worst column rel 1.10e-7 (col 47), max abs 1.09e-7, ref scale 9.94e-1 +RESULT gdn-back forward y worst column rel 1.19e-3 (col 6101), max abs 1.53e-5, ref scale 5.57e-2 +RESULT gdn-back dout (o_proj back) worst column rel 1.27e-2 (col 5249), max abs 1.02e-4, ref scale 4.46e-2 +RESULT gdn-back dout, bf16 operands worst column rel 1.90e-5 (col 5249), max abs 3.17e-7, ref scale 4.47e-2 +RESULT gdn-back dy (norm back) worst column rel 2.51e-7 (col 455), max abs 5.20e-6, ref scale 4.33e1 +RESULT gdn-back dz worst column rel 1.23e-6 (col 1276), max abs 4.89e-9, ref scale 3.61e-2 +RESULT gdn-back dq (chain) worst column rel 8.41e-7 (col 1089), max abs 2.93e-6, ref scale 1.54e1 +RESULT gdn-back dk (chain) worst column rel 1.08e-6 (col 1755), max abs 2.55e-7, ref scale 1.50e0 +RESULT gdn-back dv (chain) worst column rel 7.96e-7 (col 3048), max abs 4.30e-8, ref scale 1.82e-1 +RESULT gdn-back dg (chain) worst column rel 8.78e-7 (col 41), max abs 2.06e-7, ref scale 1.39e0 +RESULT gdn-back dbeta (chain) worst column rel 6.15e-7 (col 10), max abs 1.42e-7, ref scale 7.35e-1 +RESULT gdn-back dact q (L2 back) worst column rel 7.59e-7 (col 1089), max abs 8.93e-8, ref scale 3.96e-1 +RESULT gdn-back dact k (L2 back) worst column rel 1.15e-6 (col 1755), max abs 1.30e-7, ref scale 4.26e-1 +RESULT gdn-back dproj q (conv back) worst column rel 1.08e-6 (col 1527), max abs 2.46e-8, ref scale 1.24e-1 +RESULT gdn-back dproj k (conv back) worst column rel 1.12e-6 (col 1755), max abs 3.79e-8, ref scale 1.11e-1 +RESULT gdn-back dproj v (conv back) worst column rel 9.21e-7 (col 3048), max abs 2.27e-8, ref scale 9.31e-2 +RESULT gdn-back dproj b (beta back) worst column rel 5.12e-7 (col 10), max abs 1.78e-8, ref scale 8.92e-2 +RESULT gdn-back dproj a (g back) worst column rel 1.00e-6 (col 39), max abs 3.63e-8, ref scale 1.02e-1 +RESULT gdn-back dx (in_proj back) worst column rel 9.84e-3 (col 3358), max abs 1.99e-4, ref scale 8.87e-2 +RESULT gdn-back dx, bf16 operands worst column rel 5.18e-5 (col 103), max abs 2.52e-6, ref scale 8.87e-2 +PASS gdn-back: forward for training and every input gradient within tolerance of float64 +qwen35 tf_q35_gdn_slots_1 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_1 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_1 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_1 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_1 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_121: no compiled usage recorded +qwen35 tf_q35_rope on sm_80: no compiled usage recorded +qwen35 tf_q35_rope on sm_86: no compiled usage recorded +qwen35 tf_q35_rope on sm_89: no compiled usage recorded +qwen35 tf_q35_rope on sm_120: no compiled usage recorded +qwen35 tf_q35_rope on sm_121: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_80: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_86: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_89: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_120: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_121: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_80: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_86: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_89: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_120: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_121: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_80: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_86: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_89: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_120: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_121: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_80: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_86: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_89: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_120: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_121: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_80: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_86: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_89: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_120: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_121: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_80: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_86: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_89: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_120: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_121: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_80: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_86: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_89: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_120: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_121: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_80: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_86: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_89: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_120: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_121: no compiled usage recorded +qwen35 tf_q35_tap on sm_80: no compiled usage recorded +qwen35 tf_q35_tap on sm_86: no compiled usage recorded +qwen35 tf_q35_tap on sm_89: no compiled usage recorded +qwen35 tf_q35_tap on sm_120: no compiled usage recorded +qwen35 tf_q35_tap on sm_121: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_80: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_86: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_89: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_120: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_121: no compiled usage recorded +tf_topk_slices on sm_80: no compiled usage recorded +tf_topk_slices on sm_86: no compiled usage recorded +tf_topk_slices on sm_89: no compiled usage recorded +tf_topk_slices on sm_120: no compiled usage recorded +tf_topk_slices on sm_121: no compiled usage recorded +tf_topk_merge on sm_80: no compiled usage recorded +tf_topk_merge on sm_86: no compiled usage recorded +tf_topk_merge on sm_89: no compiled usage recorded +tf_topk_merge on sm_120: no compiled usage recorded +tf_topk_merge on sm_121: no compiled usage recorded +tf_q35_norm on sm_80: no compiled usage recorded +tf_q35_norm on sm_86: no compiled usage recorded +tf_q35_norm on sm_89: no compiled usage recorded +tf_q35_norm on sm_120: no compiled usage recorded +tf_q35_norm on sm_121: no compiled usage recorded +tf_q35_sums on sm_80: no compiled usage recorded +tf_q35_sums on sm_86: no compiled usage recorded +tf_q35_sums on sm_89: no compiled usage recorded +tf_q35_sums on sm_120: no compiled usage recorded +tf_q35_sums on sm_121: no compiled usage recorded +tf_q35_argmax_slices on sm_80: no compiled usage recorded +tf_q35_argmax_slices on sm_86: no compiled usage recorded +tf_q35_argmax_slices on sm_89: no compiled usage recorded +tf_q35_argmax_slices on sm_120: no compiled usage recorded +tf_q35_argmax_slices on sm_121: no compiled usage recorded +tf_q35_argmax_merge on sm_80: no compiled usage recorded +tf_q35_argmax_merge on sm_86: no compiled usage recorded +tf_q35_argmax_merge on sm_89: no compiled usage recorded +tf_q35_argmax_merge on sm_120: no compiled usage recorded +tf_q35_argmax_merge on sm_121: no compiled usage recorded +sample tf_draw on sm_80: no compiled usage recorded +sample tf_draw on sm_86: no compiled usage recorded +sample tf_draw on sm_89: no compiled usage recorded +sample tf_draw on sm_120: no compiled usage recorded +sample tf_draw on sm_121: no compiled usage recorded +sample tf_logprob_rows on sm_80: no compiled usage recorded +sample tf_logprob_rows on sm_86: no compiled usage recorded +sample tf_logprob_rows on sm_89: no compiled usage recorded +sample tf_logprob_rows on sm_120: no compiled usage recorded +sample tf_logprob_rows on sm_121: no compiled usage recorded +sample tf_grammar_mask on sm_80: no compiled usage recorded +sample tf_grammar_mask on sm_86: no compiled usage recorded +sample tf_grammar_mask on sm_89: no compiled usage recorded +sample tf_grammar_mask on sm_120: no compiled usage recorded +sample tf_grammar_mask on sm_121: no compiled usage recorded +glm_glue tf_glm_top1 on sm_80: no compiled usage recorded +glm_glue tf_glm_top1 on sm_86: no compiled usage recorded +glm_glue tf_glm_top1 on sm_89: no compiled usage recorded +glm_glue tf_glm_top1 on sm_120: no compiled usage recorded +glm_glue tf_glm_top1 on sm_121: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_80: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_86: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_89: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_120: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_121: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_80: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_86: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_89: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_120: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_121: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_80: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_86: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_89: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_120: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii on sm_121: no compiled usage recorded +dsv41_router_glue router_group_rot on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE on sm_121: no compiled usage recorded +dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi on sm_121: no compiled usage recorded +dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm on sm_121: no compiled usage recorded +dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij on sm_121: no compiled usage recorded +dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij on sm_121: no compiled usage recorded +dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx on sm_121: no compiled usage recorded +dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif on sm_121: no compiled usage recorded +dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx on sm_121: no compiled usage recorded +dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx on sm_121: no compiled usage recorded +dsv41_kv_glue kv_norm_store on sm_121: no compiled usage recorded +dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd on sm_121: no compiled usage recorded +dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf on sm_121: no compiled usage recorded +dsv41_sampling dsv41_sampling_stats_max on sm_121: no compiled usage recorded +dsv41_sampling dsv41_sampling_stats_sum on sm_121: no compiled usage recorded +dsv41_sampling dsv41_sampling_stats_fin on sm_121: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_80: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_86: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_89: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_120: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_121: no compiled usage recorded +tree_round tf_tree_round_match on sm_80: no compiled usage recorded +tree_round tf_tree_round_match on sm_86: no compiled usage recorded +tree_round tf_tree_round_match on sm_89: no compiled usage recorded +tree_round tf_tree_round_match on sm_120: no compiled usage recorded +tree_round tf_tree_round_match on sm_121: no compiled usage recorded +tree_round tf_round_keep_rows on sm_80: no compiled usage recorded +tree_round tf_round_keep_rows on sm_86: no compiled usage recorded +tree_round tf_round_keep_rows on sm_89: no compiled usage recorded +tree_round tf_round_keep_rows on sm_120: no compiled usage recorded +tree_round tf_round_keep_rows on sm_121: no compiled usage recorded +tree_round tf_round_gather_rows on sm_80: no compiled usage recorded +tree_round tf_round_gather_rows on sm_86: no compiled usage recorded +tree_round tf_round_gather_rows on sm_89: no compiled usage recorded +tree_round tf_round_gather_rows on sm_120: no compiled usage recorded +tree_round tf_round_gather_rows on sm_121: no compiled usage recorded +tree_round tf_round_keep_head on sm_80: no compiled usage recorded +tree_round tf_round_keep_head on sm_86: no compiled usage recorded +tree_round tf_round_keep_head on sm_89: no compiled usage recorded +tree_round tf_round_keep_head on sm_120: no compiled usage recorded +tree_round tf_round_keep_head on sm_121: no compiled usage recorded +act_rows tf_swiglu_rows on sm_80: no compiled usage recorded +act_rows tf_swiglu_rows on sm_86: no compiled usage recorded +act_rows tf_swiglu_rows on sm_89: no compiled usage recorded +act_rows tf_swiglu_rows on sm_120: no compiled usage recorded +act_rows tf_swiglu_rows on sm_121: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_80: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_86: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_89: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_120: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_121: no compiled usage recorded +act_rows tf_gate_rows on sm_80: no compiled usage recorded +act_rows tf_gate_rows on sm_86: no compiled usage recorded +act_rows tf_gate_rows on sm_89: no compiled usage recorded +act_rows tf_gate_rows on sm_120: no compiled usage recorded +act_rows tf_gate_rows on sm_121: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_80: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_86: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_89: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_120: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_121: no compiled usage recorded +act_rows tf_group_sums_rows on sm_80: no compiled usage recorded +act_rows tf_group_sums_rows on sm_86: no compiled usage recorded +act_rows tf_group_sums_rows on sm_89: no compiled usage recorded +act_rows tf_group_sums_rows on sm_120: no compiled usage recorded +act_rows tf_group_sums_rows on sm_121: no compiled usage recorded +act_rows tf_dconv_rows on sm_80: no compiled usage recorded +act_rows tf_dconv_rows on sm_86: no compiled usage recorded +act_rows tf_dconv_rows on sm_89: no compiled usage recorded +act_rows tf_dconv_rows on sm_120: no compiled usage recorded +act_rows tf_dconv_rows on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_121: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_80: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_86: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_89: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_120: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_121: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_80: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_86: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_89: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_120: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_121: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_80: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_86: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_89: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_120: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_121: no compiled usage recorded +FAIL limits: 1161 registered CUDA launches against sm_80/86/89/120/121, 885 problems +PASS limits live sm_121 most shared bytes a block (opt-in): the driver says 101376, the table 101376 +PASS limits live sm_121 threads a block: the driver says 1024, the table 1024 +INFO limits live: images built by nvcc 13.3, checked against nvcc 13.3's records +FAIL limits live flashnext_gdn flashnext_gdn_chain: 1024 threads launched, the driver allows 1024; 61 registers, 2060 + 0 shared bytes (recorded 64, 2060) +PASS limits live flashnext_hc fn_add_streams: 256 threads launched, the driver allows 1024; 35 registers, 0 + 0 shared bytes (recorded 35, 0) +PASS limits live flashnext_attn fn_attn_chunks: 128 threads launched, the driver allows 128; 72 registers, 640 + 43520 shared bytes (recorded 72, 640) +PASS limits live flashnext_attn fn_attn_chunks_ahead: 128 threads launched, the driver allows 128; 72 registers, 768 + 77312 shared bytes (recorded 72, 768) +PASS limits live flashnext_attn fn_attn_gate: 256 threads launched, the driver allows 256; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +FAIL limits live flashnext_attn fn_attn_merge: 256 threads launched, the driver allows 256; 25 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live flashnext_attn fn_attn_prep: 256 threads launched, the driver allows 256; 30 registers, 128 + 0 shared bytes (recorded 30, 128) +PASS limits live flashnext_batch fn_gdn_front: 256 threads launched, the driver allows 256; 30 registers, 1024 + 0 shared bytes (recorded 30, 1024) +PASS limits live flashnext_batch fn_gdn_norm: 128 threads launched, the driver allows 128; 24 registers, 528 + 0 shared bytes (recorded 24, 528) +PASS limits live flashnext_batch fn_gdn_rows: 128 threads launched, the driver allows 128; 64 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live flashnext_batch fn_gdn_rows_to: 128 threads launched, the driver allows 128; 64 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live flashnext_hc fn_hc_dense_mix: 32 threads launched, the driver allows 1024; 36 registers, 128 + 0 shared bytes (recorded 36, 128) +PASS limits live flashnext_hc fn_hc_down: 128 threads launched, the driver allows 128; 64 registers, 64 + 11136 shared bytes (recorded 64, 64) +PASS limits live flashnext_hc fn_hc_normalize: 256 threads launched, the driver allows 1024; 40 registers, 16 + 0 shared bytes (recorded 40, 16) +PASS limits live flashnext_hc fn_hc_reduce_act: 352 threads launched, the driver allows 1024; 60 registers, 4096 + 0 shared bytes (recorded 60, 4096) +PASS limits live flashnext_hc fn_hc_upmix: 256 threads launched, the driver allows 256; 62 registers, 2048 + 18688 shared bytes (recorded 62, 2048) +PASS limits live flashnext_hc fn_hc_writeback0: 256 threads launched, the driver allows 256; 37 registers, 256 + 0 shared bytes (recorded 37, 256) +PASS limits live flashnext_hc fn_hc_writeback_moe: 256 threads launched, the driver allows 256; 47 registers, 256 + 0 shared bytes (recorded 47, 256) +PASS limits live flashnext_hc fn_hc_writeback_slices: 256 threads launched, the driver allows 256; 40 registers, 256 + 0 shared bytes (recorded 40, 256) +PASS limits live flashnext_hc fn_ple_conv: 512 threads launched, the driver allows 512; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live flashnext_hc fn_ple_gate: 512 threads launched, the driver allows 512; 33 registers, 128 + 0 shared bytes (recorded 33, 128) +PASS limits live flashnext_attn fn_qsa_pool: 128 threads launched, the driver allows 128; 39 registers, 1152 + 0 shared bytes (recorded 39, 1152) +PASS limits live flashnext_attn fn_qsa_scores: 256 threads launched, the driver allows 256; 48 registers, 4096 + 0 shared bytes (recorded 48, 4096) +PASS limits live flashnext_batch fn_qsa_select: 1024 threads launched, the driver allows 1024; 44 registers, 1168 + 0 shared bytes (recorded 44, 1168) +PASS limits live flashnext_mtp fn_row_best: 1024 threads launched, the driver allows 1024; 21 registers, 392 + 0 shared bytes (recorded 21, 392) +PASS limits live flashnext_hc fn_shared_scatter: 256 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live flashnext_hc fn_shared_swiglu: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live flashnext_batch fn_shift_window: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live flashnext_batch fn_sum_slices: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live embed_rows tf_embed_rows_bf16_copies: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live norms tf_rms_rows_sums: 1024 threads launched, the driver allows 1024; 34 registers, 132 + 0 shared bytes (recorded 34, 132) +PASS limits live route_rows tf_route_logits: 128 threads launched, the driver allows 128; 35 registers, 0 + 26112 shared bytes (recorded 35, 0) +PASS limits live route_rows tf_route_logits_few: 128 threads launched, the driver allows 128; 34 registers, 0 + 76032 shared bytes (recorded 34, 0) +PASS limits live route_rows tf_route_topk: 128 threads launched, the driver allows 128; 36 registers, 64 + 0 shared bytes (recorded 36, 64) +PASS limits live norms tf_rms_rows_bf16: 256 threads launched, the driver allows 256; 80 registers, 32 + 0 shared bytes (recorded 80, 32) +PASS limits live norms tf_rms_rows: 1024 threads launched, the driver allows 1024; 43 registers, 132 + 0 shared bytes (recorded 43, 132) +PASS limits live router tf_nemo_router: 256 threads launched, the driver allows 256; 40 registers, 0 + 28800 shared bytes (recorded 40, 0) +PASS limits live router tf_nemo_topk: 32 threads launched, the driver allows 32; 96 registers, 0 + 0 shared bytes (recorded 96, 0) +FAIL limits live router tf_nemo_topk_wide: 32 threads launched, the driver allows 32; 161 registers, 0 + 0 shared bytes (recorded 160, 0) +PASS limits live latent tf_super_topk_few: 1024 threads launched, the driver allows 1024; 32 registers, 4352 + 0 shared bytes (recorded 32, 4352) +PASS limits live qmm_group cluster of 8: 128 threads launched, the driver allows 128; 48 registers, 0 + 35328 shared bytes (recorded 48, 0) +PASS limits live qmm_group one CTA a tile: 128 threads launched, the driver allows 128; 48 registers, 0 + 35328 shared bytes (recorded 48, 0) +PASS limits live lane_gemv: 128 threads launched, the driver allows 128; 66 registers, 0 + 35328 shared bytes (recorded 66, 0) +PASS limits live lane_gemv split-K: 128 threads launched, the driver allows 128; 72 registers, 128 + 17664 shared bytes (recorded 72, 128) +PASS limits live prompt16 FP4: 128 threads launched, the driver allows 128; 212 registers, 0 + 41984 shared bytes (recorded 212, 0) +PASS limits live prompt16 FP8: 128 threads launched, the driver allows 128; 212 registers, 0 + 49152 shared bytes (recorded 212, 0) +PASS limits live prompt16 FP4 fp32 out: 128 threads launched, the driver allows 128; 212 registers, 0 + 41984 shared bytes (recorded 212, 0) +PASS limits live qmmf FP4 fp32 out 16-row tile: 128 threads launched, the driver allows 128; 56 registers, 0 + 17408 shared bytes (recorded 56, 0) +PASS limits live qmmf FP4 fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 48 registers, 0 + 17408 shared bytes (recorded 48, 0) +PASS limits live qmmf FP4 fp32 out 32-row tile: 128 threads launched, the driver allows 128; 64 registers, 0 + 25600 shared bytes (recorded 64, 0) +PASS limits live qmmf FP4 fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 61 registers, 0 + 25600 shared bytes (recorded 61, 0) +PASS limits live qmmf FP4 fp32 out 64-row tile: 128 threads launched, the driver allows 128; 78 registers, 0 + 41984 shared bytes (recorded 78, 0) +PASS limits live qmmf FP4 fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 78 registers, 0 + 41984 shared bytes (recorded 78, 0) +PASS limits live qmmf FP4 fp32 out fused slices: 128 threads launched, the driver allows 128; 168 registers, 0 + 41984 shared bytes (recorded 168, 0) +PASS limits live qmmf MXFP8 16-row tile: 128 threads launched, the driver allows 128; 60 registers, 0 + 25088 shared bytes (recorded 60, 0) +PASS limits live qmmf MXFP8 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 56 registers, 0 + 25088 shared bytes (recorded 56, 0) +PASS limits live qmmf MXFP8 32-row tile: 128 threads launched, the driver allows 128; 57 registers, 0 + 33280 shared bytes (recorded 57, 0) +PASS limits live qmmf MXFP8 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 66 registers, 0 + 33280 shared bytes (recorded 66, 0) +PASS limits live qmmf MXFP8 64-row tile: 128 threads launched, the driver allows 128; 108 registers, 0 + 49664 shared bytes (recorded 108, 0) +PASS limits live qmmf MXFP8 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 103 registers, 0 + 49664 shared bytes (recorded 103, 0) +PASS limits live qmmf MXFP8 64-row tile, fused slices: 128 threads launched, the driver allows 128; 158 registers, 0 + 49664 shared bytes (recorded 158, 0) +PASS limits live qmmf reduce_kernel: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qmmf FP8 fp32 out 16-row tile: 128 threads launched, the driver allows 128; 50 registers, 0 + 24576 shared bytes (recorded 50, 0) +PASS limits live qmmf FP8 fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 40 registers, 0 + 24576 shared bytes (recorded 40, 0) +PASS limits live qmmf FP8 fp32 out 32-row tile: 128 threads launched, the driver allows 128; 56 registers, 0 + 32768 shared bytes (recorded 56, 0) +PASS limits live qmmf FP8 fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 48 registers, 0 + 32768 shared bytes (recorded 48, 0) +PASS limits live qmmf FP8 fp32 out 64-row tile: 128 threads launched, the driver allows 128; 72 registers, 0 + 49152 shared bytes (recorded 72, 0) +PASS limits live qmmf FP8 fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 64 registers, 0 + 49152 shared bytes (recorded 64, 0) +PASS limits live qmmf FP8 fp32 out 64-row tile, fused slices: 128 threads launched, the driver allows 128; 140 registers, 0 + 49152 shared bytes (recorded 140, 0) +PASS limits live qmmf MXFP8 fp32 out 16-row tile: 128 threads launched, the driver allows 128; 60 registers, 0 + 25088 shared bytes (recorded 60, 0) +PASS limits live qmmf MXFP8 fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 56 registers, 0 + 25088 shared bytes (recorded 56, 0) +PASS limits live qmmf MXFP8 fp32 out 32-row tile: 128 threads launched, the driver allows 128; 57 registers, 0 + 33280 shared bytes (recorded 57, 0) +PASS limits live qmmf MXFP8 fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 66 registers, 0 + 33280 shared bytes (recorded 66, 0) +PASS limits live qmmf MXFP8 fp32 out 64-row tile: 128 threads launched, the driver allows 128; 108 registers, 0 + 49664 shared bytes (recorded 108, 0) +PASS limits live qmmf MXFP8 fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 103 registers, 0 + 49664 shared bytes (recorded 103, 0) +PASS limits live qmmf MXFP8 fp32 out 64-row tile, fused slices: 128 threads launched, the driver allows 128; 158 registers, 0 + 49664 shared bytes (recorded 158, 0) +PASS limits live qmmf FP8G fp32 out 16-row tile: 128 threads launched, the driver allows 128; 58 registers, 0 + 25600 shared bytes (recorded 58, 0) +PASS limits live qmmf FP8G fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 56 registers, 0 + 25600 shared bytes (recorded 56, 0) +PASS limits live qmmf FP8G fp32 out 32-row tile: 128 threads launched, the driver allows 128; 56 registers, 0 + 33792 shared bytes (recorded 56, 0) +PASS limits live qmmf FP8G fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 57 registers, 0 + 33792 shared bytes (recorded 57, 0) +PASS limits live qmmf FP8G fp32 out 64-row tile: 128 threads launched, the driver allows 128; 108 registers, 0 + 50176 shared bytes (recorded 108, 0) +PASS limits live qmmf FP8G fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 122 registers, 0 + 50176 shared bytes (recorded 122, 0) +PASS limits live qmmf FP8G fp32 out 64-row tile, fused slices: 128 threads launched, the driver allows 128; 155 registers, 0 + 50176 shared bytes (recorded 155, 0) +PASS limits live qmmf BF16 fp32 out 16-row tile: 128 threads launched, the driver allows 128; 48 registers, 0 + 40960 shared bytes (recorded 48, 0) +PASS limits live qmmf BF16 fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 40 registers, 0 + 40960 shared bytes (recorded 40, 0) +PASS limits live qmmf BF16 fp32 out 32-row tile: 128 threads launched, the driver allows 128; 55 registers, 0 + 49152 shared bytes (recorded 55, 0) +PASS limits live qmmf BF16 fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 48 registers, 0 + 49152 shared bytes (recorded 48, 0) +PASS limits live qmmf BF16 fp32 out 64-row tile: 128 threads launched, the driver allows 128; 72 registers, 0 + 65536 shared bytes (recorded 72, 0) +PASS limits live qmmf BF16 fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 64 registers, 0 + 65536 shared bytes (recorded 64, 0) +PASS limits live qmmf BF16 fp32 out 64-row tile, fused slices: 128 threads launched, the driver allows 128; 140 registers, 0 + 65536 shared bytes (recorded 140, 0) +PASS limits live qmmf reduce_kernel: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live ck tf_ck4_up_relu2_few: 128 threads launched, the driver allows 128; 124 registers, 0 + 0 shared bytes (recorded 124, 0) +PASS limits live ck tf_ck4_down_f32_few: 128 threads launched, the driver allows 128; 128 registers, 0 + 0 shared bytes (recorded 128, 0) +FAIL limits live mamba tf_nemo_scan: 128 threads launched, the driver allows 128; 115 registers, 41088 + 0 shared bytes (recorded 118, 41088) +PASS limits live rank_sum tf_rank_sum: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live vit vision_kernel<80, 8> (RADIO): 256 threads launched, the driver allows 256; 122 registers, 0 + 45056 shared bytes (recorded 122, 0) +PASS limits live vit full_kernel<72> (SigLIP2): 256 threads launched, the driver allows 256; 169 registers, 0 + 65536 shared bytes (recorded 169, 0) +PASS limits live vit tf_vit_layer_norm: 256 threads launched, the driver allows 256; 40 registers, 32 + 0 shared bytes (recorded 40, 32) +PASS limits live vit tf_vit_pointwise: 256 threads launched, the driver allows 256; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live vit tf_vit_bias: 256 threads launched, the driver allows 256; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live vit tf_vit_unshuffle: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live qwen_vision tf_qv_embed: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen_vision tf_qv_layernorm: 256 threads launched, the driver allows 256; 23 registers, 132 + 0 shared bytes (recorded 23, 132) +PASS limits live qwen_vision tf_qv_qkv_rope: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen_vision tf_qv_attention72: 64 threads launched, the driver allows 64; 186 registers, 36864 + 0 shared bytes (recorded 186, 36864) +PASS limits live qwen_vision tf_qv_attention64: 64 threads launched, the driver allows 64; 162 registers, 32768 + 0 shared bytes (recorded 162, 32768) +PASS limits live qwen_vision tf_qv_bias_gelu: 256 threads launched, the driver allows 256; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live qwen_vision tf_qv_bias_residual: 256 threads launched, the driver allows 256; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live qwen_vision tf_qv_bias_cast: 256 threads launched, the driver allows 256; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live qwen_vision tf_qv_splice: 256 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live flashnext_attn fn_attn_prep_media: 256 threads launched, the driver allows 256; 30 registers, 128 + 0 shared bytes (recorded 30, 128) +PASS limits live flashnext_attn fn_qsa_pool_media: 128 threads launched, the driver allows 128; 39 registers, 1152 + 0 shared bytes (recorded 39, 1152) +PASS limits live exl3_linear tf_exl3_rot_in: 128 threads launched, the driver allows 128; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live exl3_linear tf_exl3_linear_2_2_4: 128 threads launched, the driver allows 128; 154 registers, 16 + 16384 shared bytes (recorded 154, 16) +PASS limits live exl3_linear tf_exl3_linear_4_2_4: 128 threads launched, the driver allows 128; 154 registers, 16 + 16384 shared bytes (recorded 154, 16) +PASS limits live exl3_linear tf_exl3_linear_6_2_4: 128 threads launched, the driver allows 128; 159 registers, 16 + 16384 shared bytes (recorded 159, 16) +PASS limits live exl3_linear tf_exl3_linear_8_2_4: 128 threads launched, the driver allows 128; 156 registers, 16 + 16384 shared bytes (recorded 156, 16) +PASS limits live exl3_linear tf_exl3_linear_10_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_12_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_14_2_4: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_2_4: 128 threads launched, the driver allows 128; 125 registers, 16 + 16384 shared bytes (recorded 125, 16) +PASS limits live exl3_linear tf_exl3_linear_3_2_4: 128 threads launched, the driver allows 128; 156 registers, 16 + 16384 shared bytes (recorded 156, 16) +PASS limits live exl3_linear tf_exl3_linear_5_2_4: 128 threads launched, the driver allows 128; 157 registers, 16 + 16384 shared bytes (recorded 157, 16) +PASS limits live exl3_linear tf_exl3_linear_7_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_9_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_11_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_13_2_4: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_2_1_4: 128 threads launched, the driver allows 128; 158 registers, 16 + 16384 shared bytes (recorded 158, 16) +PASS limits live exl3_linear tf_exl3_linear_4_1_4: 128 threads launched, the driver allows 128; 158 registers, 16 + 16384 shared bytes (recorded 158, 16) +PASS limits live exl3_linear tf_exl3_linear_6_1_4: 128 threads launched, the driver allows 128; 162 registers, 16 + 16384 shared bytes (recorded 162, 16) +PASS limits live exl3_linear tf_exl3_linear_8_1_4: 128 threads launched, the driver allows 128; 160 registers, 16 + 16384 shared bytes (recorded 160, 16) +PASS limits live exl3_linear tf_exl3_linear_10_1_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_12_1_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_14_1_4: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_1_4: 128 threads launched, the driver allows 128; 123 registers, 16 + 16384 shared bytes (recorded 123, 16) +PASS limits live exl3_linear tf_exl3_linear_2_0_4: 128 threads launched, the driver allows 128; 156 registers, 16 + 16384 shared bytes (recorded 156, 16) +PASS limits live exl3_linear tf_exl3_linear_4_0_4: 128 threads launched, the driver allows 128; 156 registers, 16 + 16384 shared bytes (recorded 156, 16) +PASS limits live exl3_linear tf_exl3_linear_6_0_4: 128 threads launched, the driver allows 128; 160 registers, 16 + 16384 shared bytes (recorded 160, 16) +PASS limits live exl3_linear tf_exl3_linear_8_0_4: 128 threads launched, the driver allows 128; 162 registers, 16 + 16384 shared bytes (recorded 162, 16) +PASS limits live exl3_linear tf_exl3_linear_10_0_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_12_0_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_14_0_4: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_0_4: 128 threads launched, the driver allows 128; 125 registers, 16 + 16384 shared bytes (recorded 125, 16) +PASS limits live exl3_linear tf_exl3_linear_2_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_4_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_6_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_8_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_10_2_8: 256 threads launched, the driver allows 256; 228 registers, 16 + 32768 shared bytes (recorded 228, 16) +PASS limits live exl3_linear tf_exl3_linear_12_2_8: 256 threads launched, the driver allows 256; 226 registers, 16 + 32768 shared bytes (recorded 226, 16) +PASS limits live exl3_linear tf_exl3_linear_14_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_3_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_5_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_7_2_8: 256 threads launched, the driver allows 256; 236 registers, 16 + 32768 shared bytes (recorded 236, 16) +PASS limits live exl3_linear tf_exl3_linear_9_2_8: 256 threads launched, the driver allows 256; 224 registers, 16 + 32768 shared bytes (recorded 224, 16) +PASS limits live exl3_linear tf_exl3_linear_11_2_8: 256 threads launched, the driver allows 256; 226 registers, 16 + 32768 shared bytes (recorded 226, 16) +PASS limits live exl3_linear tf_exl3_linear_13_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_2_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_4_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_6_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_8_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_10_1_8: 256 threads launched, the driver allows 256; 241 registers, 16 + 32768 shared bytes (recorded 241, 16) +PASS limits live exl3_linear tf_exl3_linear_12_1_8: 256 threads launched, the driver allows 256; 240 registers, 16 + 32768 shared bytes (recorded 240, 16) +PASS limits live exl3_linear tf_exl3_linear_14_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_2_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_4_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_6_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_8_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_10_0_8: 256 threads launched, the driver allows 256; 231 registers, 16 + 32768 shared bytes (recorded 231, 16) +PASS limits live exl3_linear tf_exl3_linear_12_0_8: 256 threads launched, the driver allows 256; 231 registers, 16 + 32768 shared bytes (recorded 231, 16) +PASS limits live exl3_linear tf_exl3_linear_14_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_experts tf_exl3_group: 1024 threads launched, the driver allows 1024; 42 registers, 128 + 49152 shared bytes (recorded 42, 128) +PASS limits live exl3_experts tf_exl3_experts_rot_in: 32 threads launched, the driver allows 1024; 19 registers, 0 + 0 shared bytes (recorded 19, 0) +PASS limits live exl3_experts tf_exl3_grouped_0: 128 threads launched, the driver allows 512; 127 registers, 32832 + 0 shared bytes (recorded 127, 32832) +PASS limits live exl3_experts tf_exl3_grouped_1: 128 threads launched, the driver allows 512; 128 registers, 32832 + 0 shared bytes (recorded 128, 32832) +PASS limits live exl3_experts tf_exl3_grouped_2: 128 threads launched, the driver allows 512; 128 registers, 32832 + 0 shared bytes (recorded 128, 32832) +PASS limits live exl3_experts tf_exl3_gateup_epilogue: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live exl3_experts tf_exl3_down_combine: 1024 threads launched, the driver allows 1024; 42 registers, 16384 + 0 shared bytes (recorded 42, 16384) +PASS limits live exl3_experts tf_exl3_plan_rotate: 32 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +FAIL limits live exl3_experts tf_exl3_plan_finish: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 42, 0) +PASS limits live exl3_experts tf_exl3_prompt_rotate: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +FAIL limits live exl3_experts tf_exl3_prompt_gu: 256 threads launched, the driver allows 256; 128 registers, 43264 + 0 shared bytes (recorded 127, 43264) +PASS limits live exl3_experts tf_exl3_prompt_down: 256 threads launched, the driver allows 256; 127 registers, 43264 + 0 shared bytes (recorded 127, 43264) +PASS limits live exl3_candidate tf_exl3_tc_0: 256 threads launched, the driver allows 256; 128 registers, 8196 + 0 shared bytes (recorded 128, 8196) +PASS limits live exl3_candidate tf_exl3_tc_1: 256 threads launched, the driver allows 256; 128 registers, 8196 + 0 shared bytes (recorded 128, 8196) +PASS limits live exl3_candidate tf_exl3_tc_2: 256 threads launched, the driver allows 256; 128 registers, 8196 + 0 shared bytes (recorded 128, 8196) +PASS limits live exl3_candidate tf_exl3_prompt_0: 256 threads launched, the driver allows 256; 75 registers, 32768 + 0 shared bytes (recorded 75, 32768) +PASS limits live exl3_candidate tf_exl3_prompt_1: 256 threads launched, the driver allows 256; 77 registers, 32768 + 0 shared bytes (recorded 77, 32768) +FAIL limits live exl3_candidate tf_exl3_prompt_2: 256 threads launched, the driver allows 256; 79 registers, 32768 + 0 shared bytes (recorded 80, 32768) +PASS limits live exl3_candidate tf_exl3_rotate_f: 32 threads launched, the driver allows 32; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live exl3_candidate tf_exl3_rotate_h: 32 threads launched, the driver allows 32; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live exl3_candidate tf_exl3_codebook: 256 threads launched, the driver allows 1024; 8 registers, 0 + 0 shared bytes (recorded 8, 0) +PASS limits live exl3_candidate tf_exl3_unpack: 32 threads launched, the driver allows 32; 21 registers, 0 + 0 shared bytes (recorded 21, 0) +PASS limits live experts3 decode pair: 128 threads launched, the driver allows 128; 255 registers, 0 + 0 shared bytes (recorded 255, 0) +PASS limits live experts3 decode down: 128 threads launched, the driver allows 128; 194 registers, 0 + 0 shared bytes (recorded 194, 0) +PASS limits live experts3_prefill pair: 256 threads launched, the driver allows 256; 150 registers, 0 + 46080 shared bytes (recorded 150, 0) +PASS limits live experts3_prefill down: 256 threads launched, the driver allows 256; 98 registers, 0 + 47104 shared bytes (recorded 98, 0) +PASS limits live experts3 tf_experts3_widen: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live experts3 tf_experts3_planes: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live h3 h3_norm: 256 threads launched, the driver allows 256; 32 registers, 128 + 0 shared bytes (recorded 32, 128) +PASS limits live h3 h3_gate_add: 256 threads launched, the driver allows 256; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live h3 h3_gate_mix: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live h3 h3_swiglu: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live h3 h3_heads: 256 threads launched, the driver allows 256; 255 registers, 0 + 0 shared bytes (recorded 255, 0) +PASS limits live h3 h3_pool: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live h3 h3_tile_scores: 256 threads launched, the driver allows 256; 21 registers, 0 + 0 shared bytes (recorded 21, 0) +PASS limits live h3 h3_coarse: 32 threads launched, the driver allows 1024; 39 registers, 0 + 0 shared bytes (recorded 39, 0) +PASS limits live h3 h3_attention: 128 threads launched, the driver allows 128; 208 registers, 0 + 49152 shared bytes (recorded 208, 0) +PASS limits live h3 h3_topk: 32 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live drama3 d3_gemv: 256 threads launched, the driver allows 256; 64 registers, 0 + 12288 shared bytes (recorded 64, 0) +PASS limits live drama3 d3_rmsnorm: 256 threads launched, the driver allows 1024; 18 registers, 144 + 0 shared bytes (recorded 18, 144) +PASS limits live drama3 d3_add_rmsnorm: 256 threads launched, the driver allows 1024; 24 registers, 144 + 0 shared bytes (recorded 24, 144) +PASS limits live drama3 d3_embed_backbone: 256 threads launched, the driver allows 1024; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live drama3 d3_conv_silu: 128 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live drama3 d3_gdn: 32 threads launched, the driver allows 32; 255 registers, 1024 + 0 shared bytes (recorded 255, 1024) +PASS limits live drama3 d3_gated_norm: 256 threads launched, the driver allows 1024; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live drama3 d3_qk_rope: 256 threads launched, the driver allows 1024; 26 registers, 1168 + 0 shared bytes (recorded 26, 1168) +PASS limits live drama3 d3_attn_decode: 256 threads launched, the driver allows 256; 40 registers, 8256 + 0 shared bytes (recorded 40, 8256) +PASS limits live drama3 d3_attn_partial: 256 threads launched, the driver allows 256; 40 registers, 8256 + 0 shared bytes (recorded 40, 8256) +PASS limits live drama3 d3_attn_merge: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live drama3 d3_q_group: 256 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live drama3 d3_causal_softmax: 256 threads launched, the driver allows 1024; 20 registers, 272 + 0 shared bytes (recorded 20, 272) +PASS limits live drama3 d3_attn_gate: 256 threads launched, the driver allows 1024; 19 registers, 0 + 0 shared bytes (recorded 19, 0) +PASS limits live drama3 d3_bf16_f32: 256 threads launched, the driver allows 1024; 16 registers, 0 + 0 shared bytes (recorded 16, 0) +PASS limits live drama3 d3_router: 256 threads launched, the driver allows 1024; 26 registers, 592 + 0 shared bytes (recorded 26, 592) +PASS limits live drama3 d3_swiglu: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live drama3 d3_moe_combine: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +FAIL limits live drama3 d3_act_quant: 128 threads launched, the driver allows 1024; 22 registers, 144 + 0 shared bytes (recorded 21, 144) +PASS limits live drama3 d3_tower_mix: 256 threads launched, the driver allows 1024; 36 registers, 8336 + 0 shared bytes (recorded 36, 8336) +PASS limits live drama3 d3_sample_main: 1024 threads launched, the driver allows 1024; 47 registers, 400 + 65536 shared bytes (recorded 47, 400) +PASS limits live drama3 d3_sample_main_fast: 1024 threads launched, the driver allows 1024; 37 registers, 3616 + 0 shared bytes (recorded 37, 3616) +PASS limits live drama3 d3_sample_code: 1024 threads launched, the driver allows 1024; 39 registers, 400 + 32768 shared bytes (recorded 39, 400) +PASS limits live drama3 d3_sample_code_fast: 1024 threads launched, the driver allows 1024; 25 registers, 3616 + 0 shared bytes (recorded 25, 3616) +PASS limits live drama3 d3_argmax_main: 1024 threads launched, the driver allows 1024; 13 registers, 256 + 0 shared bytes (recorded 13, 256) +PASS limits live drama3 d3_finish_frame: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live drama3 d3_next_input: 32 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live drama3 d3_copy_i32: 64 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live drama3_codec c_vq_gather: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live drama3_codec c_vq_encode: 256 threads launched, the driver allows 1024; 39 registers, 424 + 0 shared bytes (recorded 39, 424) +PASS limits live drama3_codec c_rmsnorm: 256 threads launched, the driver allows 1024; 18 registers, 132 + 0 shared bytes (recorded 18, 132) +FAIL limits live drama3_codec c_layernorm: 256 threads launched, the driver allows 1024; 25 registers, 132 + 0 shared bytes (recorded 24, 132) +PASS limits live drama3_codec c_rope_kv: 512 threads launched, the driver allows 1024; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live drama3_codec c_attn: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live drama3_codec c_add_gamma: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live drama3_codec c_silu_mul: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live drama3_codec c_gelu: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live drama3_codec c_bias: 256 threads launched, the driver allows 1024; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live drama3_codec c_snake: 256 threads launched, the driver allows 1024; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live drama3_codec c_dwconv7: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live drama3_codec c_copy: 256 threads launched, the driver allows 1024; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live drama3_fp8 decode gate-up: 128 threads launched, the driver allows 128; 163 registers, 0 + 0 shared bytes (recorded 163, 0) +PASS limits live drama3_fp8 decode down: 128 threads launched, the driver allows 128; 118 registers, 0 + 0 shared bytes (recorded 118, 0) +PASS limits live drama3_fp8 prompt gate-up: 128 threads launched, the driver allows 128; 212 registers, 32768 + 0 shared bytes (recorded 212, 32768) +PASS limits live drama3_fp8 prompt down: 128 threads launched, the driver allows 128; 213 registers, 32768 + 0 shared bytes (recorded 213, 32768) +FAIL limits live router tf_nemo_router_tiles: 256 threads launched, the driver allows 256; 183 registers, 0 + 40960 shared bytes (recorded 177, 0) +PASS limits live latent tf_super_combine_sums: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live latent tf_super_moe_out_sums: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +FAIL limits live ck tf_ck4_up_nvfp4_prompt: 256 threads launched, the driver allows 256; 115 registers, 46080 + 0 shared bytes (recorded 114, 46080) +PASS limits live ck tf_ck4_down_f32_prompt: 256 threads launched, the driver allows 256; 126 registers, 36864 + 0 shared bytes (recorded 126, 36864) +PASS limits live ck tf_ck4_down_sum_prompt: 256 threads launched, the driver allows 256; 224 registers, 36864 + 0 shared bytes (recorded 224, 36864) +PASS limits live ck tf_ck8_128: 256 threads launched, the driver allows 256; 76 registers, 0 + 49152 shared bytes (recorded 76, 0) +PASS limits live ck tf_ck8_128c cluster of 8: 256 threads launched, the driver allows 256; 104 registers, 0 + 65536 shared bytes (recorded 104, 0) +PASS limits live glm_mm16_16: 128 threads launched, the driver allows 128; 56 registers, 0 + 31104 shared bytes (recorded 56, 0) +PASS limits live glm_mm16_32: 128 threads launched, the driver allows 128; 64 registers, 0 + 37248 shared bytes (recorded 64, 0) +PASS limits live glm_mm16_64: 256 threads launched, the driver allows 256; 64 registers, 0 + 49536 shared bytes (recorded 64, 0) +PASS limits live glm_mm16_128: 256 threads launched, the driver allows 256; 100 registers, 0 + 74112 shared bytes (recorded 100, 0) +PASS limits live glm_mm_tile16: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live glm_mm4_16: 128 threads launched, the driver allows 128; 50 registers, 0 + 12672 shared bytes (recorded 50, 0) +PASS limits live glm_mm4_32: 128 threads launched, the driver allows 128; 65 registers, 0 + 18816 shared bytes (recorded 65, 0) +PASS limits live glm_mm4_64: 256 threads launched, the driver allows 256; 61 registers, 0 + 31104 shared bytes (recorded 61, 0) +PASS limits live glm_mm4_128: 256 threads launched, the driver allows 256; 106 registers, 0 + 55680 shared bytes (recorded 106, 0) +PASS limits live glm_mm_tile4: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live glm_mm6_16: 128 threads launched, the driver allows 128; 54 registers, 0 + 15744 shared bytes (recorded 54, 0) +PASS limits live glm_mm6_32: 128 threads launched, the driver allows 128; 70 registers, 0 + 21888 shared bytes (recorded 70, 0) +PASS limits live glm_mm6_64: 256 threads launched, the driver allows 256; 64 registers, 0 + 34176 shared bytes (recorded 64, 0) +PASS limits live glm_mm6_128: 256 threads launched, the driver allows 256; 106 registers, 0 + 58752 shared bytes (recorded 106, 0) +PASS limits live glm_mm_tile6: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live glm_mm8_16: 128 threads launched, the driver allows 128; 58 registers, 0 + 18816 shared bytes (recorded 58, 0) +PASS limits live glm_mm8_32: 128 threads launched, the driver allows 128; 74 registers, 0 + 24960 shared bytes (recorded 74, 0) +PASS limits live glm_mm8_64: 256 threads launched, the driver allows 256; 63 registers, 0 + 37248 shared bytes (recorded 63, 0) +PASS limits live glm_mm8_128: 256 threads launched, the driver allows 256; 106 registers, 0 + 61824 shared bytes (recorded 106, 0) +PASS limits live glm_mm_tile8: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live glm_mm16_rows: 256 threads launched, the driver allows 256; 99 registers, 0 + 74112 shared bytes (recorded 99, 0) +PASS limits live glm_mm_reduce: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live df_tap: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live df_block_qkv: 64 threads launched, the driver allows 1024; 32 registers, 8 + 0 shared bytes (recorded 32, 8) +PASS limits live df_ctx_kv: 64 threads launched, the driver allows 1024; 28 registers, 8 + 0 shared bytes (recorded 28, 8) +PASS limits live df_conv: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live df_swiglu: 256 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live df_attn_part: 256 threads launched, the driver allows 256; 75 registers, 0 + 0 shared bytes (recorded 75, 0) +PASS limits live df_attn_merge: 128 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live df_sum2: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live df_top16_part: 256 threads launched, the driver allows 256; 30 registers, 32864 + 0 shared bytes (recorded 30, 32864) +PASS limits live df_top16_merge: 256 threads launched, the driver allows 256; 28 registers, 8416 + 0 shared bytes (recorded 28, 8416) +PASS limits live df_gsum: 128 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +FAIL limits live tree attention tf_tree_paths: 128 threads launched, the driver allows 1024; 22 registers, 0 + 0 shared bytes (recorded 30, 0) +FAIL limits live tree attention tf_tree_shared: 128 threads launched, the driver allows 128; 213 registers, 0 + 65536 shared bytes (recorded 212, 0) +FAIL limits live tree attention tf_tree_shared_w1: 32 threads launched, the driver allows 32; 168 registers, 0 + 32768 shared bytes (recorded 195, 0) +FAIL limits live tree attention tf_tree_shared_w1s3: 32 threads launched, the driver allows 32; 168 registers, 0 + 49152 shared bytes (recorded 198, 0) +PASS limits live tree attention tf_tree_part: 128 threads launched, the driver allows 128; 228 registers, 0 + 65536 shared bytes (recorded 228, 0) +PASS limits live tree attention tf_tree_part_w1: 32 threads launched, the driver allows 32; 168 registers, 0 + 32768 shared bytes (recorded 168, 0) +FAIL limits live tree attention tf_tree_part_w1s3: 32 threads launched, the driver allows 32; 168 registers, 0 + 49152 shared bytes (recorded 202, 0) +PASS limits live tree attention tf_tree_shared_256: 128 threads launched, the driver allows 128; 255 registers, 0 + 73728 shared bytes (recorded 255, 0) +FAIL limits live tree attention tf_tree_shared_256_w1: 32 threads launched, the driver allows 32; 253 registers, 0 + 40960 shared bytes (recorded 249, 0) +FAIL limits live tree attention tf_tree_shared_256_w1s3: 32 threads launched, the driver allows 32; 252 registers, 0 + 57344 shared bytes (recorded 250, 0) +PASS limits live tree attention tf_tree_part_256: 128 threads launched, the driver allows 128; 255 registers, 0 + 73728 shared bytes (recorded 255, 0) +FAIL limits live tree attention tf_tree_part_256_w1: 32 threads launched, the driver allows 32; 255 registers, 0 + 40960 shared bytes (recorded 253, 0) +PASS limits live tree attention tf_tree_part_256_w1s3: 32 threads launched, the driver allows 32; 251 registers, 0 + 57344 shared bytes (recorded 251, 0) +FAIL limits live tree attention tf_tree_merge: 256 threads launched, the driver allows 256; 110 registers, 0 + 0 shared bytes (recorded 128, 0) +FAIL limits live block attention tf_block_attention (up to 8 query heads a key head): 256 threads launched, the driver allows 256; 236 registers, 0 + 16384 shared bytes (recorded 228, 0) +FAIL limits live window attention tf_window_part: 256 threads launched, the driver allows 256; 54 registers, 45824 + 0 shared bytes (recorded 56, 45824) +FAIL limits live window attention tf_window_merge: 256 threads launched, the driver allows 256; 38 registers, 0 + 0 shared bytes (recorded 48, 0) +FAIL limits live window attention tf_window_decode_tc: 128 threads launched, the driver allows 128; 182 registers, 0 + 65536 shared bytes (recorded 178, 0) +FAIL limits live window attention wattn_kernel<128, 8, 4, 8>: 256 threads launched, the driver allows 256; 218 registers, 0 + 65536 shared bytes (recorded 222, 0) +FAIL limits live window attention wattn_kernel<128, 8, 8, 8>: 256 threads launched, the driver allows 256; 218 registers, 0 + 65536 shared bytes (recorded 222, 0) +PASS limits live moe tf_moe_topk_sigmoid: 256 threads launched, the driver allows 256; 48 registers, 16704 + 0 shared bytes (recorded 48, 16704) +FAIL limits live moe tf_moe_topk_sigmoid_warp: 256 threads launched, the driver allows 256; 63 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live moe tf_moe_combine: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dense rows tf_dense_rows_1: 256 threads launched, the driver allows 256; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +FAIL limits live dense rows tf_dense_rows_4: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 45, 0) +FAIL limits live dense rows tf_dense_rows_16: 256 threads launched, the driver allows 256; 96 registers, 0 + 0 shared bytes (recorded 98, 0) +PASS limits live dense rows tf_dense_rows_32: 256 threads launched, the driver allows 256; 180 registers, 0 + 0 shared bytes (recorded 180, 0) +PASS limits live act_rows tf_geglu_rows 256 threads: 256 threads launched, the driver allows 256; 33 registers, 0 + 0 shared bytes (recorded 33, 0) +PASS limits live act_rows tf_geglu_rows 128 threads: 128 threads launched, the driver allows 256; 33 registers, 0 + 0 shared bytes (recorded 33, 0) +PASS limits live act_rows tf_swiglu_rows_round 256 threads: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live act_rows tf_swiglu_rows_round 128 threads: 128 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live embed_rows tf_embed_rows_affine: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qk_rows tf_qk_rows_once 512 threads: 512 threads launched, the driver allows 512; 34 registers, 64 + 0 shared bytes (recorded 34, 64) +PASS limits live qk_rows tf_qk_rows_round 512 threads: 512 threads launched, the driver allows 512; 34 registers, 64 + 0 shared bytes (recorded 34, 64) +PASS limits live norms tf_add_rms_rows_bf16h_tail: 256 threads launched, the driver allows 256; 147 registers, 64 + 0 shared bytes (recorded 147, 64) +PASS limits live norms tf_add_rms_rows_bf16h_tail_round: 256 threads launched, the driver allows 256; 147 registers, 64 + 0 shared bytes (recorded 147, 64) +PASS limits live flashnext_fact factored prompt matmul 64x128: 256 threads launched, the driver allows 256; 128 registers, 0 + 41472 shared bytes (recorded 128, 0) +PASS limits live flashnext_fact factored prompt matmul 64x128 folded: 256 threads launched, the driver allows 256; 128 registers, 0 + 41472 shared bytes (recorded 128, 0) +PASS limits live flashnext_hcf hyper-connection down 32x192 folded: 256 threads launched, the driver allows 256; 126 registers, 0 + 49664 shared bytes (recorded 126, 0) +PASS limits live flashnext_hcf hyper-connection up 64x128: 256 threads launched, the driver allows 256; 128 registers, 0 + 43264 shared bytes (recorded 128, 0) +PASS limits live flashnext_fexp factored experts gate and up: 256 threads launched, the driver allows 256; 116 registers, 0 + 41472 shared bytes (recorded 116, 0) +PASS limits live flashnext_fexp factored experts down: 256 threads launched, the driver allows 256; 120 registers, 0 + 41472 shared bytes (recorded 120, 0) +PASS limits live flashnext_batch fn_gdn_chain_tpr: 256 threads launched, the driver allows 256; 152 registers, 0 + 49664 shared bytes (recorded 152, 0) +PASS limits live flashnext_attn qsa_tile scores: 256 threads launched, the driver allows 256; 224 registers, 16384 + 0 shared bytes (recorded 224, 16384) +PASS limits live flashnext_attn fn_attn_fused: 128 threads launched, the driver allows 128; 122 registers, 640 + 26624 shared bytes (recorded 122, 640) +PASS limits live flashnext_attn fn_attn_chunks_one: 128 threads launched, the driver allows 128; 96 registers, 640 + 26624 shared bytes (recorded 96, 640) +PASS limits live flashnext_hc fn_hc_rinv: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live flashnext_hc fn_hc_mix_h: 256 threads launched, the driver allows 256; 38 registers, 1024 + 0 shared bytes (recorded 38, 1024) +PASS limits live route_rows tf_route_logits_wide: 256 threads launched, the driver allows 256; 58 registers, 0 + 55296 shared bytes (recorded 58, 0) +PASS limits live gdn_train tf_gdn_front threads 128: 128 threads launched, the driver allows 128; 40 registers, 1024 + 0 shared bytes (recorded 40, 1024) +PASS limits live gdn_train tf_gdn_scan_fwd threads 256: 256 threads launched, the driver allows 256; 60 registers, 1152 + 0 shared bytes (recorded 60, 1152) +PASS limits live gdn_train tf_gdn_norm_back threads 128: 128 threads launched, the driver allows 128; 24 registers, 644 + 0 shared bytes (recorded 24, 644) +FAIL limits live gdn_train tf_gdn_scan_back threads 256: 256 threads launched, the driver allows 256; 102 registers, 9536 + 0 shared bytes (recorded 100, 9536) +PASS limits live gdn_train tf_gdn_qk_back threads 128: 128 threads launched, the driver allows 128; 40 registers, 128 + 0 shared bytes (recorded 40, 128) +PASS limits live gdn_train tf_gdn_conv_back threads 256: 256 threads launched, the driver allows 1024; 35 registers, 0 + 0 shared bytes (recorded 35, 0) +PASS limits live gdn_train tf_gdn_dequant threads 256: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 32: 32 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 64: 64 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 96: 96 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 128: 128 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 160: 160 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 192: 192 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 224: 224 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 256: 256 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 288: 288 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 320: 320 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 352: 352 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 384: 384 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 416: 416 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 448: 448 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 480: 480 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 512: 512 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 544: 544 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 576: 576 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 608: 608 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 640: 640 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 672: 672 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 704: 704 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 736: 736 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 768: 768 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 800: 800 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 832: 832 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 864: 864 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 896: 896 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 928: 928 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 960: 960 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 992: 992 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 1024: 1024 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live train_q4 tf_q4_back threads 128: 128 threads launched, the driver allows 128; 60 registers, 10240 + 0 shared bytes (recorded 60, 10240) +PASS limits live train_q4 tf_q4_back_bf threads 128: 128 threads launched, the driver allows 128; 56 registers, 10240 + 0 shared bytes (recorded 56, 10240) +PASS limits live train_q4 tf_q4_sum_parts threads 256: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_q4 tf_q4_experts_mm threads 64: 64 threads launched, the driver allows 64; 79 registers, 10304 + 0 shared bytes (recorded 79, 10304) +PASS limits live train_q4 tf_q4_experts_nt threads 64: 64 threads launched, the driver allows 64; 78 registers, 10816 + 0 shared bytes (recorded 78, 10816) +PASS limits live train_q4 tf_q4_swiglu_back threads 128: 128 threads launched, the driver allows 128; 46 registers, 128 + 0 shared bytes (recorded 46, 128) +PASS limits live train_q4 tf_q4_experts_nt_packed threads 64: 64 threads launched, the driver allows 64; 74 registers, 10816 + 0 shared bytes (recorded 74, 10816) +PASS limits live train_q4 tf_q4_experts_mm_packed threads 64: 64 threads launched, the driver allows 64; 80 registers, 10304 + 0 shared bytes (recorded 80, 10304) +PASS limits live train_q4 tf_q4_route_back threads 256: 256 threads launched, the driver allows 256; 40 registers, 136 + 0 shared bytes (recorded 40, 136) +PASS limits live train_q4 tf_q4_axpy threads 256: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live train_attn tf_gattn_q threads 256: 256 threads launched, the driver allows 256; 48 registers, 18560 + 0 shared bytes (recorded 48, 18560) +PASS limits live train_attn tf_gattn_gate_back threads 256: 256 threads launched, the driver allows 1024; 17 registers, 0 + 0 shared bytes (recorded 17, 0) +PASS limits live train_attn tf_gattn_prep threads 32: 32 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 32: 32 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 32: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 64: 64 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 64: 64 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 64: 64 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 96: 96 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 96: 96 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 96: 96 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 128: 128 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 128: 128 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 128: 128 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 160: 160 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 160: 160 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 160: 160 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 192: 192 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 192: 192 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 192: 192 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 224: 224 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 224: 224 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 224: 224 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 256: 256 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 256: 256 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 256: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train tf_train_softmax threads 1024: 1024 threads launched, the driver allows 1024; 41 registers, 128 + 0 shared bytes (recorded 41, 128) +PASS limits live train tf_train_logits threads 256: 256 threads launched, the driver allows 256; 33 registers, 0 + 0 shared bytes (recorded 33, 0) +PASS limits live train tf_train_sort_pairs threads 256: 256 threads launched, the driver allows 256; 40 registers, 4096 + 0 shared bytes (recorded 40, 4096) +PASS limits live train tf_train_experts_tiles threads 1: 1 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live train tf_train_pairs_out threads 256: 256 threads launched, the driver allows 1024; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live train tf_train_zero threads 256: 256 threads launched, the driver allows 1024; 8 registers, 0 + 0 shared bytes (recorded 8, 0) +PASS limits live train tf_train_narrow threads 256: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live train tf_train_gemm threads 128: 128 threads launched, the driver allows 128; 64 registers, 10240 + 0 shared bytes (recorded 64, 10240) +PASS limits live flashnext_train tf_fn_hc_branch_back threads 256: 256 threads launched, the driver allows 256; 48 registers, 128 + 0 shared bytes (recorded 48, 128) +PASS limits live flashnext_train tf_fn_hc_mix_back threads 256: 256 threads launched, the driver allows 1024; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live flashnext_train tf_fn_hc_act_back threads 256: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live flashnext_train tf_fn_hc_norm_back threads 256: 256 threads launched, the driver allows 256; 38 registers, 128 + 0 shared bytes (recorded 38, 128) +PASS limits live flashnext_train tf_fn_pocket_in threads 128: 128 threads launched, the driver allows 128; 31 registers, 0 + 0 shared bytes (recorded 31, 0) +PASS limits live flashnext_train tf_fn_pocket_in_rows threads 256: 256 threads launched, the driver allows 256; 64 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live flashnext_train tf_fn_pocket_out threads 128: 128 threads launched, the driver allows 128; 64 registers, 512 + 0 shared bytes (recorded 64, 512) +PASS limits live flashnext_train tf_fn_pocket_out_rows threads 128: 128 threads launched, the driver allows 128; 72 registers, 2048 + 0 shared bytes (recorded 72, 2048) +PASS limits live flashnext_train tf_fn_pocket_back threads 256: 256 threads launched, the driver allows 256; 40 registers, 1536 + 0 shared bytes (recorded 40, 1536) +PASS limits live flashnext_train tf_fn_pocket_grad threads 256: 256 threads launched, the driver allows 256; 37 registers, 128 + 0 shared bytes (recorded 37, 128) +PASS limits live flashnext_batch fn_moe_sum threads 256: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live flashnext_hc fn_hc_upmix_up threads 256: 256 threads launched, the driver allows 256; 62 registers, 2048 + 0 shared bytes (recorded 62, 2048) +PASS limits live 27B tf_train_zero: 256 threads launched, the driver allows 1024; 8 registers, 0 + 0 shared bytes (recorded 8, 0) +PASS limits live 27B tf_q27_gemm_q4w_bf: 256 threads launched, the driver allows 256; 128 registers, 37888 + 0 shared bytes (recorded 128, 37888) +PASS limits live 27B tf_q27_gemm_q4w: 256 threads launched, the driver allows 256; 126 registers, 37888 + 0 shared bytes (recorded 126, 37888) +PASS limits live 27B tf_q27_gemm_q4_bf: 128 threads launched, the driver allows 128; 56 registers, 10240 + 0 shared bytes (recorded 56, 10240) +PASS limits live 27B tf_q27_gemm_q4: 128 threads launched, the driver allows 128; 56 registers, 10240 + 0 shared bytes (recorded 56, 10240) +PASS limits live 27B tf_sum_parts: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live 27B tf_q27_fit_step: 256 threads launched, the driver allows 256; 39 registers, 5120 + 0 shared bytes (recorded 39, 5120) +PASS limits live 27B tf_q27_zero_rows: 256 threads launched, the driver allows 1024; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live 27B tf_train_softmax: 1024 threads launched, the driver allows 1024; 41 registers, 128 + 0 shared bytes (recorded 41, 128) +PASS limits live 27B tf_train_rms_back: 256 threads launched, the driver allows 256; 37 registers, 128 + 0 shared bytes (recorded 37, 128) +PASS limits live 27B tf_axpy: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live 27B tf_q27_add_bf16: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live 27B tf_q27_cols: 256 threads launched, the driver allows 1024; 16 registers, 0 + 0 shared bytes (recorded 16, 0) +PASS limits live 27B tf_q27_pocket_rows: 256 threads launched, the driver allows 256; 40 registers, 512 + 0 shared bytes (recorded 40, 512) +PASS limits live 27B tf_q27_pocket_back: 256 threads launched, the driver allows 256; 40 registers, 1024 + 0 shared bytes (recorded 40, 1024) +PASS limits live 27B tf_q27_pocket_grad: 256 threads launched, the driver allows 256; 37 registers, 128 + 0 shared bytes (recorded 37, 128) +PASS limits live 27B tf_q27_swiglu_back: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live 27B tf_attn_prep: 256 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live 27B tf_attn_gate_back: 256 threads launched, the driver allows 1024; 17 registers, 0 + 0 shared bytes (recorded 17, 0) +PASS limits live 27B tf_attn_q: 256 threads launched, the driver allows 256; 48 registers, 18560 + 0 shared bytes (recorded 48, 18560) +PASS limits live 27B tf_train_attn_kv: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live 27B tf_attn_prep_back: 256 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live 27B tf_q27_fit_dots: 256 threads launched, the driver allows 256; 40 registers, 0 + 20480 shared bytes (recorded 40, 0) +PASS limits live 27B tf_q27_fit_dots_tree: 256 threads launched, the driver allows 256; 40 registers, 0 + 20480 shared bytes (recorded 40, 0) +PASS limits live 27B tf_gdn_front: 128 threads launched, the driver allows 128; 40 registers, 1024 + 0 shared bytes (recorded 40, 1024) +PASS limits live 27B tf_gdn_scan_fwd: 256 threads launched, the driver allows 256; 60 registers, 1152 + 0 shared bytes (recorded 60, 1152) +PASS limits live 27B tf_gdn_norm_back: 128 threads launched, the driver allows 128; 24 registers, 644 + 0 shared bytes (recorded 24, 644) +FAIL limits live 27B tf_gdn_scan_back: 256 threads launched, the driver allows 256; 102 registers, 9536 + 0 shared bytes (recorded 100, 9536) +PASS limits live 27B tf_gdn_qk_back: 128 threads launched, the driver allows 128; 40 registers, 128 + 0 shared bytes (recorded 40, 128) +PASS limits live 27B tf_gdn_gates_back: 64 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live 27B tf_gdn_conv_back: 256 threads launched, the driver allows 1024; 35 registers, 0 + 0 shared bytes (recorded 35, 0) +PASS limits live 27B tf_gdn_dequant: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live 27B tf_train_narrow: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live 27B tf_train_gemm: 128 threads launched, the driver allows 128; 64 registers, 10240 + 0 shared bytes (recorded 64, 10240) +PASS limits live paging tf_nemo_kv_write: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live paging tf_nemo_kv_write_segs: 256 threads launched, the driver allows 1024; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live paging tf_nemo_attn_chunk: 256 threads launched, the driver allows 256; 118 registers, 45568 + 0 shared bytes (recorded 118, 45568) +PASS limits live paging tf_nemo_attn_chunk_pf: 256 threads launched, the driver allows 256; 128 registers, 45568 + 0 shared bytes (recorded 128, 45568) +PASS limits live paging tf_nemo_attn_chunk_segs: 256 threads launched, the driver allows 256; 118 registers, 45568 + 0 shared bytes (recorded 118, 45568) +PASS limits live paging pattn128: 256 threads launched, the driver allows 256; 231 registers, 0 + 65536 shared bytes (recorded 231, 0) +PASS limits live tf_moe_router: 256 threads launched, the driver allows 256; 48 registers, 16512 + 0 shared bytes (recorded 48, 16512) +PASS limits live tf_moe_topk: 32 threads launched, the driver allows 32; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live tf_moe_combine: 256 threads launched, the driver allows 256; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +FAIL limits live experts plan_rows: 1024 threads launched, the driver allows 1024; 34 registers, 12672 + 0 shared bytes (recorded 32, 12672) +PASS limits live experts plan_rank_rows: 1024 threads launched, the driver allows 1024; 42 registers, 4096 + 0 shared bytes (recorded 42, 4096) +PASS limits live experts plan_scatter_rows: 256 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live tf_q36m_norm_rope: 32 threads launched, the driver allows 32; 42 registers, 0 + 0 shared bytes (recorded 42, 0) +PASS limits live tf_q36m_attn: 32 threads launched, the driver allows 32; 52 registers, 0 + 0 shared bytes (recorded 52, 0) +PASS limits live tf_q36m_attn_tree: 32 threads launched, the driver allows 32; 57 registers, 0 + 0 shared bytes (recorded 57, 0) +PASS limits live qwen35 tf_q35_gdn_slots_1: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen35 tf_q35_gdn_slots_2: 128 threads launched, the driver allows 128; 48 registers, 0 + 0 shared bytes (recorded 48, 0) +PASS limits live qwen35 tf_q35_gdn_slots_4: 128 threads launched, the driver allows 128; 56 registers, 0 + 0 shared bytes (recorded 56, 0) +PASS limits live qwen35 tf_q35_gdn_slots_8: 128 threads launched, the driver allows 128; 72 registers, 0 + 0 shared bytes (recorded 72, 0) +PASS limits live qwen35 tf_q35_gdn_slots_16: 128 threads launched, the driver allows 128; 112 registers, 0 + 0 shared bytes (recorded 112, 0) +PASS limits live qwen35 tf_q35_gdn_chain: 128 threads launched, the driver allows 128; 56 registers, 0 + 0 shared bytes (recorded 56, 0) +PASS limits live qwen35 tf_q35_rope: 256 threads launched, the driver allows 256; 34 registers, 16384 + 0 shared bytes (recorded 34, 16384) +PASS limits live qwen35 tf_q35_attn_partial: 1024 threads launched, the driver allows 1024; 39 registers, 0 + 0 shared bytes (recorded 39, 0) +PASS limits live qwen35 tf_q35_attn_merge: 32 threads launched, the driver allows 32; 42 registers, 0 + 0 shared bytes (recorded 42, 0) +PASS limits live qwen35_kv8 tf_q35_rope8: 256 threads launched, the driver allows 256; 41 registers, 16416 + 0 shared bytes (recorded 41, 16416) +PASS limits live qwen35_kv8 tf_q35_attn_partial8: 1024 threads launched, the driver allows 1024; 37 registers, 0 + 0 shared bytes (recorded 37, 0) +PASS limits live qwen35_keep tf_q35k_apply: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen35_keep tf_q35k_compact: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live qwen35_kv8 tf_q35k_compact8: 256 threads launched, the driver allows 256; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live qwen35_draft tf_q35d_gather_rows: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live qwen35 tf_q35_gdn_pre: 32 threads launched, the driver allows 32; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen35 tf_q35_conv_commit: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live qwen35 tf_q35_tap: 256 threads launched, the driver allows 256; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live qwen35_keep tf_q35k_conv: 256 threads launched, the driver allows 1024; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live qwen35_draft tf_q35d_dyn_conv_batch: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live qwen35_draft tf_q35d_norm_rope_batch: 32 threads launched, the driver allows 32; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live qwen35_draft tf_q35d_mask_rows: 256 threads launched, the driver allows 1024; 16 registers, 0 + 0 shared bytes (recorded 16, 0) +PASS limits live qwen35_draft tf_q35d_attn_partial: 32 threads launched, the driver allows 32; 50 registers, 0 + 0 shared bytes (recorded 50, 0) +PASS limits live qwen35_draft tf_q35d_attn_merge: 32 threads launched, the driver allows 32; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live tf_topk_slices: 256 threads launched, the driver allows 256; 28 registers, 16448 + 0 shared bytes (recorded 28, 16448) +FAIL limits live tf_topk_merge: 256 threads launched, the driver allows 256; 38 registers, 64 + 0 shared bytes (recorded 36, 64) +PASS limits live tf_q35_norm: 256 threads launched, the driver allows 256; 48 registers, 16416 + 0 shared bytes (recorded 48, 16416) +PASS limits live tf_q35_sums: 64 threads launched, the driver allows 64; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live tf_q35_argmax_slices: 256 threads launched, the driver allows 256; 27 registers, 64 + 0 shared bytes (recorded 27, 64) +PASS limits live tf_q35_argmax_merge: 32 threads launched, the driver allows 32; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live sample tf_draw: 1024 threads launched, the driver allows 1024; 64 registers, 9752 + 0 shared bytes (recorded 64, 9752) +PASS limits live sample tf_logprob_rows: 1024 threads launched, the driver allows 1024; 40 registers, 520 + 0 shared bytes (recorded 40, 520) +PASS limits live sample tf_grammar_mask: 256 threads launched, the driver allows 256; 17 registers, 0 + 0 shared bytes (recorded 17, 0) +PASS limits live glm_glue tf_glm_top1: 1024 threads launched, the driver allows 1024; 36 registers, 8192 + 0 shared bytes (recorded 36, 8192) +FAIL limits live glm_glue tf_glm_lat_write_fp8: 512 threads launched, the driver allows 512; 25 registers, 64 + 0 shared bytes (recorded 0, 0) +FAIL limits live glm_glue tf_glm_chunks_16_fp8: 256 threads launched, the driver allows 256; 64 registers, 0 + 83328 shared bytes (recorded 0, 0) +FAIL limits live glm_glue tf_glm_chunks_32_fp8: 256 threads launched, the driver allows 256; 130 registers, 0 + 101120 shared bytes (recorded 0, 0) +PASS limits live sample tf_draw_pitched: 1024 threads launched, the driver allows 1024; 64 registers, 9752 + 0 shared bytes (recorded 64, 9752) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 121 registers, 32832 + 0 shared bytes (recorded 121, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 72 registers, 32832 + 0 shared bytes (recorded 72, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 167 registers, 32832 + 0 shared bytes (recorded 167, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 72 registers, 32832 + 0 shared bytes (recorded 72, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 168 registers, 32832 + 0 shared bytes (recorded 168, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 72 registers, 32832 + 0 shared bytes (recorded 72, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 142 registers, 32832 + 0 shared bytes (recorded 142, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 80 registers, 32832 + 0 shared bytes (recorded 80, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 164 registers, 32832 + 0 shared bytes (recorded 164, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 99 registers, 32832 + 0 shared bytes (recorded 99, 32832) +FAIL limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 166 registers, 32832 + 0 shared bytes (recorded 167, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 99 registers, 32832 + 0 shared bytes (recorded 99, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 96 registers, 16448 + 0 shared bytes (recorded 96, 16448) +FAIL limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 124 registers, 16448 + 0 shared bytes (recorded 128, 16448) +FAIL limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 126 registers, 16448 + 0 shared bytes (recorded 128, 16448) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 186 registers, 16640 + 0 shared bytes (recorded 186, 16640) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 238 registers, 16640 + 0 shared bytes (recorded 238, 16640) +FAIL limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 240 registers, 16640 + 0 shared bytes (recorded 241, 16640) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 185 registers, 32896 + 0 shared bytes (recorded 185, 32896) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 250 registers, 32896 + 0 shared bytes (recorded 250, 32896) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 250 registers, 32896 + 0 shared bytes (recorded 250, 32896) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8: 128 threads launched, the driver allows 128; 125 registers, 32848 + 0 shared bytes (recorded 125, 32848) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10: 128 threads launched, the driver allows 128; 167 registers, 32848 + 0 shared bytes (recorded 168, 32848) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12: 128 threads launched, the driver allows 128; 167 registers, 32848 + 0 shared bytes (recorded 167, 32848) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8: 128 threads launched, the driver allows 128; 146 registers, 32848 + 0 shared bytes (recorded 146, 32848) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10: 128 threads launched, the driver allows 128; 163 registers, 32848 + 0 shared bytes (recorded 168, 32848) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12: 128 threads launched, the driver allows 128; 168 registers, 32848 + 0 shared bytes (recorded 166, 32848) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8: 128 threads launched, the driver allows 128; 121 registers, 32896 + 0 shared bytes (recorded 121, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 167, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 167, 32896) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8: 128 threads launched, the driver allows 128; 140 registers, 32896 + 0 shared bytes (recorded 140, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 164, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 164, 32896) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8: 128 threads launched, the driver allows 128; 121 registers, 32896 + 0 shared bytes (recorded 121, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 167, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 167, 32896) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8: 128 threads launched, the driver allows 128; 140 registers, 32896 + 0 shared bytes (recorded 140, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 164, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 164, 32896) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 118 registers, 32832 + 0 shared bytes (recorded 118, 32832) +FAIL limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 167 registers, 32832 + 0 shared bytes (recorded 166, 32832) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 168 registers, 32832 + 0 shared bytes (recorded 168, 32832) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 134 registers, 32832 + 0 shared bytes (recorded 134, 32832) +FAIL limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 168 registers, 32832 + 0 shared bytes (recorded 167, 32832) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 168 registers, 32832 + 0 shared bytes (recorded 168, 32832) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 89 registers, 16448 + 0 shared bytes (recorded 89, 16448) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 96 registers, 16448 + 0 shared bytes (recorded 96, 16448) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 128 registers, 16448 + 0 shared bytes (recorded 128, 16448) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 23 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 23 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii: 1024 threads launched, the driver allows 1024; 42 registers, 128 + 49152 shared bytes (recorded 42, 128) +PASS limits live dsv41_router_glue router_group_rot: 1024 threads launched, the driver allows 1024; 25 registers, 6528 + 0 shared bytes (recorded 25, 6528) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii: 1024 threads launched, the driver allows 1024; 40 registers, 16384 + 0 shared bytes (recorded 40, 16384) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii: 1024 threads launched, the driver allows 1024; 40 registers, 16384 + 0 shared bytes (recorded 40, 16384) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 38912 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 58368 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 33792 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 33792 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 162 registers, 272 + 58368 shared bytes (recorded 162, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 38912 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 71680 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 238 registers, 528 + 53760 shared bytes (recorded 238, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 272 + 58368 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 161 registers, 272 + 38912 shared bytes (recorded 161, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 124 registers, 272 + 38912 shared bytes (recorded 124, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 166 registers, 528 + 71680 shared bytes (recorded 166, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 166 registers, 528 + 53760 shared bytes (recorded 166, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 243 registers, 528 + 67584 shared bytes (recorded 243, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 26 registers, 48 + 0 shared bytes (recorded 26, 48) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 36864 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 33792 shared bytes (recorded 128, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 40960 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 61440 shared bytes (recorded 128, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 33792 shared bytes (recorded 238, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 36864 shared bytes (recorded 128, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 162 registers, 272 + 61440 shared bytes (recorded 161, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 40960 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 127 registers, 528 + 73728 shared bytes (recorded 127, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 55296 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 272 + 61440 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 159 registers, 272 + 40960 shared bytes (recorded 159, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 124 registers, 272 + 40960 shared bytes (recorded 124, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 164 registers, 528 + 73728 shared bytes (recorded 164, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 164 registers, 528 + 55296 shared bytes (recorded 164, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 242 registers, 528 + 67584 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 20 registers, 64 + 0 shared bytes (recorded 20, 64) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 39936 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 43008 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 64512 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 33792 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 39936 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 151 registers, 272 + 64512 shared bytes (recorded 151, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 43008 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 75776 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 56832 shared bytes (recorded 240, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 174 registers, 272 + 64512 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 143 registers, 272 + 43008 shared bytes (recorded 143, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 43008 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 528 + 75776 shared bytes (recorded 168, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 167 registers, 528 + 56832 shared bytes (recorded 167, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 242 registers, 528 + 67584 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 26 registers, 80 + 0 shared bytes (recorded 26, 80) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 43008 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 45056 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 67584 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 33792 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 43008 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 151 registers, 272 + 67584 shared bytes (recorded 151, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 45056 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 77824 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 58368 shared bytes (recorded 240, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 174 registers, 272 + 67584 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 143 registers, 272 + 45056 shared bytes (recorded 143, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 45056 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 528 + 77824 shared bytes (recorded 168, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 167 registers, 528 + 58368 shared bytes (recorded 167, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 242 registers, 528 + 67584 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 26 registers, 96 + 0 shared bytes (recorded 26, 96) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 46080 shared bytes (recorded 128, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 47104 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 70656 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 35328 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 46080 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 155 registers, 272 + 70656 shared bytes (recorded 155, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 125 registers, 272 + 47104 shared bytes (recorded 125, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 127 registers, 528 + 79872 shared bytes (recorded 127, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 59904 shared bytes (recorded 242, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 182 registers, 272 + 70656 shared bytes (recorded 178, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 149 registers, 272 + 47104 shared bytes (recorded 149, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 123 registers, 272 + 47104 shared bytes (recorded 123, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 528 + 79872 shared bytes (recorded 168, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 160 registers, 528 + 59904 shared bytes (recorded 160, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 240 registers, 528 + 67584 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 22 registers, 112 + 0 shared bytes (recorded 22, 112) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 49152 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 49152 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 73728 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 36864 shared bytes (recorded 240, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 49152 shared bytes (recorded 128, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 162 registers, 272 + 73728 shared bytes (recorded 162, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 49152 shared bytes (recorded 127, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 126 registers, 528 + 81920 shared bytes (recorded 125, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 238 registers, 528 + 61440 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 272 + 73728 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 149 registers, 272 + 49152 shared bytes (recorded 149, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 125 registers, 272 + 49152 shared bytes (recorded 125, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 164 registers, 528 + 81920 shared bytes (recorded 164, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 167 registers, 528 + 61440 shared bytes (recorded 167, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 242 registers, 528 + 67584 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 20 registers, 128 + 0 shared bytes (recorded 20, 128) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 157 registers, 272 + 55296 shared bytes (recorded 157, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 36864 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 53248 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 79872 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 39936 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 155 registers, 272 + 55296 shared bytes (recorded 155, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 155 registers, 272 + 79872 shared bytes (recorded 155, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 158 registers, 272 + 53248 shared bytes (recorded 156, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 127 registers, 528 + 86016 shared bytes (recorded 127, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 238 registers, 528 + 64512 shared bytes (recorded 238, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 158 registers, 272 + 79872 shared bytes (recorded 156, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 149 registers, 272 + 53248 shared bytes (recorded 149, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 119 registers, 272 + 53248 shared bytes (recorded 119, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 152 registers, 528 + 86016 shared bytes (recorded 152, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 160 registers, 528 + 64512 shared bytes (recorded 160, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 240 registers, 528 + 67584 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 21 registers, 160 + 0 shared bytes (recorded 21, 160) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 49152 shared bytes (recorded 128, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 36864 shared bytes (recorded 128, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 252 registers, 272 + 79872 shared bytes (recorded 248, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 49152 shared bytes (recorded 128, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 253 registers, 272 + 49152 shared bytes (recorded 251, 272) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 86016 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 544 + 53248 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 544 + 79872 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 544 + 55296 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 250 registers, 544 + 49152 shared bytes (recorded 250, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 255 registers, 544 + 49152 shared bytes (recorded 255, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 39936 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_: 512 threads launched, the driver allows 512; 60 registers, 38980 + 0 shared bytes (recorded 60, 38980) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i: 128 threads launched, the driver allows 128; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 127 registers, 16 + 8192 shared bytes (recorded 127, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 115 registers, 16 + 16384 shared bytes (recorded 115, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 122 registers, 16 + 32768 shared bytes (recorded 122, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 161 registers, 16 + 8192 shared bytes (recorded 161, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 161 registers, 16 + 16384 shared bytes (recorded 161, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 153 registers, 16 + 32768 shared bytes (recorded 153, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 126 registers, 16 + 8192 shared bytes (recorded 126, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 117 registers, 16 + 16384 shared bytes (recorded 117, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 126 registers, 16 + 32768 shared bytes (recorded 126, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 162 registers, 16 + 8192 shared bytes (recorded 162, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 162 registers, 16 + 16384 shared bytes (recorded 162, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 154 registers, 16 + 32768 shared bytes (recorded 154, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 165 registers, 16 + 8192 shared bytes (recorded 165, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 165 registers, 16 + 16384 shared bytes (recorded 165, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 154 registers, 16 + 32768 shared bytes (recorded 154, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 166 registers, 16 + 8192 shared bytes (recorded 166, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 228 registers, 16 + 32768 shared bytes (recorded 228, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 156 registers, 16 + 8192 shared bytes (recorded 156, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 154 registers, 16 + 16384 shared bytes (recorded 154, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 167 registers, 16 + 16384 shared bytes (recorded 167, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 220 registers, 16 + 32768 shared bytes (recorded 220, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 167 registers, 16 + 16384 shared bytes (recorded 167, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 218 registers, 16 + 32768 shared bytes (recorded 218, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 23 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 127 registers, 16 + 8192 shared bytes (recorded 127, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 127 registers, 16 + 16384 shared bytes (recorded 127, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 21 registers, 0 + 0 shared bytes (recorded 21, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 126 registers, 16 + 8192 shared bytes (recorded 126, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 122 registers, 16 + 16384 shared bytes (recorded 122, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 127 registers, 16 + 32768 shared bytes (recorded 127, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE: 128 threads launched, the driver allows 128; 38 registers, 32 + 0 shared bytes (recorded 38, 32) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 183 registers, 16 + 16384 shared bytes (recorded 183, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 178 registers, 16 + 32768 shared bytes (recorded 178, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 238 registers, 16 + 16384 shared bytes (recorded 238, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 233 registers, 16 + 32768 shared bytes (recorded 233, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 205 registers, 16 + 16384 shared bytes (recorded 205, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 201 registers, 16 + 32768 shared bytes (recorded 201, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 255 registers, 16 + 16384 shared bytes (recorded 255, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 255 registers, 16 + 32768 shared bytes (recorded 255, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 255 registers, 16 + 16384 shared bytes (recorded 255, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 255 registers, 16 + 32768 shared bytes (recorded 255, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 172 registers, 16 + 16384 shared bytes (recorded 172, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 168 registers, 16 + 32768 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 192 registers, 16 + 32768 shared bytes (recorded 192, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 196 registers, 16 + 32768 shared bytes (recorded 196, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 31 registers, 0 + 0 shared bytes (recorded 31, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 206 registers, 16 + 32768 shared bytes (recorded 206, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 29 registers, 0 + 0 shared bytes (recorded 29, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 212 registers, 16 + 32768 shared bytes (recorded 212, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 37 registers, 0 + 0 shared bytes (recorded 37, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 222 registers, 16 + 32768 shared bytes (recorded 222, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii: 256 threads launched, the driver allows 1024; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 246 registers, 16 + 16384 shared bytes (recorded 246, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 239 registers, 16 + 32768 shared bytes (recorded 239, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 244 registers, 16 + 16384 shared bytes (recorded 244, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 245 registers, 16 + 32768 shared bytes (recorded 245, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 240 registers, 16 + 16384 shared bytes (recorded 240, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 242 registers, 16 + 32768 shared bytes (recorded 242, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 248 registers, 16 + 16384 shared bytes (recorded 248, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 248 registers, 16 + 32768 shared bytes (recorded 248, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 243 registers, 16 + 16384 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 242 registers, 16 + 32768 shared bytes (recorded 242, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 192 registers, 16 + 32768 shared bytes (recorded 192, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 226 registers, 16 + 8192 shared bytes (recorded 226, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 226 registers, 16 + 16384 shared bytes (recorded 226, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 226 registers, 16 + 32768 shared bytes (recorded 226, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 194 registers, 16 + 32768 shared bytes (recorded 194, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 232 registers, 16 + 8192 shared bytes (recorded 232, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 232 registers, 16 + 16384 shared bytes (recorded 232, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 232 registers, 16 + 32768 shared bytes (recorded 232, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 230 registers, 16 + 8192 shared bytes (recorded 230, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 230 registers, 16 + 16384 shared bytes (recorded 230, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 230 registers, 16 + 32768 shared bytes (recorded 230, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 243 registers, 16 + 8192 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 243 registers, 16 + 16384 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 243 registers, 16 + 32768 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 212 registers, 16 + 8192 shared bytes (recorded 212, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 212 registers, 16 + 16384 shared bytes (recorded 212, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 212 registers, 16 + 32768 shared bytes (recorded 212, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 243 registers, 16 + 8192 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 243 registers, 16 + 16384 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 243 registers, 16 + 32768 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 243 registers, 16 + 8192 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 243 registers, 16 + 16384 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 243 registers, 16 + 32768 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 173 registers, 16 + 32768 shared bytes (recorded 173, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 136 registers, 0 + 24576 shared bytes (recorded 136, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 136 registers, 0 + 40960 shared bytes (recorded 136, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 241 registers, 0 + 32768 shared bytes (recorded 241, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 241 registers, 0 + 49152 shared bytes (recorded 241, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 147 registers, 0 + 24576 shared bytes (recorded 147, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 147 registers, 0 + 40960 shared bytes (recorded 147, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 241 registers, 0 + 32768 shared bytes (recorded 241, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 241 registers, 0 + 49152 shared bytes (recorded 241, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 145 registers, 0 + 24576 shared bytes (recorded 145, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 145 registers, 0 + 40960 shared bytes (recorded 145, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 239 registers, 0 + 32768 shared bytes (recorded 239, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 239 registers, 0 + 49152 shared bytes (recorded 239, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 159 registers, 0 + 24576 shared bytes (recorded 159, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 164 registers, 0 + 40960 shared bytes (recorded 164, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 248 registers, 0 + 32768 shared bytes (recorded 248, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 248 registers, 0 + 49152 shared bytes (recorded 248, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 156 registers, 0 + 24576 shared bytes (recorded 156, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 160 registers, 0 + 40960 shared bytes (recorded 160, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 245 registers, 0 + 32768 shared bytes (recorded 245, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 245 registers, 0 + 49152 shared bytes (recorded 245, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 167 registers, 16 + 8192 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 173 registers, 16 + 32768 shared bytes (recorded 173, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 167 registers, 16 + 8192 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 171 registers, 16 + 32768 shared bytes (recorded 171, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 167 registers, 16 + 8192 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 173 registers, 16 + 32768 shared bytes (recorded 173, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 167 registers, 16 + 16384 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 167 registers, 16 + 16384 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 174 registers, 16 + 32768 shared bytes (recorded 174, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 166 registers, 16 + 8192 shared bytes (recorded 166, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 174 registers, 16 + 32768 shared bytes (recorded 174, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 174 registers, 16 + 32768 shared bytes (recorded 174, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 173 registers, 16 + 32768 shared bytes (recorded 173, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 114 registers, 0 + 98816 shared bytes (recorded 117, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 114 registers, 0 + 98816 shared bytes (recorded 117, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 114 registers, 0 + 98816 shared bytes (recorded 117, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 114 registers, 0 + 98816 shared bytes (recorded 117, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 111 registers, 0 + 98816 shared bytes (recorded 113, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 111 registers, 0 + 98816 shared bytes (recorded 113, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 111 registers, 0 + 98816 shared bytes (recorded 113, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 111 registers, 0 + 98816 shared bytes (recorded 113, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 90, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 90, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE: 512 threads launched, the driver allows 512; 55 registers, 3232 + 63488 shared bytes (recorded 54, 3232) +PASS limits live dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE: 512 threads launched, the driver allows 512; 55 registers, 3232 + 63488 shared bytes (recorded 55, 3232) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 113 registers, 20496 + 2048 shared bytes (recorded 113, 20496) +FAIL limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 125 registers, 21392 + 2048 shared bytes (recorded 124, 21392) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 20496 + 2048 shared bytes (recorded 80, 20496) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 62 registers, 21392 + 2048 shared bytes (recorded 62, 21392) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 20496 + 2048 shared bytes (recorded 80, 20496) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 89 registers, 21392 + 2048 shared bytes (recorded 89, 21392) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 96 registers, 20496 + 2048 shared bytes (recorded 96, 20496) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 113 registers, 20656 + 2048 shared bytes (recorded 113, 20656) +FAIL limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 21552 + 2048 shared bytes (recorded 126, 21552) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 20656 + 2048 shared bytes (recorded 80, 20656) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 64 registers, 21552 + 2048 shared bytes (recorded 64, 21552) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 20656 + 2048 shared bytes (recorded 80, 20656) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 91 registers, 21552 + 2048 shared bytes (recorded 91, 21552) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 94 registers, 20656 + 2048 shared bytes (recorded 94, 20656) +PASS limits live dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE: 128 threads launched, the driver allows 128; 22 registers, 18048 + 0 shared bytes (recorded 22, 18048) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 141 registers, 0 + 47872 shared bytes (recorded 141, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 143 registers, 0 + 47872 shared bytes (recorded 143, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 141 registers, 0 + 72448 shared bytes (recorded 141, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 143 registers, 0 + 72448 shared bytes (recorded 143, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 47872 shared bytes (recorded 88, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 72448 shared bytes (recorded 88, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 150 registers, 0 + 47872 shared bytes (recorded 150, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 148 registers, 0 + 72448 shared bytes (recorded 148, 0) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 84 registers, 2052 + 0 shared bytes (recorded 84, 2052) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 84 registers, 2052 + 0 shared bytes (recorded 84, 2052) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 79 registers, 2052 + 0 shared bytes (recorded 79, 2052) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 79 registers, 2052 + 0 shared bytes (recorded 79, 2052) +FAIL limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 89 registers, 10244 + 0 shared bytes (recorded 92, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 100 registers, 20484 + 0 shared bytes (recorded 100, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 100 registers, 40964 + 0 shared bytes (recorded 100, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 101 registers, 40964 + 0 shared bytes (recorded 101, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 80 registers, 4100 + 0 shared bytes (recorded 80, 4100) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 4100 + 0 shared bytes (recorded 80, 4100) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 4100 + 0 shared bytes (recorded 86, 4100) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 86 registers, 4100 + 0 shared bytes (recorded 86, 4100) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 110 registers, 10244 + 0 shared bytes (recorded 110, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 111 registers, 20484 + 0 shared bytes (recorded 111, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 111 registers, 40964 + 0 shared bytes (recorded 111, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 118 registers, 40964 + 0 shared bytes (recorded 118, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 84 registers, 8196 + 0 shared bytes (recorded 84, 8196) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 8196 + 0 shared bytes (recorded 88, 8196) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 80 registers, 8196 + 0 shared bytes (recorded 80, 8196) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 90 registers, 8196 + 0 shared bytes (recorded 90, 8196) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 191 registers, 10244 + 0 shared bytes (recorded 191, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 192 registers, 20484 + 0 shared bytes (recorded 192, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 205 registers, 40964 + 0 shared bytes (recorded 205, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 40964 + 0 shared bytes (recorded 128, 40964) +FAIL limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 104 registers, 16388 + 0 shared bytes (recorded 91, 16388) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 109 registers, 16388 + 0 shared bytes (recorded 109, 16388) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 105 registers, 16388 + 0 shared bytes (recorded 105, 16388) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 116 registers, 16388 + 0 shared bytes (recorded 116, 16388) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 142 registers, 5636 + 0 shared bytes (recorded 142, 5636) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 140 registers, 10244 + 0 shared bytes (recorded 140, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 140 registers, 20484 + 0 shared bytes (recorded 140, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 142 registers, 40964 + 0 shared bytes (recorded 142, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 122 registers, 32772 + 0 shared bytes (recorded 122, 32772) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 199 registers, 32772 + 0 shared bytes (recorded 199, 32772) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 177 registers, 32772 + 0 shared bytes (recorded 177, 32772) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 216 registers, 32772 + 0 shared bytes (recorded 216, 32772) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 210 registers, 5636 + 0 shared bytes (recorded 210, 5636) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 158 registers, 10244 + 0 shared bytes (recorded 158, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 158 registers, 20484 + 0 shared bytes (recorded 158, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 170 registers, 40964 + 0 shared bytes (recorded 170, 40964) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 47104 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 132 registers, 0 + 57344 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 47104 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 40960 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 132 registers, 0 + 40960 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 36864 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 32768 shared bytes (recorded 88, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 0 + 71680 shared bytes (recorded 242, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 26 registers, 10240 + 0 shared bytes (recorded 26, 10240) +FAIL limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 47104 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 134 registers, 0 + 57344 shared bytes (recorded 134, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 47104 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 40960 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 132 registers, 0 + 40960 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 36864 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 32768 shared bytes (recorded 88, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 0 + 71680 shared bytes (recorded 240, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 30 registers, 10240 + 0 shared bytes (recorded 30, 10240) +FAIL limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 48640 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 133 registers, 0 + 59392 shared bytes (recorded 133, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 48640 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 43008 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 43008 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 37888 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 87 registers, 0 + 34816 shared bytes (recorded 87, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 244 registers, 0 + 73216 shared bytes (recorded 244, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 31 registers, 10752 + 0 shared bytes (recorded 31, 10752) +FAIL limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 48640 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 136 registers, 0 + 59392 shared bytes (recorded 136, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 48640 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 132 registers, 0 + 43008 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 133 registers, 0 + 43008 shared bytes (recorded 133, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 37888 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 91 registers, 0 + 34816 shared bytes (recorded 91, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 0 + 73216 shared bytes (recorded 242, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 33 registers, 10752 + 0 shared bytes (recorded 33, 10752) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 50176 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 133 registers, 0 + 61440 shared bytes (recorded 133, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 50176 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 45056 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 45056 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 38912 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 87 registers, 0 + 36864 shared bytes (recorded 87, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 244 registers, 0 + 74752 shared bytes (recorded 244, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 31 registers, 11264 + 0 shared bytes (recorded 31, 11264) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 50176 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 134 registers, 0 + 61440 shared bytes (recorded 134, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 50176 shared bytes (recorded 128, 0) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 132 registers, 0 + 45056 shared bytes (recorded 128, 0) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 133 registers, 0 + 45056 shared bytes (recorded 134, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 38912 shared bytes (recorded 126, 0) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 91 registers, 0 + 36864 shared bytes (recorded 90, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 0 + 74752 shared bytes (recorded 242, 0) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 31 registers, 11264 + 0 shared bytes (recorded 32, 11264) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 132 registers, 0 + 53248 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 132 registers, 0 + 65536 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 53248 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 49152 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 49152 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 40960 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 40960 shared bytes (recorded 88, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 0 + 77824 shared bytes (recorded 240, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 27 registers, 12288 + 0 shared bytes (recorded 27, 12288) +PASS limits live dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi: 256 threads launched, the driver allows 1024; 48 registers, 8 + 0 shared bytes (recorded 48, 8) +PASS limits live dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm: 32 threads launched, the driver allows 32; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij: 256 threads launched, the driver allows 256; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx: 1024 threads launched, the driver allows 1024; 64 registers, 9232 + 0 shared bytes (recorded 64, 9232) +PASS limits live dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx: 256 threads launched, the driver allows 256; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x: 256 threads launched, the driver allows 256; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh: 256 threads launched, the driver allows 256; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE: 256 threads launched, the driver allows 256; 36 registers, 2336 + 0 shared bytes (recorded 36, 2336) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE: 256 threads launched, the driver allows 256; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE: 256 threads launched, the driver allows 256; 24 registers, 32 + 0 shared bytes (recorded 24, 32) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE: 256 threads launched, the driver allows 256; 37 registers, 2336 + 0 shared bytes (recorded 37, 2336) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE: 256 threads launched, the driver allows 256; 35 registers, 0 + 0 shared bytes (recorded 35, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE: 256 threads launched, the driver allows 256; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt: 256 threads launched, the driver allows 256; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt: 256 threads launched, the driver allows 256; 64 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx: 256 threads launched, the driver allows 256; 23 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx: 256 threads launched, the driver allows 256; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_: 256 threads launched, the driver allows 256; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf: 256 threads launched, the driver allows 256; 16 registers, 0 + 0 shared bytes (recorded 16, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_: 256 threads launched, the driver allows 256; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf: 1024 threads launched, the driver allows 1024; 42 registers, 2080 + 0 shared bytes (recorded 42, 2080) +PASS limits live dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd: 256 threads launched, the driver allows 256; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx: 1024 threads launched, the driver allows 1024; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx: 32 threads launched, the driver allows 32; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx: 256 threads launched, the driver allows 256; 60 registers, 24576 + 0 shared bytes (recorded 60, 24576) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi: 1024 threads launched, the driver allows 1024; 48 registers, 9244 + 0 shared bytes (recorded 48, 9244) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf: 256 threads launched, the driver allows 256; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi: 256 threads launched, the driver allows 256; 27 registers, 0 + 0 shared bytes (recorded 27, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx: 256 threads launched, the driver allows 256; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live dsv41_kv_glue kv_norm_store: 128 threads launched, the driver allows 128; 26 registers, 1056 + 0 shared bytes (recorded 26, 1056) +PASS limits live dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd: 1024 threads launched, the driver allows 1024; 43 registers, 264 + 0 shared bytes (recorded 43, 264) +PASS limits live dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf: 256 threads launched, the driver allows 256; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live dsv41_sampling dsv41_sampling_stats_max: 256 threads launched, the driver allows 256; 39 registers, 32 + 0 shared bytes (recorded 39, 32) +PASS limits live dsv41_sampling dsv41_sampling_stats_sum: 256 threads launched, the driver allows 256; 38 registers, 64 + 0 shared bytes (recorded 38, 64) +PASS limits live dsv41_sampling dsv41_sampling_stats_fin: 32 threads launched, the driver allows 32; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +FAIL limits live tree_round tf_round_bf16_argmax: 256 threads launched, the driver allows 256; 23 registers, 3072 + 0 shared bytes (recorded 0, 0) +FAIL limits live tree_round tf_tree_round_match: 64 threads launched, the driver allows 1024; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live tree_round tf_round_keep_rows: 16 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live tree_round tf_round_gather_rows: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live tree_round tf_round_keep_head: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_swiglu_rows: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_swiglu_rows, 16 rows or fewer: 128 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_gate_rows: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_gate_rows, 16 rows or fewer: 64 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_group_sums_rows: 256 threads launched, the driver allows 256; 16 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_dconv_rows: 256 threads launched, the driver allows 256; 37 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_64: 128 threads launched, the driver allows 128; 46 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_64, 16 rows or fewer: 64 threads launched, the driver allows 128; 46 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_128: 128 threads launched, the driver allows 128; 77 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_128, 16 rows or fewer: 64 threads launched, the driver allows 128; 77 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_256: 128 threads launched, the driver allows 128; 145 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_256, 16 rows or fewer: 64 threads launched, the driver allows 128; 145 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live qk_rows tf_qk_rows at dim 64: 64 threads launched, the driver allows 256; 29 registers, 32 + 0 shared bytes (recorded 0, 0) +FAIL limits live qk_rows tf_qk_rows at dim 256: 256 threads launched, the driver allows 256; 29 registers, 32 + 0 shared bytes (recorded 0, 0) +FAIL limits live qk_rows tf_qk_rows_w: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live affine4 verify32 one CTA a tile: 128 threads launched, the driver allows 128; 72 registers, 0 + 32640 shared bytes (recorded 92, 0) +FAIL limits live affine4 verify32 cluster of 8: 128 threads launched, the driver allows 128; 72 registers, 0 + 32640 shared bytes (recorded 92, 0) +FAIL limits live affine4 verify64 one CTA a tile: 128 threads launched, the driver allows 128; 122 registers, 0 + 43008 shared bytes (recorded 124, 0) +FAIL limits live affine4 verify64 cluster of 8: 128 threads launched, the driver allows 128; 122 registers, 0 + 43008 shared bytes (recorded 124, 0) +FAIL limits live affine4 verify128 one CTA a tile: 128 threads launched, the driver allows 128; 252 registers, 0 + 57600 shared bytes (recorded 240, 0) +FAIL limits live affine4 verify128 cluster of 8: 128 threads launched, the driver allows 128; 252 registers, 0 + 57600 shared bytes (recorded 240, 0) +RED limits live control: past its bounds: 512 threads launched, the driver allows 256; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +RED limits live control: heavy at 1,024 threads: 1024 threads launched, the driver allows 384; 166 registers, 0 + 0 shared bytes (recorded 166, 0) +SUMMARY 140 kernels: 140 pass gates 1-4, 0 fail, 0 known findings; 32 controls: 32 red as planted, 0 missed; 1018 limit problems; gates 868503 ms, limits 77074 ms +FAIL kernel-contract: 0 gate failures, 0 controls missed, 1018 limit problems +FAIL kernel-contract: TestFailed +kernel-contract ++- run exe tf-cuda-test failure +error: process exited with code 1 +failed command: ./.zig-cache/o/ace78c2504a4ae7b3863dd35794ef7cc/tf-cuda-test kernel-contract + +Build Summary: 134/136 steps succeeded (1 failed) +kernel-contract transitive failure ++- run exe tf-cuda-test failure diff --git a/docs/receipts/glm-kv-pages-0b094b74/single/glm-paged-check.log b/docs/receipts/glm-kv-pages-0b094b74/single/glm-paged-check.log new file mode 100644 index 0000000000..207e864102 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single/glm-paged-check.log @@ -0,0 +1,9 @@ +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +PASS pages at TP=1: 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after diff --git a/docs/receipts/glm-kv-pages-0b094b74/single/kernel-contract.log b/docs/receipts/glm-kv-pages-0b094b74/single/kernel-contract.log new file mode 100644 index 0000000000..9fd024837e --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single/kernel-contract.log @@ -0,0 +1,3151 @@ +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (prefill_attention.fatbin) w +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (fp8_experts.fatbin) w +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (flashnext_attn.fatbin) w +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (drama3_fp8.fatbin) w +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (glm_mm.fatbin) w +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + + +kernel-contract ++- run exe tf-cuda-test + +- compile exe tf-cuda-test fast native + +- run /usr/local/cuda/bin/nvcc (flashnext_fact.fatbin) w +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + + +PASS nonfinite cuda RMS norm tf_rms_rows_bf16 (16-byte form): 2692 non-finite and 18812 finite outputs match the float64 classes +PASS repeat cuda RMS norm tf_rms_rows_bf16 (16-byte form): five runs byte-identical (91392 bytes), inputs and guards untouched, no overlap +PASS shapes cuda RMS norm tf_rms_rows_bf16 (16-byte form): 2 shapes within the float64 bound (worst 0.995 of the bound), 5 edges refused by the kernel +PASS sums cuda RMS norm tf_rms_rows_bf16 (16-byte form): within its bound at k 7168; fp16 sums would miss 256x, bf16 0.6x (a bf16 output follows the sum of squares as ss^-1/2: a bf16 sum's error is a few output ulps (the fp32-out form proves it)) +PASS nonfinite cuda RMS norm tf_rms_rows (f16 in, fp32 out): 2692 non-finite and 18812 finite outputs match the float64 classes +PASS repeat cuda RMS norm tf_rms_rows (f16 in, fp32 out): five runs byte-identical (182784 bytes), inputs and guards untouched, no overlap +PASS shapes cuda RMS norm tf_rms_rows (f16 in, fp32 out): 2 shapes within the float64 bound (worst 0.186 of the bound), 5 edges refused by the kernel +PASS sums cuda RMS norm tf_rms_rows (f16 in, fp32 out): exact sums at k 9216, where fp16 sums would miss by 1864135x and bf16 by 3137x +FAIL nonfinite control: fmax row max: 1 of 8 outputs differ in class; first at 0: finite where float64 gives nan +CONTROL control: fmax row max: red at nonfinite, as planted +FAIL repeat control: atomic-add row sum: 4 of 4 repeats differ from the first run's bytes, first at byte 0 +CONTROL control: atomic-add row sum: red at repeat, as planted +FAIL repeat control: row sum writing its input: 4 of 4 repeats differ from the first run's bytes, first at byte 0 +CONTROL control: row sum writing its input: red at repeat, as planted +FAIL shapes control: row sum losing a tail input: at 8x1x1 output 0 is 0.000000e0 where float64 gives -5.117188e-1: 2396744.29 of its bound +CONTROL control: row sum losing a tail input: red at shapes, as planted +FAIL sums control: dot summed in half: 79310475.6x over its exact-sum bound at k 4096: not fp32 sums (fp16 would miss 79310476x, bf16 16777216x) +CONTROL control: dot summed in half: red at sums, as planted +PASS nonfinite cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), output y: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), output y: five runs byte-identical (265200 bytes), inputs and guards untouched, no overlap +PASS shapes cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), output y: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), output y: within its bound at k 5376; fp16 sums would miss 51x, bf16 0.7x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), residual h: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), residual h: five runs byte-identical (265200 bytes), inputs and guards untouched, no overlap +PASS shapes cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), residual h: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda norm tail tf_add_rms_rows_bf16h_tail_round (MLX's rows), residual h: within its bound at k 5376; fp16 sums would miss 51x, bf16 0.5x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), output y: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), output y: five runs byte-identical (265200 bytes), inputs and guards untouched, no overlap +PASS shapes cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), output y: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), output y: within its bound at k 5376; fp16 sums would miss 51x, bf16 0.4x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), residual h: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), residual h: five runs byte-identical (265200 bytes), inputs and guards untouched, no overlap +PASS shapes cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), residual h: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda norm tail tf_add_rms_rows_bf16h_tail (rounded once), residual h: within its bound at k 5376; fp16 sums would miss 51x, bf16 0.4x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +FAIL shapes control: MLX tail without the branch's normalized-row rounding: at 17x3840x3840: row 0 h[6] = 4.1015625e-1 outside float64's [4.12109375e-1, 4.12109375e-1] +CONTROL control: MLX tail without the branch's normalized-row rounding: red at shapes, as planted +FAIL shapes control: MLX tail without the branch output's rounding: at 17x3840x3840: row 0 h[2] = -1.484375e0 outside float64's [-1.4921875e0, -1.4921875e0] +CONTROL control: MLX tail without the branch output's rounding: red at shapes, as planted +FAIL shapes control: MLX tail without the sum's rounding before the scalar: at 17x3840x3840: row 0 h[0] = -1.7480469e-1 outside float64's [-1.7578125e-1, -1.7578125e-1] +CONTROL control: MLX tail without the sum's rounding before the scalar: red at shapes, as planted +FAIL shapes control: MLX tail without the output's normalized-row rounding: at 17x3840x3840: row 0 y[10] = 1.0703125e0 outside float64's [1.078125e0, 1.078125e0] +CONTROL control: MLX tail without the output's normalized-row rounding: red at shapes, as planted +PASS nonfinite cuda nemotron router logits tf_nemo_router: 640 non-finite and 5504 finite outputs match the float64 classes +PASS repeat cuda nemotron router logits tf_nemo_router: five runs byte-identical (49152 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron router logits tf_nemo_router: 9 shapes within the float64 bound (worst 0.003 of the bound), 0 edges refused by the kernel +PASS sums cuda nemotron router logits tf_nemo_router: exact sums at k 2688, where fp16 sums would miss by 15316x and bf16 by 107354x +PASS nonfinite cuda nemotron router logits tf_nemo_router_tiles: 640 non-finite and 5504 finite outputs match the float64 classes +PASS repeat cuda nemotron router logits tf_nemo_router_tiles: five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron router logits tf_nemo_router_tiles: 12 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda nemotron router logits tf_nemo_router_tiles: exact sums at k 4096, where fp16 sums would miss by 20826x and bf16 by 177109x +PASS nonfinite cuda nemotron top-k tf_nemo_topk: the stated rule holds: a NaN logit is never picked ahead of a number; ties go to the lower expert; two shared slots follow the top-k +PASS repeat cuda nemotron top-k tf_nemo_topk: five runs byte-identical (1024 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron top-k tf_nemo_topk: 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda nemotron top-k tf_nemo_topk: declares none sums +PASS nonfinite cuda nemotron latent top-k tf_super_topk_few: the stated rule holds: tf_super_topk's bytes, and its host twin's picks: a NaN never ahead of a number, ties to the lower expert, every pick past the numbers expert 0 +PASS repeat cuda nemotron latent top-k tf_super_topk_few: five runs byte-identical (264 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron latent top-k tf_super_topk_few: 7 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda nemotron latent top-k tf_super_topk_few: declares none sums +PASS nonfinite cuda nemotron mamba scan tf_nemo_scan: the stated rule holds: glue_ref.scan's bytes (y, the window's dt, the state after the kept rows), NaN wherever it gives NaN +PASS repeat cuda nemotron mamba scan tf_nemo_scan: five runs byte-identical (4489216 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron mamba scan tf_nemo_scan: 11 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda nemotron mamba scan tf_nemo_scan: declares none sums +PASS nonfinite cuda nemotron latent combine tf_super_combine_sums (bf16 share sums): the stated rule holds: as the kernel it replaces gives them from the fp32 shares: NaN and infinite shares and weights +PASS repeat cuda nemotron latent combine tf_super_combine_sums (bf16 share sums): five runs byte-identical (131072 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron latent combine tf_super_combine_sums (bf16 share sums): 4 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda nemotron latent combine tf_super_combine_sums (bf16 share sums): exact sums at k 22, where fp16 sums would miss by infx and bf16 by 3840x +PASS nonfinite cuda nemotron latent shared add tf_super_moe_out_sums (bf16 share sums): the stated rule holds: as the kernel it replaces gives them from the fp32 shares: NaN and infinite shares and weights +PASS repeat cuda nemotron latent shared add tf_super_moe_out_sums (bf16 share sums): five runs byte-identical (524288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda nemotron latent shared add tf_super_moe_out_sums (bf16 share sums): 6 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda nemotron latent shared add tf_super_moe_out_sums (bf16 share sums): within its bound at k 1; fp16 sums would miss infx, bf16 1.0x (one fp32 add of two bf16 values a output) +PASS nonfinite cuda qmm_group (cluster tile): 41216 non-finite and 41216 finite outputs match the float64 classes +PASS repeat cuda qmm_group (cluster tile): five runs byte-identical (329728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qmm_group (cluster tile): 12 shapes within the float64 bound (worst 0.981 of the bound), 0 edges refused by the kernel +PASS sums cuda qmm_group (cluster tile): exact sums at k 4096, where fp16 sums would miss by 8671x and bf16 by 52173x +PASS nonfinite cuda lane_gemv: 41216 non-finite and 41216 finite outputs match the float64 classes +PASS repeat cuda lane_gemv: five runs byte-identical (329728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda lane_gemv: 12 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda lane_gemv: exact sums at k 4096, where fp16 sums would miss by 11313x and bf16 by 59663x +PASS nonfinite cuda lane_gemv split-K: 41216 non-finite and 41216 finite outputs match the float64 classes +PASS repeat cuda lane_gemv split-K: five runs byte-identical (329728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda lane_gemv split-K: 8 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 4 edges refused by the kernel +PASS sums cuda lane_gemv split-K: exact sums at k 4096, where fp16 sums would miss by 9203x and bf16 by 73828x +PASS nonfinite cuda prompt16 FP4 tile 4: 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda prompt16 FP4 tile 4: five runs byte-identical (65536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda prompt16 FP4 tile 4: 13 shapes within the float64 bound (worst 0.994 of the bound), 0 edges refused by the kernel +PASS sums cuda prompt16 FP4 tile 4: exact sums at k 2560, where fp16 sums would miss by 416x and bf16 by 5985x +PASS nonfinite cuda prompt16 FP4 tile 4, fp32 out: 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda prompt16 FP4 tile 4, fp32 out: five runs byte-identical (131072 bytes), inputs and guards untouched, no overlap +PASS shapes cuda prompt16 FP4 tile 4, fp32 out: 15 shapes within the float64 bound (worst 0.099 of the bound), 0 edges refused by the kernel +PASS sums cuda prompt16 FP4 tile 4, fp32 out: exact sums at k 3072, where fp16 sums would miss by 19387004x and bf16 by 195734179x +PASS nonfinite cuda qmmf FP4 fp32 out (slices in a cluster): 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda qmmf FP4 fp32 out (slices in a cluster): five runs byte-identical (32768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qmmf FP4 fp32 out (slices in a cluster): 18 shapes within the float64 bound (worst 0.034 of the bound), 0 edges refused by the kernel +PASS sums cuda qmmf FP4 fp32 out (slices in a cluster): exact sums at k 6144, where fp16 sums would miss by 49982121x and bf16 by 512104525x +PASS nonfinite cuda qmmf FP4 fp32 out (slices by reduce_kernel): 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda qmmf FP4 fp32 out (slices by reduce_kernel): five runs byte-identical (32768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qmmf FP4 fp32 out (slices by reduce_kernel): 18 shapes within the float64 bound (worst 0.038 of the bound), 0 edges refused by the kernel +PASS sums cuda qmmf FP4 fp32 out (slices by reduce_kernel): exact sums at k 6144, where fp16 sums would miss by 71303165x and bf16 by 609572157x +PASS nonfinite cuda affine4 verify32 one-row bits: 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda affine4 verify32 one-row bits: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify32 one-row bits: 39 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify32 one-row bits: exact sums at k 25600, where fp16 sums would miss by 10555x and bf16 by 83630x +PASS nonfinite cuda affine4 verify32 FP64: 2048 non-finite and 2048 finite outputs match the float64 classes +PASS repeat cuda affine4 verify32 FP64: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify32 FP64: 37 shapes within the float64 bound (worst 0.971 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify32 FP64: exact sums at k 25600, where fp16 sums would miss by 9007x and bf16 by 72060x +PASS nonfinite cuda affine4 verify64 one-row bits: 260 non-finite and 260 finite outputs match the float64 classes +PASS repeat cuda affine4 verify64 one-row bits: five runs byte-identical (4290 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify64 one-row bits: 35 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify64 one-row bits: exact sums at k 25600, where fp16 sums would miss by 8939x and bf16 by 87125x +PASS nonfinite cuda affine4 verify64 FP64: 260 non-finite and 260 finite outputs match the float64 classes +PASS repeat cuda affine4 verify64 FP64: five runs byte-identical (4290 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify64 FP64: 33 shapes within the float64 bound (worst 0.967 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify64 FP64: exact sums at k 25600, where fp16 sums would miss by 10594x and bf16 by 110120x +PASS nonfinite cuda affine4 verify128 one-row bits: 260 non-finite and 260 finite outputs match the float64 classes +PASS repeat cuda affine4 verify128 one-row bits: five runs byte-identical (8320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify128 one-row bits: 39 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify128 one-row bits: exact sums at k 25600, where fp16 sums would miss by 7225x and bf16 by 256x +PASS nonfinite cuda affine4 verify128 FP64: 260 non-finite and 260 finite outputs match the float64 classes +PASS repeat cuda affine4 verify128 FP64: five runs byte-identical (8320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda affine4 verify128 FP64: 37 shapes within the float64 bound (worst 0.976 of the bound), 0 edges refused by the kernel +PASS sums cuda affine4 verify128 FP64: exact sums at k 25600, where fp16 sums would miss by 7657x and bf16 by 52173x +FAIL shapes control: verify32 truncated one-row bits: at 17x65x512 its bytes differ from the original one-row affine4 decode's from byte 2080 +CONTROL control: verify32 truncated one-row bits: red at shapes, as planted +FAIL shapes control: verify32 truncated FP64: at 17x65x512 output 1040 is -nan where float64 gives 2.149148e0: nan of its bound +CONTROL control: verify32 truncated FP64: red at shapes, as planted +FAIL shapes control: verify64 truncated one-row bits: at 33x65x512 its bytes differ from the original one-row affine4 decode's from byte 2080 +CONTROL control: verify64 truncated one-row bits: red at shapes, as planted +FAIL shapes control: verify64 truncated FP64: at 33x65x512 output 1040 is -nan where float64 gives -1.232138e0: nan of its bound +CONTROL control: verify64 truncated FP64: red at shapes, as planted +FAIL shapes control: verify128 truncated one-row bits: at 65x65x512 its bytes differ from the original one-row affine4 decode's from byte 2080 +CONTROL control: verify128 truncated one-row bits: red at shapes, as planted +FAIL shapes control: verify128 truncated FP64: at 65x65x512 output 1040 is -nan where float64 gives 4.566685e0: nan of its bound +CONTROL control: verify128 truncated FP64: red at shapes, as planted +PASS nonfinite cuda MXFP8 lane clustered slices: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda MXFP8 lane clustered slices: five runs byte-identical (140 bytes), inputs and guards untouched, no overlap +PASS shapes cuda MXFP8 lane clustered slices: 19 shapes within the float64 bound (worst 0.992 of the bound), 0 edges refused by the kernel +PASS sums cuda MXFP8 lane clustered slices: exact sums at k 2560, where fp16 sums would miss by 593x and bf16 by 5030x +PASS nonfinite cuda MXFP8 lane explicit slice reduction: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda MXFP8 lane explicit slice reduction: five runs byte-identical (140 bytes), inputs and guards untouched, no overlap +PASS shapes cuda MXFP8 lane explicit slice reduction: 19 shapes within the float64 bound (worst 0.985 of the bound), 0 edges refused by the kernel +PASS sums cuda MXFP8 lane explicit slice reduction: exact sums at k 2560, where fp16 sums would miss by 515x and bf16 by 5059x +PASS nonfinite cuda BF16 fp32 clustered: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda BF16 fp32 clustered: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda BF16 fp32 clustered: 20 shapes within the float64 bound (worst 0.050 of the bound), 0 edges refused by the kernel +PASS sums cuda BF16 fp32 clustered: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 1791630067x +PASS nonfinite cuda BF16 fp32 reduce: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda BF16 fp32 reduce: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda BF16 fp32 reduce: 20 shapes within the float64 bound (worst 0.041 of the bound), 0 edges refused by the kernel +PASS sums cuda BF16 fp32 reduce: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2273872009x +PASS nonfinite cuda FP8 fp32 clustered: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda FP8 fp32 clustered: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda FP8 fp32 clustered: 20 shapes within the float64 bound (worst 0.111 of the bound), 0 edges refused by the kernel +PASS sums cuda FP8 fp32 clustered: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2482112847x +PASS nonfinite cuda FP8 fp32 reduce: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda FP8 fp32 reduce: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda FP8 fp32 reduce: 20 shapes within the float64 bound (worst 0.128 of the bound), 0 edges refused by the kernel +PASS sums cuda FP8 fp32 reduce: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2651940327x +PASS nonfinite cuda block FP8 fp32 clustered: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda block FP8 fp32 clustered: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda block FP8 fp32 clustered: 20 shapes within the float64 bound (worst 0.034 of the bound), 0 edges refused by the kernel +PASS sums cuda block FP8 fp32 clustered: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2332606604x +PASS nonfinite cuda block FP8 fp32 reduce: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda block FP8 fp32 reduce: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda block FP8 fp32 reduce: 20 shapes within the float64 bound (worst 0.042 of the bound), 0 edges refused by the kernel +PASS sums cuda block FP8 fp32 reduce: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2236320382x +PASS nonfinite cuda MXFP8 fp32 clustered: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda MXFP8 fp32 clustered: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda MXFP8 fp32 clustered: 20 shapes within the float64 bound (worst 0.039 of the bound), 0 edges refused by the kernel +PASS sums cuda MXFP8 fp32 clustered: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2097668222x +PASS nonfinite cuda MXFP8 fp32 reduce: 280 non-finite and 70 finite outputs match the float64 classes +PASS repeat cuda MXFP8 fp32 reduce: five runs byte-identical (280 bytes), inputs and guards untouched, no overlap +PASS shapes cuda MXFP8 fp32 reduce: 20 shapes within the float64 bound (worst 0.047 of the bound), 0 edges refused by the kernel +PASS sums cuda MXFP8 fp32 reduce: exact sums at k 10240, where fp16 sums would miss by infx and bf16 by 2505038444x +PASS nonfinite cuda tf_rank_sum (a reduce-scatter's rank-order sum): 4 non-finite and 32764 finite outputs match the float64 classes +PASS repeat cuda tf_rank_sum (a reduce-scatter's rank-order sum): five runs byte-identical (262144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tf_rank_sum (a reduce-scatter's rank-order sum): 7 shapes within the float64 bound (worst 0.550 of the bound), 0 edges refused by the kernel +PASS sums cuda tf_rank_sum (a reduce-scatter's rank-order sum): exact sums at k 5, where fp16 sums would miss by infx and bf16 by 19617358994x +PASS nonfinite cuda Flash Next fn_hc_writeback0: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda Flash Next fn_hc_writeback0: five runs byte-identical (2720 bytes), inputs and guards untouched, no overlap +PASS shapes cuda Flash Next fn_hc_writeback0: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda Flash Next fn_hc_writeback0: exact sums at k 8, where fp16 sums would miss by infx and bf16 by 96849x +PASS nonfinite cuda Flash Next fn_hc_writeback_moe: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda Flash Next fn_hc_writeback_moe: five runs byte-identical (350880 bytes), inputs and guards untouched, no overlap +PASS shapes cuda Flash Next fn_hc_writeback_moe: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda Flash Next fn_hc_writeback_moe: exact sums at k 8, where fp16 sums would miss by infx and bf16 by 88794x +PASS nonfinite cuda Flash Next fn_hc_writeback_slices: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda Flash Next fn_hc_writeback_slices: five runs byte-identical (350880 bytes), inputs and guards untouched, no overlap +PASS shapes cuda Flash Next fn_hc_writeback_slices: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda Flash Next fn_hc_writeback_slices: exact sums at k 8, where fp16 sums would miss by infx and bf16 by 80093x +PASS nonfinite cuda fn_sum_slices f32: the stated rule holds: propagates NaN and infinities; finite outputs use ordered f32 and one final RNE +PASS repeat cuda fn_sum_slices f32: five runs byte-identical (696320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_sum_slices f32: 13 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda fn_sum_slices f32: exact sums at k 8, where fp16 sums would miss by 12582912x and bf16 by 12582912x +PASS nonfinite cuda fn_sum_slices bf16: the stated rule holds: propagates NaN and infinities; finite outputs use ordered f32 and one final RNE +PASS repeat cuda fn_sum_slices bf16: five runs byte-identical (348160 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_sum_slices bf16: 13 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda fn_sum_slices bf16: exact sums at k 8, where fp16 sums would miss by 192x and bf16 by 192x +PASS nonfinite cuda fn_shift_window contiguous: the stated rule holds: copies all bf16 bits, including NaN payloads, infinities and signed zero +PASS repeat cuda fn_shift_window contiguous: five runs byte-identical (184320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_shift_window contiguous: 18 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda fn_shift_window contiguous: declares none sums +PASS nonfinite cuda fn_shift_window strided: the stated rule holds: copies all bf16 bits, including NaN payloads, infinities and signed zero +PASS repeat cuda fn_shift_window strided: five runs byte-identical (184320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_shift_window strided: 18 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda fn_shift_window strided: declares none sums +PASS nonfinite cuda Flash Next fn_row_best: the stated rule holds: NaNs do not win; all NaN/-Inf retain -Inf and INT_MAX; a NaN or infinite maximum makes LSE NaN +PASS repeat cuda Flash Next fn_row_best: five runs byte-identical (192 bytes), inputs and guards untouched, no overlap +PASS shapes cuda Flash Next fn_row_best: 14 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda Flash Next fn_row_best: exact sums at k 248320, where fp16 sums would miss by 2159920x and bf16 by 929x +PASS nonfinite cuda fn_qsa_select packed scores: the stated rule holds: float total order includes signed NaN payloads and signed zero; ties choose earlier blocks +PASS repeat cuda fn_qsa_select packed scores: five runs byte-identical (65728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_qsa_select packed scores: 23 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +N/A sums cuda fn_qsa_select packed scores: declares none sums +PASS nonfinite cuda fn_qsa_select padded scores: the stated rule holds: float total order includes signed NaN payloads and signed zero; ties choose earlier blocks +PASS repeat cuda fn_qsa_select padded scores: five runs byte-identical (65728 bytes), inputs and guards untouched, no overlap +PASS shapes cuda fn_qsa_select padded scores: 23 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +N/A sums cuda fn_qsa_select padded scores: declares none sums +PASS nonfinite cuda h3 modulated norm h3_norm: 5381 non-finite and 80635 finite outputs match the float64 classes +PASS repeat cuda h3 modulated norm h3_norm: five runs byte-identical (344064 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 modulated norm h3_norm: 1 shapes within the float64 bound (worst 0.579 of the bound), 4 edges refused by the kernel +PASS sums cuda h3 modulated norm h3_norm: within its bound at k 5376; fp16 sums would miss 43x, bf16 0.1x (a bf16 output follows the sum of squares as ss^-1/2: a bf16 sum's error is a few output ulps) +PASS nonfinite cuda h3 gated add h3_gate_add: 3 non-finite and 43005 finite outputs match the float64 classes +PASS repeat cuda h3 gated add h3_gate_add: five runs byte-identical (172032 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 gated add h3_gate_add: 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +N/A sums cuda h3 gated add h3_gate_add: declares none sums +PASS nonfinite cuda h3 pooled branch mix h3_gate_mix: 3 non-finite and 2045 finite outputs match the float64 classes +PASS repeat cuda h3 pooled branch mix h3_gate_mix: five runs byte-identical (32768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 pooled branch mix h3_gate_mix: 5 shapes within the float64 bound (worst 0.876 of the bound), 0 edges refused by the kernel +N/A sums cuda h3 pooled branch mix h3_gate_mix: declares none sums +PASS nonfinite cuda h3 SwiGLU h3_swiglu: 3 non-finite and 114685 finite outputs match the float64 classes +PASS repeat cuda h3 SwiGLU h3_swiglu: five runs byte-identical (458752 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 SwiGLU h3_swiglu: 5 shapes within the float64 bound (worst 0.927 of the bound), 0 edges refused by the kernel +N/A sums cuda h3 SwiGLU h3_swiglu: declares none sums +PASS nonfinite cuda h3 head layout h3_heads: 131 non-finite and 786301 finite outputs match the float64 classes +PASS repeat cuda h3 head layout h3_heads: five runs byte-identical (1572864 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 head layout h3_heads: 1 shapes within the float64 bound (worst 0.555 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 head layout h3_heads: within its bound at k 96; fp16 sums would miss 0x, bf16 0.3x (a bf16 output follows the sum of squares as ss^-1/2: a bf16 sum's error is a few output ulps) +PASS nonfinite cuda h3 tile means h3_pool: 1 non-finite and 6143 finite outputs match the float64 classes +PASS repeat cuda h3 tile means h3_pool: five runs byte-identical (49152 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 tile means h3_pool: 1 shapes within the float64 bound (worst 0.009 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 tile means h3_pool: within its bound at k 64; fp16 sums would miss 117x, bf16 865.4x (a mean of at most 64 bf16 values: no sum reaches past fp16's range) +PASS nonfinite cuda h3 tile scores h3_tile_scores: 24 non-finite and 104 finite outputs match the float64 classes +PASS repeat cuda h3 tile scores h3_tile_scores: five runs byte-identical (2048 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 tile scores h3_tile_scores: 5 shapes within the float64 bound (worst 0.131 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 tile scores h3_tile_scores: exact sums at k 128, where fp16 sums would miss by 335544x and bf16 by 11072962x +PASS nonfinite cuda h3 pooled branch h3_coarse: 256 non-finite and 1792 finite outputs match the float64 classes +PASS repeat cuda h3 pooled branch h3_coarse: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 pooled branch h3_coarse: 2 shapes within the float64 bound (worst 0.213 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 pooled branch h3_coarse: within its bound at k 16; fp16 sums would miss 1139x, bf16 8240.4x (softmax weights in [0, 1], rounded to bf16 before the value product: the bf16 output hides the sums' error) +PASS nonfinite cuda h3 routed attention h3_attention: 128 non-finite and 99712 finite outputs match the float64 classes +PASS repeat cuda h3 routed attention h3_attention: five runs byte-identical (432640 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 routed attention h3_attention: 6 shapes within the float64 bound (worst 0.767 of the bound), 0 edges refused by the kernel +PASS sums cuda h3 routed attention h3_attention: within its bound at k 1024; fp16 sums would miss 6x, bf16 61.4x (softmax weights in [0, 1], rounded to bf16 before the value product: the bf16 output hides the sums' error) +PASS nonfinite cuda h3 tile top-k h3_topk: the stated rule holds: a NaN score ranks as -inf, ties go to the lower tile, so every row keeps exactly n tiles +PASS repeat cuda h3 tile top-k h3_topk: five runs byte-identical (143360 bytes), inputs and guards untouched, no overlap +PASS shapes cuda h3 tile top-k h3_topk: 2 shapes exact by its own check (worst 0.000 of the bound), 4 edges refused by the kernel +N/A sums cuda h3 tile top-k h3_topk: declares none sums +PASS nonfinite cuda drama3 d3_rmsnorm (bf16 rows, plain gamma): 2052 non-finite and 8188 finite outputs match the float64 classes +PASS repeat cuda drama3 d3_rmsnorm (bf16 rows, plain gamma): five runs byte-identical (4096 bytes), inputs and guards untouched, no overlap +PASS shapes cuda drama3 d3_rmsnorm (bf16 rows, plain gamma): 2 shapes within the float64 bound (worst 0.995 of the bound), 4 edges refused by the kernel +PASS sums cuda drama3 d3_rmsnorm (bf16 rows, plain gamma): within its bound at k 2048; fp16 sums would miss 256x, bf16 0.0x (a bf16 output follows the sum of squares as ss^-1/2: a bf16 sum's error is a few output ulps) +PASS nonfinite cuda ck experts up tf_ck4_up_relu2_few: the stated rule holds: as the prompt kernel it replaces gives them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts up tf_ck4_up_relu2_few: five runs byte-identical (10752 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts up tf_ck4_up_relu2_few: 15 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts up tf_ck4_up_relu2_few: exact sums at k 4096, where fp16 sums would miss by 128x and bf16 by 128x +PASS nonfinite cuda ck experts down tf_ck4_down_f32_few: the stated rule holds: as the prompt kernel it replaces gives them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts down tf_ck4_down_f32_few: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts down tf_ck4_down_f32_few: 15 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts down tf_ck4_down_f32_few: exact sums at k 2688, where fp16 sums would miss by infx and bf16 by 14620145x +PASS nonfinite cuda ck experts prompt up tf_ck4_up_nvfp4_prompt (NVFP4 rows out): the stated rule holds: as the kernels it replaces give them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts prompt up tf_ck4_up_nvfp4_prompt (NVFP4 rows out): five runs byte-identical (193536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts prompt up tf_ck4_up_nvfp4_prompt (NVFP4 rows out): 16 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts prompt up tf_ck4_up_nvfp4_prompt (NVFP4 rows out): exact sums at k 4096, where fp16 sums would miss by infx and bf16 by infx +PASS nonfinite cuda ck experts prompt down tf_ck4_down_f32_prompt: the stated rule holds: as the kernels it replaces give them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts prompt down tf_ck4_down_f32_prompt: five runs byte-identical (1048576 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts prompt down tf_ck4_down_f32_prompt: 16 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts prompt down tf_ck4_down_f32_prompt: exact sums at k 2688, where fp16 sums would miss by infx and bf16 by 25165823x +PASS nonfinite cuda ck experts prompt down tf_ck4_down_sum_prompt (two shares summed, bf16): the stated rule holds: as the kernels it replaces give them: NaN and infinite alphas, a NaN block scale +PASS repeat cuda ck experts prompt down tf_ck4_down_sum_prompt (two shares summed, bf16): five runs byte-identical (262144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck experts prompt down tf_ck4_down_sum_prompt (two shares summed, bf16): 16 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck experts prompt down tf_ck4_down_sum_prompt (two shares summed, bf16): exact sums at k 2688, where fp16 sums would miss by infx and bf16 by 337x +PASS nonfinite cuda ck FP8 prompt tile tf_ck8_128c (slices in a cluster): the stated rule holds: as the 64-row tile it replaces gives them: a NaN input and a NaN weight code +PASS repeat cuda ck FP8 prompt tile tf_ck8_128c (slices in a cluster): five runs byte-identical (524288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck FP8 prompt tile tf_ck8_128c (slices in a cluster): 10 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck FP8 prompt tile tf_ck8_128c (slices in a cluster): exact sums at k 8192, where fp16 sums would miss by 2116x and bf16 by 13187x +PASS nonfinite cuda ck FP8 prompt tile tf_ck8_128 (slices by tf_ck8_reduce): the stated rule holds: as the 64-row tile it replaces gives them: a NaN input and a NaN weight code +PASS repeat cuda ck FP8 prompt tile tf_ck8_128 (slices by tf_ck8_reduce): five runs byte-identical (524288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda ck FP8 prompt tile tf_ck8_128 (slices by tf_ck8_reduce): 10 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda ck FP8 prompt tile tf_ck8_128 (slices by tf_ck8_reduce): exact sums at k 8192, where fp16 sums would miss by 2523x and bf16 by 19162x +PASS nonfinite cuda tree_round tf_round_bf16_argmax: the stated rule holds: a row holding NaN or an infinity picks no token and flags it (1 NaN, 2 +inf, 4 -inf); ties go to the lower token +PASS repeat cuda tree_round tf_round_bf16_argmax: five runs byte-identical (128 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_round_bf16_argmax: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_round_bf16_argmax: declares none sums +PASS nonfinite cuda tree_round tf_tree_round_match: the stated rule holds: a flagged pick or a malformed tree gives core/tree_round.oracle's invalid result, byte for byte +PASS repeat cuda tree_round tf_tree_round_match: five runs byte-identical (1344 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_tree_round_match: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_tree_round_match: declares none sums +PASS nonfinite cuda tree_round tf_round_keep_rows: the stated rule holds: a tree whose result is not ok moves nothing; kept rows move bit for bit, NaN and infinities included +PASS repeat cuda tree_round tf_round_keep_rows: five runs byte-identical (544 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_round_keep_rows: 2 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_round_keep_rows: declares none sums +PASS nonfinite cuda tree_round tf_round_gather_rows: the stated rule holds: a tree whose result is not ok moves nothing; kept rows move bit for bit, NaN and infinities included +PASS repeat cuda tree_round tf_round_gather_rows: five runs byte-identical (1310720 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_round_gather_rows: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_round_gather_rows: declares none sums +PASS nonfinite cuda tree_round tf_round_keep_head: the stated rule holds: a tree whose result is not ok moves nothing; kept rows move bit for bit, NaN and infinities included +PASS repeat cuda tree_round tf_round_keep_head: five runs byte-identical (3973120 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree_round tf_round_keep_head: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree_round tf_round_keep_head: declares none sums +PASS nonfinite cuda act_rows tf_swiglu_rows: 5 non-finite and 87035 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_swiglu_rows: five runs byte-identical (34816 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_swiglu_rows: 9 shapes within the float64 bound (worst 0.995 of the bound), 0 edges refused by the kernel +N/A sums cuda act_rows tf_swiglu_rows: declares none sums +PASS nonfinite cuda act_rows tf_swiglu_rows group sums (of its stored bf16 outputs): 5 non-finite and 1355 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_swiglu_rows group sums (of its stored bf16 outputs): five runs byte-identical (1088 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_swiglu_rows group sums (of its stored bf16 outputs): 9 shapes within the float64 bound (worst 0.204 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_swiglu_rows group sums (of its stored bf16 outputs): exact sums at k 1, where fp16 sums would miss by 246724x and bf16 by 2960685x +PASS nonfinite cuda act_rows tf_gate_rows: 4 non-finite and 30716 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_gate_rows: five runs byte-identical (12288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_gate_rows: 7 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +N/A sums cuda act_rows tf_gate_rows: declares none sums +PASS nonfinite cuda act_rows tf_gate_rows group sums (of its stored bf16 outputs): 4 non-finite and 476 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_gate_rows group sums (of its stored bf16 outputs): five runs byte-identical (384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_gate_rows group sums (of its stored bf16 outputs): 7 shapes within the float64 bound (worst 0.076 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_gate_rows group sums (of its stored bf16 outputs): exact sums at k 256, where fp16 sums would miss by 123362x and bf16 by 2960685x +PASS nonfinite cuda act_rows tf_group_sums_rows: 3 non-finite and 397 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_group_sums_rows: five runs byte-identical (320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_group_sums_rows: 7 shapes within the float64 bound (worst 0.034 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_group_sums_rows: exact sums at k 17408, where fp16 sums would miss by 2097152x and bf16 by 67108864x +PASS nonfinite cuda act_rows tf_dconv_rows: 9 non-finite and 40951 finite outputs match the float64 classes +PASS repeat cuda act_rows tf_dconv_rows: five runs byte-identical (163840 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_dconv_rows: 9 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_dconv_rows: exact sums at k 16, where fp16 sums would miss by 256x and bf16 by 256x +PASS nonfinite cuda act_rows tf_dconv_rows with residual: the stated rule holds: own check, core/act_ref's bound: the conv is rounded to bf16 before the residual add, an error of the conv's size +PASS repeat cuda act_rows tf_dconv_rows with residual: five runs byte-identical (163840 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act_rows tf_dconv_rows with residual: 9 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda act_rows tf_dconv_rows with residual: exact sums at k 16, where fp16 sums would miss by 205x and bf16 by 205x +PASS nonfinite cuda act tf_geglu_rows (MLX's bf16 steps): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda act tf_geglu_rows (MLX's bf16 steps): five runs byte-identical (538560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act tf_geglu_rows (MLX's bf16 steps): 12 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda act tf_geglu_rows (MLX's bf16 steps): declares none sums +PASS nonfinite cuda act tf_swiglu_rows_round (MLX's bf16 steps): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda act tf_swiglu_rows_round (MLX's bf16 steps): five runs byte-identical (538560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda act tf_swiglu_rows_round (MLX's bf16 steps): 12 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda act tf_swiglu_rows_round (MLX's bf16 steps): declares none sums +FAIL shapes control: GeGLU without x^3's rounding: at 17x15360x1: row 0 out[283] = 4.4189453125e-2 outside float64's [4.39453125e-2, 4.39453125e-2] +CONTROL control: GeGLU without x^3's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without c1 x^3's rounding: at 17x15360x1: row 0 out[14] = 2.546875e0 outside float64's [2.53125e0, 2.53125e0] +CONTROL control: GeGLU without c1 x^3's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without the inner sum's rounding: at 17x15360x1: row 0 out[15] = -3.453125e0 outside float64's [-3.4375e0, -3.4375e0] +CONTROL control: GeGLU without the inner sum's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without c0 inner's rounding: at 17x15360x1: row 0 out[10] = 1.259765625e-1 outside float64's [1.279296875e-1, 1.279296875e-1] +CONTROL control: GeGLU without c0 inner's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without tanh's rounding: at 17x15360x1: row 0 out[0] = 7.8582763671875e-4 outside float64's [0e0, 0e0] +CONTROL control: GeGLU without tanh's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without 1 + tanh's rounding: at 17x15360x1: row 0 out[20] = -7.1875e-1 outside float64's [-7.1484375e-1, -7.1484375e-1] +CONTROL control: GeGLU without 1 + tanh's rounding: red at shapes, as planted +FAIL shapes control: GeGLU without the gelu's rounding: at 17x15360x1: row 0 out[5] = -7.080078125e-2 outside float64's [-7.03125e-2, -7.03125e-2] +CONTROL control: GeGLU without the gelu's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without exp's rounding: at 17x15360x1: row 0 out[11] = -3.92578125e-1 outside float64's [-3.90625e-1, -3.90625e-1] +CONTROL control: SwiGLU steps without exp's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without 1 + e's rounding: at 17x15360x1: row 0 out[12] = -2.5390625e-1 outside float64's [-2.51953125e-1, -2.51953125e-1] +CONTROL control: SwiGLU steps without 1 + e's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without the reciprocal's rounding: at 17x15360x1: row 0 out[0] = -4.40625e0 outside float64's [-4.4375e0, -4.4375e0] +CONTROL control: SwiGLU steps without the reciprocal's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without 1 - the reciprocal's rounding: at 17x15360x1: row 0 out[10] = -1.640625e-1 outside float64's [-1.630859375e-1, -1.630859375e-1] +CONTROL control: SwiGLU steps without 1 - the reciprocal's rounding: red at shapes, as planted +FAIL shapes control: SwiGLU steps without silu's rounding: at 17x15360x1: row 0 out[3] = 3.984375e0 outside float64's [3.96875e0, 3.96875e0] +CONTROL control: SwiGLU steps without silu's rounding: red at shapes, as planted +PASS nonfinite cuda gdn_prep tf_gdn_prep_64 qk: 256 non-finite and 1024 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_64 qk: five runs byte-identical (512 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_64 qk: 7 shapes within the float64 bound (worst 0.687 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_64 qk: within its bound at k 4; fp16 sums would miss infx, bf16 0.2x (these stress inputs do not separate bf16 conv/square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_64 v: 5 non-finite and 1915 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_64 v: five runs byte-identical (768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_64 v: 7 shapes within the float64 bound (worst 0.995 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_64 v: within its bound at k 4; fp16 sums would miss infx, bf16 0.8x (these stress inputs do not separate bf16 tap sums beyond the rounded output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_64 gb: the stated rule holds: own check, the PR's float64 bound: g = exp(-y) carries y times y's relative error, beyond a fixed op count +PASS repeat cuda gdn_prep tf_gdn_prep_64 gb: five runs byte-identical (48 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_64 gb: 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda gdn_prep tf_gdn_prep_64 gb: declares none sums +PASS nonfinite cuda gdn_prep tf_gdn_prep_128 qk: 385 non-finite and 20095 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_128 qk: five runs byte-identical (8192 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_128 qk: 10 shapes within the float64 bound (worst 0.689 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_128 qk: within its bound at k 4; fp16 sums would miss infx, bf16 0.2x (these stress inputs do not separate bf16 conv/square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_128 v: 7 non-finite and 30713 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_128 v: five runs byte-identical (12288 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_128 v: 10 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_128 v: within its bound at k 4; fp16 sums would miss infx, bf16 0.8x (these stress inputs do not separate bf16 tap sums beyond the rounded output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_128 gb: the stated rule holds: own check, the PR's float64 bound: g = exp(-y) carries y times y's relative error, beyond a fixed op count +PASS repeat cuda gdn_prep tf_gdn_prep_128 gb: five runs byte-identical (384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_128 gb: 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda gdn_prep tf_gdn_prep_128 gb: declares none sums +PASS nonfinite cuda gdn_prep tf_gdn_prep_256 qk: 513 non-finite and 2047 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_256 qk: five runs byte-identical (1024 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_256 qk: 5 shapes within the float64 bound (worst 0.644 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_256 qk: within its bound at k 2; fp16 sums would miss infx, bf16 0.3x (these stress inputs do not separate bf16 conv/square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_256 v: 3 non-finite and 3837 finite outputs match the float64 classes +PASS repeat cuda gdn_prep tf_gdn_prep_256 v: five runs byte-identical (1536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_256 v: 5 shapes within the float64 bound (worst 0.995 of the bound), 0 edges refused by the kernel +PASS sums cuda gdn_prep tf_gdn_prep_256 v: within its bound at k 2; fp16 sums would miss infx, bf16 0.5x (these stress inputs do not separate bf16 tap sums beyond the rounded output bound; fp16 overflows) +PASS nonfinite cuda gdn_prep tf_gdn_prep_256 gb: the stated rule holds: own check, the PR's float64 bound: g = exp(-y) carries y times y's relative error, beyond a fixed op count +PASS repeat cuda gdn_prep tf_gdn_prep_256 gb: five runs byte-identical (24 bytes), inputs and guards untouched, no overlap +PASS shapes cuda gdn_prep tf_gdn_prep_256 gb: 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda gdn_prep tf_gdn_prep_256 gb: declares none sums +PASS nonfinite cuda qk_rows tf_qk_rows_w (norm, rotation from a table): 135 non-finite and 35705 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows_w (norm, rotation from a table): five runs byte-identical (14336 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_w (norm, rotation from a table): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_w (norm, rotation from a table): within its bound at k 128; fp16 sums would miss 256x, bf16 1.9x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows_w (norm, no rotation): 132 non-finite and 35708 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows_w (norm, no rotation): five runs byte-identical (14336 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_w (norm, no rotation): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_w (norm, no rotation): within its bound at k 128; fp16 sums would miss 256x, bf16 0.9x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows_w (bf16 norm, rotation from inv_freq): 135 non-finite and 35705 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows_w (bf16 norm, rotation from inv_freq): five runs byte-identical (14336 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_w (bf16 norm, rotation from inv_freq): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_w (bf16 norm, rotation from inv_freq): within its bound at k 128; fp16 sums would miss 256x, bf16 1.8x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows_w (no norm, rotation from a table): 9 non-finite and 12791 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows_w (no norm, rotation from a table): five runs byte-identical (5120 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_w (no norm, rotation from a table): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_w (no norm, rotation from a table): exact sums at k 128, where fp16 sums would miss by infx and bf16 by 256x +PASS nonfinite cuda qk_rows tf_qk_rows (dim 256, norm, rotary 64 from a table): 262 non-finite and 40698 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows (dim 256, norm, rotary 64 from a table): five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows (dim 256, norm, rotary 64 from a table): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows (dim 256, norm, rotary 64 from a table): within its bound at k 256; fp16 sums would miss 256x, bf16 1.6x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows (dim 256, bf16 norm, rotary 64 from inv_freq): 262 non-finite and 40698 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows (dim 256, bf16 norm, rotary 64 from inv_freq): five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows (dim 256, bf16 norm, rotary 64 from inv_freq): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows (dim 256, bf16 norm, rotary 64 from inv_freq): within its bound at k 256; fp16 sums would miss 256x, bf16 1.5x (these stress inputs do not separate bf16 square sums beyond the normed output bound; fp16 overflows) +PASS nonfinite cuda qk_rows tf_qk_rows (dim 64, no norm, rotary 32 from a table): 8 non-finite and 3832 finite outputs match the float64 classes +PASS repeat cuda qk_rows tf_qk_rows (dim 64, no norm, rotary 32 from a table): five runs byte-identical (1536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows (dim 64, no norm, rotary 32 from a table): 5 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows (dim 64, no norm, rotary 32 from a table): exact sums at k 64, where fp16 sums would miss by infx and bf16 by 256x +PASS nonfinite cuda tree attention tf_tree_paths: the stated rule holds: integers only: each row's depth (at most 128) and its nearest 128 ancestors root first; other slots untouched +PASS repeat cuda tree attention tf_tree_paths: five runs byte-identical (8256 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention tf_tree_paths: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda tree attention tf_tree_paths: declares none sums +PASS nonfinite cuda tree attention dim 128, four-warp blocks (tf_tree_shared, tf_tree_part, tf_tree_merge): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 128, four-warp blocks (tf_tree_shared, tf_tree_part, tf_tree_merge): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 128, four-warp blocks (tf_tree_shared, tf_tree_part, tf_tree_merge): 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda tree attention dim 128, four-warp blocks (tf_tree_shared, tf_tree_part, tf_tree_merge): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1445465x +PASS nonfinite cuda tree attention dim 128, one-warp blocks (tf_tree_shared_w1, tf_tree_part_w1, sliced merge): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 128, one-warp blocks (tf_tree_shared_w1, tf_tree_part_w1, sliced merge): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 128, one-warp blocks (tf_tree_shared_w1, tf_tree_part_w1, sliced merge): 9 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +PASS sums cuda tree attention dim 128, one-warp blocks (tf_tree_shared_w1, tf_tree_part_w1, sliced merge): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1839101x +PASS nonfinite cuda tree attention dim 128, one-warp blocks, 3 stages (tf_tree_shared_w1s3, tf_tree_part_w1s3): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 128, one-warp blocks, 3 stages (tf_tree_shared_w1s3, tf_tree_part_w1s3): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 128, one-warp blocks, 3 stages (tf_tree_shared_w1s3, tf_tree_part_w1s3): 9 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +PASS sums cuda tree attention dim 128, one-warp blocks, 3 stages (tf_tree_shared_w1s3, tf_tree_part_w1s3): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1954573x +PASS nonfinite cuda tree attention dim 256, four-warp blocks (tf_tree_shared_256, tf_tree_part_256, tf_tree_merge): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 256, four-warp blocks (tf_tree_shared_256, tf_tree_part_256, tf_tree_merge): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 256, four-warp blocks (tf_tree_shared_256, tf_tree_part_256, tf_tree_merge): 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda tree attention dim 256, four-warp blocks (tf_tree_shared_256, tf_tree_part_256, tf_tree_merge): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1773922x +PASS nonfinite cuda tree attention dim 256, one-warp blocks (tf_tree_shared_256_w1, tf_tree_part_256_w1): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 256, one-warp blocks (tf_tree_shared_256_w1, tf_tree_part_256_w1): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 256, one-warp blocks (tf_tree_shared_256_w1, tf_tree_part_256_w1): 9 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +PASS sums cuda tree attention dim 256, one-warp blocks (tf_tree_shared_256_w1, tf_tree_part_256_w1): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 2003330x +PASS nonfinite cuda tree attention dim 256, one-warp blocks, 3 stages (tf_tree_shared_256_w1s3): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda tree attention dim 256, one-warp blocks, 3 stages (tf_tree_shared_256_w1s3): five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda tree attention dim 256, one-warp blocks, 3 stages (tf_tree_shared_256_w1s3): 9 shapes exact by its own check (worst 0.000 of the bound), 1 edges refused by the kernel +PASS sums cuda tree attention dim 256, one-warp blocks, 3 stages (tf_tree_shared_256_w1s3): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 2102379x +PASS nonfinite cuda block attention tf_block_attention (DFlash2 blocks of 16): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda block attention tf_block_attention (DFlash2 blocks of 16): five runs byte-identical (131072 bytes), inputs and guards untouched, no overlap +PASS shapes cuda block attention tf_block_attention (DFlash2 blocks of 16): 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda block attention tf_block_attention (DFlash2 blocks of 16): exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1602240x +PASS nonfinite cuda block attention tf_block_attention, causal blocks of 7: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda block attention tf_block_attention, causal blocks of 7: five runs byte-identical (57344 bytes), inputs and guards untouched, no overlap +PASS shapes cuda block attention tf_block_attention, causal blocks of 7: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda block attention tf_block_attention, causal blocks of 7: exact sums at k 2100, where fp16 sums would miss by infx and bf16 by 1704486x +PASS nonfinite cuda window decode tf_window_decode_tc, 48/4 heads, window 513, ring 640: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window decode tf_window_decode_tc, 48/4 heads, window 513, ring 640: five runs byte-identical (196608 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window decode tf_window_decode_tc, 48/4 heads, window 513, ring 640: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window decode tf_window_decode_tc, 48/4 heads, window 513, ring 640: exact sums at k 3000, where fp16 sums would miss by infx and bf16 by 204941x +PASS nonfinite cuda window decode tf_window_decode_tc, 16/1 heads, no window, ring 300: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window decode tf_window_decode_tc, 16/1 heads, no window, ring 300: five runs byte-identical (65536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window decode tf_window_decode_tc, 16/1 heads, no window, ring 300: 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window decode tf_window_decode_tc, 16/1 heads, no window, ring 300: exact sums at k 300, where fp16 sums would miss by infx and bf16 by 90367x +PASS nonfinite cuda window decode tf_window_part + tf_window_merge, 8/2 heads of 64, window 100, ring 128: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window decode tf_window_part + tf_window_merge, 8/2 heads of 64, window 100, ring 128: five runs byte-identical (16384 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window decode tf_window_part + tf_window_merge, 8/2 heads of 64, window 100, ring 128: 6 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window decode tf_window_part + tf_window_merge, 8/2 heads of 64, window 100, ring 128: exact sums at k 3000, where fp16 sums would miss by infx and bf16 by 65279x +PASS nonfinite cuda window prompt wattn_kernel<128, 8, 4, 8>, 48/4 heads, window 513, ring 2624: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window prompt wattn_kernel<128, 8, 4, 8>, 48/4 heads, window 513, ring 2624: five runs byte-identical (786432 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window prompt wattn_kernel<128, 8, 4, 8>, 48/4 heads, window 513, ring 2624: 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window prompt wattn_kernel<128, 8, 4, 8>, 48/4 heads, window 513, ring 2624: exact sums at k 2600, where fp16 sums would miss by infx and bf16 by 902610x +PASS nonfinite cuda window prompt wattn_kernel<128, 8, 8, 8>, 16/2 heads, no window, a row a position: the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda window prompt wattn_kernel<128, 8, 8, 8>, 16/2 heads, no window, a row a position: five runs byte-identical (262144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda window prompt wattn_kernel<128, 8, 8, 8>, 16/2 heads, no window, a row a position: 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda window prompt wattn_kernel<128, 8, 8, 8>, 16/2 heads, no window, a row a position: exact sums at k 100, where fp16 sums would miss by infx and bf16 by 65791x +PASS nonfinite cuda moe top-k tf_moe_topk_sigmoid_warp (Kolibri: 384 experts, top 6 + shared): the stated rule holds: picks by fp32 logit + bias, larger first, ties to the lower id, a NaN never picked; weights sigmoid(logit) +PASS repeat cuda moe top-k tf_moe_topk_sigmoid_warp (Kolibri: 384 experts, top 6 + shared): five runs byte-identical (896 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe top-k tf_moe_topk_sigmoid_warp (Kolibri: 384 experts, top 6 + shared): 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda moe top-k tf_moe_topk_sigmoid_warp (Kolibri: 384 experts, top 6 + shared): declares none sums +PASS nonfinite cuda moe top-k tf_moe_topk_sigmoid_warp (256 experts, top 8 normalized x2.5): the stated rule holds: picks by fp32 logit + bias, larger first, ties to the lower id, a NaN never picked; weights sigmoid(logit) +PASS repeat cuda moe top-k tf_moe_topk_sigmoid_warp (256 experts, top 8 normalized x2.5): five runs byte-identical (1024 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe top-k tf_moe_topk_sigmoid_warp (256 experts, top 8 normalized x2.5): 10 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda moe top-k tf_moe_topk_sigmoid_warp (256 experts, top 8 normalized x2.5): declares none sums +PASS nonfinite cuda moe top-k tf_moe_topk_sigmoid (block form, 513 to 4,096 experts): the stated rule holds: picks by fp32 logit + bias, larger first, ties to the lower id, a NaN never picked; weights sigmoid(logit) +PASS repeat cuda moe top-k tf_moe_topk_sigmoid (block form, 513 to 4,096 experts): five runs byte-identical (8320 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe top-k tf_moe_topk_sigmoid (block form, 513 to 4,096 experts): 3 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda moe top-k tf_moe_topk_sigmoid (block form, 513 to 4,096 experts): declares none sums +PASS nonfinite cuda moe combine tf_moe_combine (fp32 rows into bf16): 4 non-finite and 20476 finite outputs match the float64 classes +PASS repeat cuda moe combine tf_moe_combine (fp32 rows into bf16): five runs byte-identical (81920 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe combine tf_moe_combine (fp32 rows into bf16): 11 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda moe combine tf_moe_combine (fp32 rows into bf16): exact sums at k 9, where fp16 sums would miss by infx and bf16 by 1711x +PASS nonfinite cuda moe combine tf_moe_combine (bf16 rows into bf16): 4 non-finite and 20476 finite outputs match the float64 classes +PASS repeat cuda moe combine tf_moe_combine (bf16 rows into bf16): five runs byte-identical (81920 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe combine tf_moe_combine (bf16 rows into bf16): 11 shapes within the float64 bound (worst 0.996 of the bound), 0 edges refused by the kernel +PASS sums cuda moe combine tf_moe_combine (bf16 rows into bf16): exact sums at k 9, where fp16 sums would miss by infx and bf16 by 2033x +PASS nonfinite cuda moe combine tf_moe_combine (fp32 rows into fp32): 4 non-finite and 20476 finite outputs match the float64 classes +PASS repeat cuda moe combine tf_moe_combine (fp32 rows into fp32): five runs byte-identical (163840 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe combine tf_moe_combine (fp32 rows into fp32): 11 shapes within the float64 bound (worst 0.590 of the bound), 0 edges refused by the kernel +PASS sums cuda moe combine tf_moe_combine (fp32 rows into fp32): exact sums at k 9, where fp16 sums would miss by infx and bf16 by 294170643x +PASS nonfinite cuda moe combine tf_moe_combine (fp16 rows into fp16, the generic loop): 4 non-finite and 20476 finite outputs match the float64 classes +PASS repeat cuda moe combine tf_moe_combine (fp16 rows into fp16, the generic loop): five runs byte-identical (81920 bytes), inputs and guards untouched, no overlap +PASS shapes cuda moe combine tf_moe_combine (fp16 rows into fp16, the generic loop): 11 shapes within the float64 bound (worst 0.997 of the bound), 0 edges refused by the kernel +PASS sums cuda moe combine tf_moe_combine (fp16 rows into fp16, the generic loop): exact sums at k 9, where fp16 sums would miss by infx and bf16 by 15838x +PASS nonfinite cuda dense rows tf_dense_rows_1: 1536 non-finite and 384 finite outputs match the float64 classes +PASS repeat cuda dense rows tf_dense_rows_1: five runs byte-identical (1536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda dense rows tf_dense_rows_1: 12 shapes within the float64 bound (worst 0.117 of the bound), 0 edges refused by the kernel +PASS sums cuda dense rows tf_dense_rows_1: exact sums at k 7168, where fp16 sums would miss by 159112952x and bf16 by 537412113x +PASS nonfinite cuda dense rows tf_dense_rows_4: 1536 non-finite and 384 finite outputs match the float64 classes +PASS repeat cuda dense rows tf_dense_rows_4: five runs byte-identical (6144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda dense rows tf_dense_rows_4: 15 shapes within the float64 bound (worst 0.157 of the bound), 0 edges refused by the kernel +PASS sums cuda dense rows tf_dense_rows_4: exact sums at k 7168, where fp16 sums would miss by 194615706x and bf16 by 619514235x +PASS nonfinite cuda dense rows tf_dense_rows_16: 1536 non-finite and 1536 finite outputs match the float64 classes +PASS repeat cuda dense rows tf_dense_rows_16: five runs byte-identical (24576 bytes), inputs and guards untouched, no overlap +PASS shapes cuda dense rows tf_dense_rows_16: 15 shapes within the float64 bound (worst 0.156 of the bound), 0 edges refused by the kernel +PASS sums cuda dense rows tf_dense_rows_16: exact sums at k 7168, where fp16 sums would miss by 141315781x and bf16 by 764125929x +PASS nonfinite cuda dense rows tf_dense_rows_32: 1536 non-finite and 1536 finite outputs match the float64 classes +PASS repeat cuda dense rows tf_dense_rows_32: five runs byte-identical (49152 bytes), inputs and guards untouched, no overlap +PASS shapes cuda dense rows tf_dense_rows_32: 17 shapes within the float64 bound (worst 0.213 of the bound), 0 edges refused by the kernel +PASS sums cuda dense rows tf_dense_rows_32: exact sums at k 7168, where fp16 sums would miss by 130722475x and bf16 by 842216243x +PASS nonfinite cuda embed_rows tf_embed_rows_affine4 (MLX 4-bit rows): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine4 (MLX 4-bit rows): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine4 (MLX 4-bit rows): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine4 (MLX 4-bit rows): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (2-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (2-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (2-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (2-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (3-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (3-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (3-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (3-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (4-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (4-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (4-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (4-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (5-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (5-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (5-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (5-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (6-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (6-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (6-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (6-bit, bf16 scale): declares none sums +PASS nonfinite cuda embed_rows tf_embed_rows_affine (8-bit, bf16 scale): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda embed_rows tf_embed_rows_affine (8-bit, bf16 scale): five runs byte-identical (130560 bytes), inputs and guards untouched, no overlap +PASS shapes cuda embed_rows tf_embed_rows_affine (8-bit, bf16 scale): 7 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda embed_rows tf_embed_rows_affine (8-bit, bf16 scale): declares none sums +FAIL shapes control: scaled rows without the dequantized value's bf16 rounding: at 17x3840x1: row 0 out[4] = -2.0375e1 where float64 at the rounding points gives -2.05e1 +CONTROL control: scaled rows without the dequantized value's bf16 rounding: red at shapes, as planted +FAIL shapes control: scaled rows with the scale folded into the group's scale and bias: at 17x3840x1: row 0 out[2] = 1.37e2 where float64 at the rounding points gives 1.38e2 +CONTROL control: scaled rows with the scale folded into the group's scale and bias: red at shapes, as planted +PASS nonfinite cuda qk_rows tf_qk_rows_round (MLX's rows, dim 512, k's columns as v by a unit norm, 64 of 256 pairs rotated): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda qk_rows tf_qk_rows_round (MLX's rows, dim 512, k's columns as v by a unit norm, 64 of 256 pairs rotated): five runs byte-identical (40960 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_round (MLX's rows, dim 512, k's columns as v by a unit norm, 64 of 256 pairs rotated): 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_round (MLX's rows, dim 512, k's columns as v by a unit norm, 64 of 256 pairs rotated): within its bound at k 512; fp16 sums would miss 51x, bf16 0.7x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda qk_rows tf_qk_rows_round (MLX's rows, dim 256, unit v norm, rotation from inv_freq): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda qk_rows tf_qk_rows_round (MLX's rows, dim 256, unit v norm, rotation from inv_freq): five runs byte-identical (32768 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_round (MLX's rows, dim 256, unit v norm, rotation from inv_freq): 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_round (MLX's rows, dim 256, unit v norm, rotation from inv_freq): within its bound at k 256; fp16 sums would miss 51x, bf16 0.7x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +PASS nonfinite cuda qk_rows tf_qk_rows_once (dim 512, norm, rotation from a table, v as is): the stated rule holds: outputs equal the float64 reference's bytes +PASS repeat cuda qk_rows tf_qk_rows_once (dim 512, norm, rotation from a table, v as is): five runs byte-identical (18432 bytes), inputs and guards untouched, no overlap +PASS shapes cuda qk_rows tf_qk_rows_once (dim 512, norm, rotation from a table, v as is): 5 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda qk_rows tf_qk_rows_once (dim 512, norm, rotation from a table, v as is): within its bound at k 512; fp16 sums would miss 51x, bf16 0.4x (an output follows its sum of squares as ss^-1/2 through bf16 rounding points: a bf16 sum's error is a few output ulps, and an fp16 sum overflows to a zero output, 51x the bound that carries the points) +FAIL shapes control: MLX's qk rows without the normalized row's rounding: at 1x32x512: row 0 head 0 out[5] = 1.572265625e-1 outside float64's [1.5625e-1, 1.5625e-1] +CONTROL control: MLX's qk rows without the normalized row's rounding: red at shapes, as planted +FAIL shapes control: MLX's qk rows without the weighted row's rounding before RoPE: at 300x32x512: row 1 head 0 out[0] = -2.353515625e-1 outside float64's [-2.36328125e-1, -2.36328125e-1] +CONTROL control: MLX's qk rows without the weighted row's rounding before RoPE: red at shapes, as planted +FAIL shapes control: MLX's qk rows without v's unit norm: at 1x32x512: row 0 head 36 out[0] = -1.1875e0 outside float64's [-6.2109375e-1, -6.2109375e-1] +CONTROL control: MLX's qk rows without v's unit norm: red at shapes, as planted +PASS nonfinite cuda flashnext factored prompt matmul 64x128: 55808 non-finite and 55808 finite outputs match the float64 classes +PASS repeat cuda flashnext factored prompt matmul 64x128: five runs byte-identical (1785856 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext factored prompt matmul 64x128: 14 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda flashnext factored prompt matmul 64x128: exact sums at k 2560, where fp16 sums would miss by 6357x and bf16 by 37192x +PASS nonfinite cuda flashnext factored prompt matmul 64x128, K slices folded: 10240 non-finite and 10240 finite outputs match the float64 classes +PASS repeat cuda flashnext factored prompt matmul 64x128, K slices folded: five runs byte-identical (655360 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext factored prompt matmul 64x128, K slices folded: 12 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 3 edges refused by the kernel +PASS sums cuda flashnext factored prompt matmul 64x128, K slices folded: exact sums at k 6144, where fp16 sums would miss by 189258762x and bf16 by 1437248114x +PASS nonfinite cuda route_rows tf_route_logits: 2052 non-finite and 2052 finite outputs match the float64 classes +PASS repeat cuda route_rows tf_route_logits: five runs byte-identical (65664 bytes), inputs and guards untouched, no overlap +PASS shapes cuda route_rows tf_route_logits: 14 shapes within the float64 bound (worst 0.086 of the bound), 0 edges refused by the kernel +PASS sums cuda route_rows tf_route_logits: exact sums at k 2560, where fp16 sums would miss by 30905398x and bf16 by 376928119x +PASS nonfinite cuda route_rows tf_route_logits_few: 2052 non-finite and 2052 finite outputs match the float64 classes +PASS repeat cuda route_rows tf_route_logits_few: five runs byte-identical (32832 bytes), inputs and guards untouched, no overlap +PASS shapes cuda route_rows tf_route_logits_few: 11 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda route_rows tf_route_logits_few: exact sums at k 2560, where fp16 sums would miss by 34225521x and bf16 by 366702007x +PASS nonfinite cuda route_rows tf_route_logits_wide: 2052 non-finite and 2052 finite outputs match the float64 classes +PASS repeat cuda route_rows tf_route_logits_wide: five runs byte-identical (131328 bytes), inputs and guards untouched, no overlap +PASS shapes cuda route_rows tf_route_logits_wide: 14 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda route_rows tf_route_logits_wide: exact sums at k 2560, where fp16 sums would miss by 27682406x and bf16 by 331897099x +PASS nonfinite cuda flashnext factored experts down 64 pairs, fp32 out: 10240 non-finite and 10240 finite outputs match the float64 classes +PASS repeat cuda flashnext factored experts down 64 pairs, fp32 out: five runs byte-identical (655360 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext factored experts down 64 pairs, fp32 out: 13 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda flashnext factored experts down 64 pairs, fp32 out: exact sums at k 640, where fp16 sums would miss by 15711995x and bf16 by 5592405x +PASS nonfinite cuda flashnext factored experts gate and up 64 pairs, SwiGLU: 2560 non-finite and 2560 finite outputs match the float64 classes +PASS repeat cuda flashnext factored experts gate and up 64 pairs, SwiGLU: five runs byte-identical (81920 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext factored experts gate and up 64 pairs, SwiGLU: 13 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +PASS sums cuda flashnext factored experts gate and up 64 pairs, SwiGLU: exact sums at k 2560, where fp16 sums would miss by 428203617113232950000000x and bf16 by 128x +PASS nonfinite cuda flashnext DeltaNet prompt chain fn_gdn_chain_tpr: the stated rule holds: ys and the state bit-equal to fn_gdn_rows_to's, non-finite inputs included +PASS repeat cuda flashnext DeltaNet prompt chain fn_gdn_chain_tpr: five runs byte-identical (4718592 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext DeltaNet prompt chain fn_gdn_chain_tpr: 6 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda flashnext DeltaNet prompt chain fn_gdn_chain_tpr: declares none sums +PASS nonfinite cuda flashnext indexer scores qsa_tile::scores<8, 4>: the stated rule holds: every score byte equal to fn_qsa_scores', non-finite inputs included +PASS repeat cuda flashnext indexer scores qsa_tile::scores<8, 4>: five runs byte-identical (153600 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext indexer scores qsa_tile::scores<8, 4>: 8 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 4 edges refused by the kernel +N/A sums cuda flashnext indexer scores qsa_tile::scores<8, 4>: declares none sums +PASS nonfinite cuda flashnext hyper-connection read-out on prompt tiles (hcf down, reduce, hcf up, rinv, mix_h): the stated rule holds: act, xs, inj, mixed and xsm bit-equal to #952's chain, non-finite inputs included +PASS repeat cuda flashnext hyper-connection read-out on prompt tiles (hcf down, reduce, hcf up, rinv, mix_h): five runs byte-identical (392192 bytes), inputs and guards untouched, no overlap +PASS shapes cuda flashnext hyper-connection read-out on prompt tiles (hcf down, reduce, hcf up, rinv, mix_h): 5 shapes bit-equal to the kernel it replaces (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda flashnext hyper-connection read-out on prompt tiles (hcf down, reduce, hcf up, rinv, mix_h): declares none sums +PASS nonfinite cuda glm tf_glm_lat_write on pages out of order: 5 non-finite and 4091 finite outputs match the float64 classes +PASS repeat cuda glm tf_glm_lat_write on pages out of order: five runs byte-identical (8192 bytes), inputs and guards untouched, no overlap +PASS shapes cuda glm tf_glm_lat_write on pages out of order: 2 shapes within the float64 bound (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda glm tf_glm_lat_write on pages out of order: declares none sums +PASS nonfinite cuda glm tf_glm_pool_keys on pages out of order: 2 non-finite and 254 finite outputs match the float64 classes +PASS repeat cuda glm tf_glm_pool_keys on pages out of order: five runs byte-identical (512 bytes), inputs and guards untouched, no overlap +PASS shapes cuda glm tf_glm_pool_keys on pages out of order: 2 shapes within the float64 bound (worst 0.702 of the bound), 0 edges refused by the kernel +PASS sums cuda glm tf_glm_pool_keys on pages out of order: within its bound at k 4; fp16 sums would miss 1x, bf16 0.7x (a pool's four bf16 products of softmax weights: no input makes the sum exact in fp32) +FAIL nonfinite cuda glm tf_glm_scores on pages out of order: 1024 of 16384 outputs differ in class; first at 0: finite where float64 gives nan +KNOWN nonfinite cuda glm tf_glm_scores on pages out of order: relu is fmaxf(x, 0), as Triton's tl.maximum: a NaN dot scores 0 where float64 keeps the NaN +PASS repeat cuda glm tf_glm_scores on pages out of order: five runs byte-identical (65536 bytes), inputs and guards untouched, no overlap +PASS shapes cuda glm tf_glm_scores on pages out of order: 2 shapes within the float64 bound (worst 0.001 of the bound), 0 edges refused by the kernel +PASS sums cuda glm tf_glm_scores on pages out of order: exact sums at k 128, where fp16 sums would miss by 4784x and bf16 by 50382x +PASS nonfinite cuda glm tf_glm_dsa_tokens on pages out of order: the stated rule holds: integers only: pool ids, positions and page ids +PASS repeat cuda glm tf_glm_dsa_tokens on pages out of order: five runs byte-identical (65664 bytes), inputs and guards untouched, no overlap +PASS shapes cuda glm tf_glm_dsa_tokens on pages out of order: 2 shapes exact by its own check (worst 0.000 of the bound), 0 edges refused by the kernel +N/A sums cuda glm tf_glm_dsa_tokens on pages out of order: declares none sums +PASS nonfinite cuda glm tf_glm_chunks_16 dense on pages out of order, with tf_glm_merge: 63109 non-finite and 67963 finite outputs match the float64 classes +PASS repeat cuda glm tf_glm_chunks_16 dense on pages out of order, with tf_glm_merge: five runs byte-identical (262144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda glm tf_glm_chunks_16 dense on pages out of order, with tf_glm_merge: 1 shapes within the float64 bound (worst 0.127 of the bound), 0 edges refused by the kernel +PASS sums cuda glm tf_glm_chunks_16 dense on pages out of order, with tf_glm_merge: within its bound at k 1024; fp16 sums would miss 0x, bf16 0.5x (online softmax over 512-key chunks, then the chunk merge: no input makes its sums exact in fp32) +PASS nonfinite cuda glm tf_glm_chunks_16 sparse on pages out of order, with tf_glm_merge: 69752 non-finite and 61320 finite outputs match the float64 classes +PASS repeat cuda glm tf_glm_chunks_16 sparse on pages out of order, with tf_glm_merge: five runs byte-identical (262144 bytes), inputs and guards untouched, no overlap +PASS shapes cuda glm tf_glm_chunks_16 sparse on pages out of order, with tf_glm_merge: 1 shapes within the float64 bound (worst 0.108 of the bound), 0 edges refused by the kernel +PASS sums cuda glm tf_glm_chunks_16 sparse on pages out of order, with tf_glm_merge: within its bound at k 2051; fp16 sums would miss 0x, bf16 0.3x (online softmax over 512-key chunks, then the chunk merge: no input makes its sums exact in fp32) +PASS nonfinite cuda glm tf_glm_chunks_32 dense on pages out of order, with tf_glm_merge: 58510 non-finite and 72562 finite outputs match the float64 classes +PASS repeat cuda glm tf_glm_chunks_32 dense on pages out of order, with tf_glm_merge: five runs byte-identical (2097152 bytes), inputs and guards untouched, no overlap +PASS shapes cuda glm tf_glm_chunks_32 dense on pages out of order, with tf_glm_merge: 1 shapes within the float64 bound (worst 0.207 of the bound), 0 edges refused by the kernel +PASS sums cuda glm tf_glm_chunks_32 dense on pages out of order, with tf_glm_merge: within its bound at k 512; fp16 sums would miss 0x, bf16 0.5x (online softmax over 512-key chunks, then the chunk merge: no input makes its sums exact in fp32) +PASS nonfinite cuda glm tf_glm_chunks_32 sparse on pages out of order, with tf_glm_merge: 13862 non-finite and 117210 finite outputs match the float64 classes +PASS repeat cuda glm tf_glm_chunks_32 sparse on pages out of order, with tf_glm_merge: five runs byte-identical (2097152 bytes), inputs and guards untouched, no overlap +PASS shapes cuda glm tf_glm_chunks_32 sparse on pages out of order, with tf_glm_merge: 1 shapes within the float64 bound (worst 0.225 of the bound), 0 edges refused by the kernel +PASS sums cuda glm tf_glm_chunks_32 sparse on pages out of order, with tf_glm_merge: within its bound at k 300; fp16 sums would miss 0x, bf16 0.6x (online softmax over 512-key chunks, then the chunk merge: no input makes its sums exact in fp32) +RESULT gdn-back finite differences (6 rows, 2 key / 6 value heads): 120 coordinates of P, worst |fd - analytic| / max(|analytic|, 1e-3 max) 7.91e-7 +RESULT gdn-back finite differences (9 rows, 1 key / 3 value heads): 120 coordinates of P, worst |fd - analytic| / max(|analytic|, 1e-3 max) 1.43e-6 +--- gdn-back: 64 rows, after a prefix (conv history and state) +RESULT gdn-back time 64 rows: layer backward 3.622 ms (forward for training, chain and gates 1.041 ms, the rest the two projections' dequant and GEMM); scratch 167.1 MiB (checkpoints 48.0, replay 12.0, dequantized projection 80.5) +RESULT gdn-back steps 64 rows (ms): front 0.015, scan_fwd 0.275, norm_back 0.050, scan_back 0.755, qk_back 0.056, gates_back 0.006, conv_back 0.035 +RESULT gdn-back forward act 3 of 655360 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 3072 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 50 of 393216 bf16 values differ from float64's rounding, at most 2 ulp +RESULT gdn-back forward q worst column rel 1.98e-7 (col 1743), max abs 5.67e-9, ref scale 5.04e-2 +RESULT gdn-back forward k worst column rel 1.71e-7 (col 1920), max abs 6.66e-8, ref scale 6.00e-1 +RESULT gdn-back forward g worst column rel 9.24e-8 (col 27), max abs 8.79e-8, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 1.35e-3 (col 2595), max abs 7.63e-6, ref scale 6.30e-2 +RESULT gdn-back dout (o_proj back) worst column rel 5.54e-3 (col 13), max abs 8.45e-5, ref scale 3.49e-2 +RESULT gdn-back dout, bf16 operands worst column rel 4.93e-6 (col 1875), max abs 9.05e-8, ref scale 3.49e-2 +RESULT gdn-back dy (norm back) worst column rel 2.93e-7 (col 5351), max abs 4.79e-6, ref scale 3.17e1 +RESULT gdn-back dz worst column rel 8.23e-7 (col 5349), max abs 4.86e-9, ref scale 2.40e-2 +RESULT gdn-back dq (chain) worst column rel 6.10e-7 (col 1042), max abs 2.11e-6, ref scale 1.43e1 +RESULT gdn-back dk (chain) worst column rel 6.35e-7 (col 112), max abs 2.07e-7, ref scale 1.39e0 +RESULT gdn-back dv (chain) worst column rel 1.02e-6 (col 4340), max abs 2.93e-8, ref scale 1.55e-1 +RESULT gdn-back dg (chain) worst column rel 1.06e-6 (col 3), max abs 1.58e-7, ref scale 5.03e-1 +RESULT gdn-back dbeta (chain) worst column rel 8.08e-7 (col 44), max abs 1.72e-7, ref scale 5.41e-1 +RESULT gdn-back dact q (L2 back) worst column rel 6.51e-7 (col 986), max abs 7.69e-8, ref scale 3.46e-1 +RESULT gdn-back dact k (L2 back) worst column rel 7.57e-7 (col 515), max abs 7.45e-8, ref scale 3.63e-1 +RESULT gdn-back dproj q (conv back) worst column rel 1.06e-6 (col 542), max abs 4.73e-8, ref scale 1.23e-1 +RESULT gdn-back dproj k (conv back) worst column rel 7.68e-7 (col 515), max abs 2.84e-8, ref scale 1.51e-1 +RESULT gdn-back dproj v (conv back) worst column rel 1.03e-6 (col 4340), max abs 1.26e-8, ref scale 6.90e-2 +RESULT gdn-back dproj b (beta back) worst column rel 4.81e-7 (col 16), max abs 1.70e-8, ref scale 7.14e-2 +RESULT gdn-back dproj a (g back) worst column rel 1.13e-6 (col 3), max abs 2.09e-8, ref scale 6.49e-2 +RESULT gdn-back dx (in_proj back) worst column rel 4.95e-3 (col 1043), max abs 1.47e-4, ref scale 6.20e-2 +RESULT gdn-back dx, bf16 operands worst column rel 4.37e-5 (col 1415), max abs 1.43e-6, ref scale 6.20e-2 +--- gdn-back: 256 rows, from the sequence start +RESULT gdn-back time 256 rows: layer backward 6.724 ms (forward for training, chain and gates 3.948 ms, the rest the two projections' dequant and GEMM); scratch 391.1 MiB (checkpoints 192.0, replay 12.0, dequantized projection 80.5) +RESULT gdn-back steps 256 rows (ms): front 0.060, scan_fwd 1.047, norm_back 0.133, scan_back 2.343, qk_back 0.264, gates_back 0.006, conv_back 0.096 +RESULT gdn-back forward act 11 of 2621440 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 12288 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 177 of 1572864 bf16 values differ from float64's rounding, at most 10 ulp +RESULT gdn-back forward q worst column rel 2.00e-7 (col 593), max abs 7.63e-9, ref scale 5.58e-2 +RESULT gdn-back forward k worst column rel 1.86e-7 (col 62), max abs 7.25e-8, ref scale 6.21e-1 +RESULT gdn-back forward g worst column rel 1.08e-7 (col 30), max abs 9.63e-8, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 2.12e-3 (col 2828), max abs 3.05e-5, ref scale 6.18e-2 +RESULT gdn-back dout (o_proj back) worst column rel 6.33e-3 (col 4447), max abs 7.60e-5, ref scale 3.23e-2 +RESULT gdn-back dout, bf16 operands worst column rel 5.02e-6 (col 115), max abs 9.78e-8, ref scale 3.23e-2 +RESULT gdn-back dy (norm back) worst column rel 2.77e-7 (col 3309), max abs 5.91e-6, ref scale 3.62e1 +RESULT gdn-back dz worst column rel 8.01e-7 (col 5695), max abs 6.20e-9, ref scale 3.33e-2 +RESULT gdn-back dq (chain) worst column rel 5.15e-7 (col 1818), max abs 2.84e-6, ref scale 1.44e1 +RESULT gdn-back dk (chain) worst column rel 5.19e-7 (col 1198), max abs 2.23e-7, ref scale 1.39e0 +RESULT gdn-back dv (chain) worst column rel 7.24e-7 (col 4989), max abs 3.33e-8, ref scale 1.46e-1 +RESULT gdn-back dg (chain) worst column rel 6.25e-7 (col 35), max abs 1.22e-7, ref scale 6.07e-1 +RESULT gdn-back dbeta (chain) worst column rel 4.22e-7 (col 40), max abs 1.68e-7, ref scale 6.93e-1 +RESULT gdn-back dact q (L2 back) worst column rel 6.70e-7 (col 1682), max abs 9.95e-8, ref scale 3.88e-1 +RESULT gdn-back dact k (L2 back) worst column rel 5.90e-7 (col 1391), max abs 7.69e-8, ref scale 4.72e-1 +RESULT gdn-back dproj q (conv back) worst column rel 7.64e-7 (col 1682), max abs 4.43e-8, ref scale 1.56e-1 +RESULT gdn-back dproj k (conv back) worst column rel 7.01e-7 (col 977), max abs 2.84e-8, ref scale 1.58e-1 +RESULT gdn-back dproj v (conv back) worst column rel 6.77e-7 (col 1722), max abs 2.63e-8, ref scale 7.86e-2 +RESULT gdn-back dproj b (beta back) worst column rel 5.11e-7 (col 2), max abs 1.71e-8, ref scale 8.65e-2 +RESULT gdn-back dproj a (g back) worst column rel 4.91e-7 (col 12), max abs 2.09e-8, ref scale 6.99e-2 +RESULT gdn-back dx (in_proj back) worst column rel 5.62e-3 (col 2311), max abs 1.42e-4, ref scale 6.96e-2 +RESULT gdn-back dx, bf16 operands worst column rel 4.22e-5 (col 2107), max abs 2.18e-6, ref scale 6.97e-2 +--- gdn-back: 37 rows, after a prefix (conv history and state) +RESULT gdn-back time 37 rows: layer backward 3.142 ms (forward for training, chain and gates 0.605 ms, the rest the two projections' dequant and GEMM); scratch 137.9 MiB (checkpoints 30.0, replay 12.0, dequantized projection 80.5) +RESULT gdn-back steps 37 rows (ms): front 0.012, scan_fwd 0.167, norm_back 0.024, scan_back 0.388, qk_back 0.022, gates_back 0.004, conv_back 0.018 +RESULT gdn-back forward act 1 of 378880 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 1776 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 25 of 227328 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward q worst column rel 1.76e-7 (col 1327), max abs 6.19e-9, ref scale 5.26e-2 +RESULT gdn-back forward k worst column rel 1.51e-7 (col 362), max abs 6.43e-8, ref scale 5.82e-1 +RESULT gdn-back forward g worst column rel 9.50e-8 (col 10), max abs 9.33e-8, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 1.33e-3 (col 5959), max abs 1.53e-5, ref scale 5.32e-2 +RESULT gdn-back dout (o_proj back) worst column rel 1.32e-2 (col 5971), max abs 7.81e-5, ref scale 3.33e-2 +RESULT gdn-back dout, bf16 operands worst column rel 6.45e-6 (col 3966), max abs 8.90e-8, ref scale 3.33e-2 +RESULT gdn-back dy (norm back) worst column rel 2.67e-7 (col 5113), max abs 4.87e-6, ref scale 2.70e1 +RESULT gdn-back dz worst column rel 9.25e-7 (col 1425), max abs 4.35e-9, ref scale 2.46e-2 +RESULT gdn-back dq (chain) worst column rel 7.02e-7 (col 443), max abs 3.15e-6, ref scale 3.46e1 +RESULT gdn-back dk (chain) worst column rel 8.63e-7 (col 1350), max abs 2.49e-7, ref scale 1.29e0 +RESULT gdn-back dv (chain) worst column rel 9.06e-7 (col 1584), max abs 2.91e-8, ref scale 1.31e-1 +RESULT gdn-back dg (chain) worst column rel 5.21e-7 (col 45), max abs 2.70e-7, ref scale 1.11e0 +RESULT gdn-back dbeta (chain) worst column rel 6.01e-7 (col 0), max abs 1.20e-7, ref scale 4.72e-1 +RESULT gdn-back dact q (L2 back) worst column rel 6.79e-7 (col 723), max abs 1.34e-7, ref scale 8.70e-1 +RESULT gdn-back dact k (L2 back) worst column rel 1.24e-6 (col 1350), max abs 6.20e-8, ref scale 3.34e-1 +RESULT gdn-back dproj q (conv back) worst column rel 8.61e-7 (col 706), max abs 3.13e-8, ref scale 1.08e-1 +RESULT gdn-back dproj k (conv back) worst column rel 1.32e-6 (col 1210), max abs 1.89e-8, ref scale 1.07e-1 +RESULT gdn-back dproj v (conv back) worst column rel 1.04e-6 (col 5917), max abs 1.62e-8, ref scale 6.13e-2 +RESULT gdn-back dproj b (beta back) worst column rel 5.73e-7 (col 17), max abs 1.45e-8, ref scale 5.97e-2 +RESULT gdn-back dproj a (g back) worst column rel 5.49e-7 (col 46), max abs 3.68e-8, ref scale 7.86e-2 +RESULT gdn-back dx (in_proj back) worst column rel 1.53e-2 (col 2267), max abs 1.44e-4, ref scale 6.01e-2 +RESULT gdn-back dx, bf16 operands worst column rel 4.69e-5 (col 1115), max abs 1.66e-6, ref scale 6.01e-2 +PASS gdn-back: forward for training and every input gradient within tolerance of float64 +RESULT fd head cross-entropy dlogits: float64 reference against central differences at 4 coordinates, worst relative 7.96e-7 +RESULT head weights: 378.9 MiB (vocab 248320) +RESULT head 64 rows (loss, target probability): worst column 6.77e-8 (column 0 of 2), relative L2 3.32e-8, largest |ref| 1.16e1 +RESULT head 64 rows dlogits (bf16): largest difference 2.54e-3 of the largest |ref| 5.78e-3 +RESULT head 64 rows dmixed: worst column 6.55e-3 (column 435 of 2560), relative L2 2.56e-3, largest |ref| 3.10e-4 +RESULT head 64 rows: 4.966 ms softmax and head, 3.8 MiB of gradients and scratch (+ 30.3 MiB bf16 logits) +RESULT head 256 rows (loss, target probability): worst column 7.24e-8 (column 0 of 2), relative L2 3.18e-8, largest |ref| 1.16e1 +RESULT head 256 rows dlogits (bf16): largest difference 2.57e-3 of the largest |ref| 5.85e-3 +RESULT head 256 rows dmixed: worst column 6.03e-3 (column 1172 of 2560), relative L2 2.41e-3, largest |ref| 3.15e-4 +RESULT head 256 rows: 17.117 ms softmax and head, 7.5 MiB of gradients and scratch (+ 121.3 MiB bf16 logits) +PASS q4-back: 7 checks within tolerance (15.0 s with the float64 references) +RESULT fd moe dx: float64 reference against central differences at 8 coordinates, worst relative 1.18e-9 +RESULT moe weights: 5513.2 MiB (512 routed experts as MLX stacks and packed, the shared expert, the router, a pocket of 0) +RESULT moe 64 rows dx: worst column 2.74e-6 (column 437 of 2560), relative L2 1.15e-6, largest |ref| 7.33e0 +RESULT moe 64 rows slot scores a = dL/dw: worst column 1.74e-6 (column 3 of 11), relative L2 1.13e-6, largest |ref| 9.45e1 +RESULT moe 64 rows router logits' gradients: worst column 1.94e-6 (column 3 of 11), relative L2 1.21e-6, largest |ref| 1.96e1 +RESULT moe 64 rows: 18.557 ms on the GPU, 16.2 MiB of gradients and scratch +RESULT moe 64 rows packed dx: worst column 2.74e-6 (column 437 of 2560), relative L2 1.15e-6, largest |ref| 7.33e0 +RESULT moe 64 rows packed: 14.434 ms, dx equals the MLX stacks' bits +RESULT moe 256 rows dx: worst column 2.30e-6 (column 588 of 2560), relative L2 1.15e-6, largest |ref| 8.93e0 +RESULT moe 256 rows slot scores a = dL/dw: worst column 1.64e-6 (column 1 of 11), relative L2 1.18e-6, largest |ref| 1.47e2 +RESULT moe 256 rows router logits' gradients: worst column 1.73e-6 (column 9 of 11), relative L2 1.18e-6, largest |ref| 2.83e1 +RESULT moe 256 rows: 27.686 ms on the GPU, 64.9 MiB of gradients and scratch +RESULT moe 256 rows packed dx: worst column 2.30e-6 (column 588 of 2560), relative L2 1.15e-6, largest |ref| 8.93e0 +RESULT moe 256 rows packed: 21.404 ms, dx equals the MLX stacks' bits +PASS moe-back: 11 checks within tolerance (11.8 s with the float64 references) +RESULT fd attn dx: float64 reference against central differences at 6 coordinates, worst relative 7.62e-9 +RESULT attn weights: 29.7 MiB +RESULT attn 64 rows prepared heads: 6 of 458752 values differ from the host's bf16, the largest by 6.10e-3 of max(1, |value|) (row 40, column 4584) +RESULT attn 64 rows dpq (q and gate): worst column 1.29e-2 (column 12072 of 12288), relative L2 2.12e-3, largest |ref| 4.63e-1 +RESULT attn 64 rows dpk: worst column 7.32e-3 (column 399 of 512), relative L2 2.10e-3, largest |ref| 9.64e-1 +RESULT attn 64 rows dv: worst column 8.73e-3 (column 107 of 512), relative L2 2.09e-3, largest |ref| 4.16e0 +RESULT attn 64 rows dx: worst column 1.26e-2 (column 486 of 2560), relative L2 3.09e-3, largest |ref| 2.62e0 +RESULT attn 64 rows: 1.290 ms on the GPU, 10.1 MiB of gradients and scratch +RESULT attn 256 rows prepared heads: 26 of 1835008 values differ from the host's bf16, the largest by 6.25e-3 of max(1, |value|) (row 111, column 1607) +RESULT attn 256 rows dpq (q and gate): worst column 1.29e-2 (column 12072 of 12288), relative L2 2.12e-3, largest |ref| 4.63e-1 +RESULT attn 256 rows dpk: worst column 6.90e-3 (column 366 of 512), relative L2 2.11e-3, largest |ref| 1.02e0 +RESULT attn 256 rows dv: worst column 8.84e-3 (column 107 of 512), relative L2 2.06e-3, largest |ref| 4.29e0 +RESULT attn 256 rows dx: worst column 1.03e-2 (column 734 of 2560), relative L2 3.07e-3, largest |ref| 2.47e0 +RESULT attn 256 rows: 3.697 ms on the GPU, 49.5 MiB of gradients and scratch +PASS attn-back: 11 checks within tolerance (1.6 s with the float64 references) +RESULT fd hc block dh: float64 reference against central differences at 10 coordinates, worst relative 6.35e-8 +RESULT fd hc block dbranch: float64 reference against central differences at 6 coordinates, worst relative 2.55e-10 +RESULT fd hc final mixer dh: float64 reference against central differences at 10 coordinates, worst relative 9.80e-9 +RESULT hc weights: 4.0 MiB a read-out +RESULT hc block 64 rows dbranch: worst column 1.48e-7 (column 1547 of 2560), relative L2 4.02e-8, largest |ref| 5.27e0 +RESULT hc block 64 rows dinject: worst column 9.96e-8 (column 3 of 4), relative L2 8.78e-8, largest |ref| 5.45e1 +RESULT hc block 64 rows dh: worst column 5.82e-3 (column 429 of 10240), relative L2 1.12e-3, largest |ref| 4.82e0 +RESULT hc block 64 rows dh less the residual: worst column 7.39e-3 (column 9779 of 10240), relative L2 2.10e-3, largest |ref| 4.18e0 +RESULT hc block 64 rows: 0.664 ms on the GPU, 8.9 MiB of gradients and scratch +RESULT hc final mixer 64 rows dh: worst column 4.65e-3 (column 8916 of 10240), relative L2 1.68e-3, largest |ref| 2.02e0 +RESULT hc final mixer 64 rows: 0.755 ms on the GPU, 8.9 MiB of gradients and scratch +RESULT hc block 256 rows dbranch: worst column 1.21e-7 (column 2302 of 2560), relative L2 4.03e-8, largest |ref| 5.90e0 +RESULT hc block 256 rows dinject: worst column 1.30e-7 (column 3 of 4), relative L2 9.54e-8, largest |ref| 5.45e1 +RESULT hc block 256 rows dh: worst column 5.06e-3 (column 786 of 10240), relative L2 1.11e-3, largest |ref| 4.82e0 +RESULT hc block 256 rows dh less the residual: worst column 6.49e-3 (column 7869 of 10240), relative L2 2.06e-3, largest |ref| 4.19e0 +RESULT hc block 256 rows: 1.271 ms on the GPU, 35.6 MiB of gradients and scratch +RESULT hc final mixer 256 rows dh: worst column 3.82e-3 (column 2455 of 10240), relative L2 1.68e-3, largest |ref| 2.02e0 +RESULT hc final mixer 256 rows: 1.057 ms on the GPU, 35.6 MiB of gradients and scratch +PASS fn-hc-back: 13 checks within tolerance (0.7 s with the float64 references) +RESULT fd moe dx: float64 reference against central differences at 8 coordinates, worst relative 3.22e-9 +RESULT fd moe pocket output vectors: float64 reference against central differences at 4 coordinates, worst relative 2.31e-12 +RESULT moe weights: 5513.8 MiB (512 routed experts as MLX stacks and packed, the shared expert, the router, a pocket of 16) +RESULT moe 64 rows dx: worst column 3.29e-6 (column 2351 of 2560), relative L2 1.11e-6, largest |ref| 8.04e0 +RESULT moe 64 rows slot scores a = dL/dw: worst column 1.55e-6 (column 2 of 11), relative L2 1.12e-6, largest |ref| 1.22e2 +RESULT moe 64 rows router logits' gradients: worst column 1.51e-6 (column 2 of 11), relative L2 1.09e-6, largest |ref| 1.93e1 +RESULT moe 64 rows pocket forward: worst column 7.30e-7 (column 2325 of 2560), relative L2 2.69e-7, largest |ref| 1.04e0 +RESULT moe 64 rows pocket output vectors' gradients: worst column 1.36e-6 (column 791 of 2560), relative L2 2.97e-7, largest |ref| 1.00e0 +RESULT moe 64 rows: 17.633 ms on the GPU, 16.2 MiB of gradients and scratch +RESULT moe 64 rows packed dx: worst column 3.29e-6 (column 2351 of 2560), relative L2 1.11e-6, largest |ref| 8.04e0 +RESULT moe 64 rows packed: 14.127 ms, dx equals the MLX stacks' bits +RESULT moe 256 rows dx: worst column 2.53e-6 (column 443 of 2560), relative L2 1.13e-6, largest |ref| 1.02e1 +RESULT moe 256 rows slot scores a = dL/dw: worst column 1.83e-6 (column 9 of 11), relative L2 1.15e-6, largest |ref| 1.22e2 +RESULT moe 256 rows router logits' gradients: worst column 1.42e-6 (column 2 of 11), relative L2 1.14e-6, largest |ref| 2.27e1 +RESULT moe 256 rows pocket forward: worst column 5.27e-7 (column 892 of 2560), relative L2 2.68e-7, largest |ref| 1.06e0 +RESULT moe 256 rows pocket output vectors' gradients: worst column 1.14e-6 (column 1502 of 2560), relative L2 3.51e-7, largest |ref| 5.15e0 +RESULT moe 256 rows: 27.990 ms on the GPU, 64.9 MiB of gradients and scratch +RESULT moe 256 rows packed dx: worst column 2.53e-6 (column 443 of 2560), relative L2 1.13e-6, largest |ref| 1.02e1 +RESULT moe 256 rows packed: 21.020 ms, dx equals the MLX stacks' bits +PASS fn-moe-back: 16 checks within tolerance (8.0 s with the float64 references) +PASS fn-pocket: 444 cases byte-equal to tf_fn_pocket_rows (1 to 2048 rows, 1 to 128 neurons) +PASS fn-pocket controls: neurons in reverse change 12703 and 212652 values, inputs rotated by one 9633 and 153479 +PASS fn-pocket: no neuron in use launches nothing and keeps every output bit, -0.0 included +PASS gradient component suites (FP64 and finite-difference checks); resource receipts still required +PASS tf_moe_router: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_moe_topk: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_moe_combine: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_q36m_norm_rope: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_q36m_attn: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS tf_q36m_attn_tree: served geometry, FP64, five repeats, guards, inputs; finite corruption RED +PASS plan_rows: buckets 1, 16, 32, 64, 114, 128 at every live count (wide past 1,024 pairs) equal the plan over the live pairs, five repeats, guards +RESULT fd mlp dh: float64 reference against central differences at 6 coordinates, worst relative 1.02e-7 +RESULT mlp 64 rows dy (MLP and pocket): worst column 6.93e-3 (column 5116 of 5120), relative L2 3.08e-3, largest |ref| 1.45e1 +RESULT mlp 64 rows dh (through the post norm): worst column 7.00e-3 (column 5116 of 5120), relative L2 3.08e-3, largest |ref| 1.38e1 +RESULT mlp 64 rows pocket output (bf16): worst column 3.88e-3 (column 187 of 5120), relative L2 1.66e-3, largest |ref| 1.96e0 +RESULT mlp 64 rows pocket output gradient: worst column 6.48e-7 (column 1289 of 5120), relative L2 1.98e-7, largest |ref| 2.79e0 +RESULT mlp 256 rows dy (MLP and pocket): worst column 6.00e-3 (column 1334 of 5120), relative L2 3.07e-3, largest |ref| 1.46e1 +RESULT mlp 256 rows dh (through the post norm): worst column 6.00e-3 (column 5116 of 5120), relative L2 3.07e-3, largest |ref| 1.60e1 +RESULT mlp 256 rows pocket output (bf16): worst column 3.86e-3 (column 2234 of 5120), relative L2 1.66e-3, largest |ref| 2.64e0 +RESULT mlp 256 rows pocket output gradient: worst column 5.18e-7 (column 4774 of 5120), relative L2 2.46e-7, largest |ref| 2.11e0 +RESULT fd attn dx: float64 reference against central differences at 6 coordinates, worst relative 4.46e-8 +RESULT attn 64 rows prepared heads: the largest difference 7.81e-3 of max(1, |value|) +RESULT attn 64 rows dpq (q and gate): worst column 1.46e-2 (column 2445 of 12288), relative L2 2.06e-3, largest |ref| 8.65e-1 +RESULT attn 64 rows dpk: worst column 6.95e-3 (column 1000 of 1024), relative L2 2.02e-3, largest |ref| 1.37e0 +RESULT attn 64 rows dv: worst column 1.11e-2 (column 1010 of 1024), relative L2 2.09e-3, largest |ref| 4.38e0 +RESULT attn 64 rows dx: worst column 1.67e-2 (column 2129 of 5120), relative L2 3.07e-3, largest |ref| 2.49e0 +RESULT attn 256 rows prepared heads: the largest difference 7.81e-3 of max(1, |value|) +RESULT attn 256 rows dpq (q and gate): worst column 1.46e-2 (column 2445 of 12288), relative L2 2.09e-3, largest |ref| 8.65e-1 +RESULT attn 256 rows dpk: worst column 6.56e-3 (column 674 of 1024), relative L2 2.07e-3, largest |ref| 1.37e0 +RESULT attn 256 rows dv: worst column 1.01e-2 (column 640 of 1024), relative L2 2.10e-3, largest |ref| 4.47e0 +RESULT attn 256 rows dx: worst column 1.73e-2 (column 4503 of 5120), relative L2 3.07e-3, largest |ref| 2.55e0 +RESULT fd final norm dh: float64 reference against central differences at 6 coordinates, worst relative 5.07e-8 +RESULT head 64 rows dlast (head back): worst column 3.96e-3 (column 4322 of 5120), relative L2 1.50e-3, largest |ref| 2.11e-4 +RESULT head 64 rows dh (final norm): worst column 3.97e-3 (column 4330 of 5120), relative L2 1.50e-3, largest |ref| 2.33e-4 +RESULT head 256 rows dlast (head back): worst column 3.94e-3 (column 270 of 5120), relative L2 1.51e-3, largest |ref| 2.26e-4 +RESULT head 256 rows dh (final norm): worst column 3.96e-3 (column 1571 of 5120), relative L2 1.51e-3, largest |ref| 2.47e-4 +PASS q27-back all: 25 checks within tolerance (19.4 s with the float64 references) +RESULT q27-gemm [984, 17408] x 4-bit [17408, 5120]: 64x64 11.34 ms (15.5 TFLOPS), wide 3.70 ms (47.5 TFLOPS) +RESULT q27-gemm [984, 5120] x 4-bit [5120, 17408]: 64x64 11.39 ms (15.4 TFLOPS), wide 3.67 ms (47.7 TFLOPS) +RESULT q27-gemm [984, 12288] x 4-bit [12288, 5120]: 64x64 7.84 ms (15.8 TFLOPS), wide 2.61 ms (47.4 TFLOPS) +RESULT q27-gemm [984, 1024] x 4-bit [1024, 5120]: 64x64 0.65 ms (15.9 TFLOPS), wide 0.25 ms (40.8 TFLOPS) +RESULT q27-gemm [984, 5120] x 4-bit [5120, 6144]: 64x64 3.80 ms (16.3 TFLOPS), wide 1.30 ms (47.6 TFLOPS) +PASS q27-gemm: 181 cases byte for byte (wide tiles against 64x64 and against the DeltaNet's dequantized GEMM, gathered rows against whole) +RESULT gdn-back finite differences (6 rows, 2 key / 6 value heads): 120 coordinates of P, worst |fd - analytic| / max(|analytic|, 1e-3 max) 7.91e-7 +RESULT gdn-back finite differences (9 rows, 1 key / 3 value heads): 120 coordinates of P, worst |fd - analytic| / max(|analytic|, 1e-3 max) 1.43e-6 +--- gdn-back: 64 rows, after a prefix (conv history and state) +RESULT gdn-back time 64 rows: layer backward 5.172 ms (forward for training, chain and gates 1.052 ms, the rest the two projections' dequant and GEMM); scratch 247.6 MiB (checkpoints 48.0, replay 12.0, dequantized projection 160.9) +RESULT gdn-back steps 64 rows (ms): front 0.014, scan_fwd 0.268, norm_back 0.039, scan_back 0.673, qk_back 0.053, gates_back 0.004, conv_back 0.029 +RESULT gdn-back forward act 3 of 655360 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 3072 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 50 of 393216 bf16 values differ from float64's rounding, at most 2 ulp +RESULT gdn-back forward q worst column rel 1.98e-7 (col 1743), max abs 5.67e-9, ref scale 5.04e-2 +RESULT gdn-back forward k worst column rel 1.71e-7 (col 1920), max abs 6.66e-8, ref scale 6.00e-1 +RESULT gdn-back forward g worst column rel 9.24e-8 (col 27), max abs 8.79e-8, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 1.35e-3 (col 2595), max abs 7.63e-6, ref scale 6.30e-2 +RESULT gdn-back dout (o_proj back) worst column rel 5.04e-3 (col 860), max abs 1.28e-4, ref scale 4.79e-2 +RESULT gdn-back dout, bf16 operands worst column rel 9.71e-6 (col 3003), max abs 2.89e-7, ref scale 4.79e-2 +RESULT gdn-back dy (norm back) worst column rel 2.72e-7 (col 3697), max abs 6.90e-6, ref scale 4.66e1 +RESULT gdn-back dz worst column rel 8.70e-7 (col 411), max abs 6.54e-9, ref scale 4.32e-2 +RESULT gdn-back dq (chain) worst column rel 7.07e-7 (col 1766), max abs 4.92e-6, ref scale 2.78e1 +RESULT gdn-back dk (chain) worst column rel 7.29e-7 (col 1544), max abs 3.90e-7, ref scale 3.28e0 +RESULT gdn-back dv (chain) worst column rel 9.06e-7 (col 2625), max abs 4.27e-8, ref scale 2.05e-1 +RESULT gdn-back dg (chain) worst column rel 9.23e-7 (col 16), max abs 2.29e-7, ref scale 6.83e-1 +RESULT gdn-back dbeta (chain) worst column rel 3.65e-7 (col 45), max abs 1.26e-7, ref scale 8.15e-1 +RESULT gdn-back dact q (L2 back) worst column rel 5.46e-7 (col 393), max abs 9.10e-8, ref scale 5.62e-1 +RESULT gdn-back dact k (L2 back) worst column rel 5.80e-7 (col 787), max abs 1.15e-7, ref scale 8.97e-1 +RESULT gdn-back dproj q (conv back) worst column rel 1.00e-6 (col 1986), max abs 3.65e-8, ref scale 1.94e-1 +RESULT gdn-back dproj k (conv back) worst column rel 9.57e-7 (col 1795), max abs 5.86e-8, ref scale 2.39e-1 +RESULT gdn-back dproj v (conv back) worst column rel 1.07e-6 (col 4468), max abs 2.04e-8, ref scale 8.02e-2 +RESULT gdn-back dproj b (beta back) worst column rel 5.17e-7 (col 31), max abs 2.30e-8, ref scale 8.67e-2 +RESULT gdn-back dproj a (g back) worst column rel 8.71e-7 (col 38), max abs 3.00e-8, ref scale 7.45e-2 +RESULT gdn-back dx (in_proj back) worst column rel 5.15e-3 (col 4114), max abs 2.34e-4, ref scale 1.10e-1 +RESULT gdn-back dx, bf16 operands worst column rel 3.99e-5 (col 168), max abs 2.84e-6, ref scale 1.10e-1 +--- gdn-back: 256 rows, from the sequence start +RESULT gdn-back time 256 rows: layer backward 8.739 ms (forward for training, chain and gates 3.994 ms, the rest the two projections' dequant and GEMM); scratch 471.5 MiB (checkpoints 192.0, replay 12.0, dequantized projection 160.9) +RESULT gdn-back steps 256 rows (ms): front 0.062, scan_fwd 1.058, norm_back 0.134, scan_back 2.378, qk_back 0.271, gates_back 0.006, conv_back 0.096 +RESULT gdn-back forward act 21 of 2621440 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 12288 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 192 of 1572864 bf16 values differ from float64's rounding, at most 5 ulp +RESULT gdn-back forward q worst column rel 1.94e-7 (col 1588), max abs 6.53e-9, ref scale 5.33e-2 +RESULT gdn-back forward k worst column rel 1.59e-7 (col 1784), max abs 8.37e-8, ref scale 6.51e-1 +RESULT gdn-back forward g worst column rel 1.15e-7 (col 46), max abs 1.11e-7, ref scale 9.96e-1 +RESULT gdn-back forward y worst column rel 4.00e-3 (col 2186), max abs 6.10e-5, ref scale 5.64e-2 +RESULT gdn-back dout (o_proj back) worst column rel 5.91e-3 (col 3943), max abs 1.11e-4, ref scale 4.96e-2 +RESULT gdn-back dout, bf16 operands worst column rel 1.02e-5 (col 4059), max abs 3.01e-7, ref scale 4.96e-2 +RESULT gdn-back dy (norm back) worst column rel 3.20e-7 (col 4726), max abs 7.90e-6, ref scale 5.12e1 +RESULT gdn-back dz worst column rel 6.40e-7 (col 1694), max abs 8.72e-9, ref scale 4.92e-2 +RESULT gdn-back dq (chain) worst column rel 5.21e-7 (col 284), max abs 4.45e-6, ref scale 2.89e1 +RESULT gdn-back dk (chain) worst column rel 5.28e-7 (col 402), max abs 3.32e-7, ref scale 2.36e0 +RESULT gdn-back dv (chain) worst column rel 6.85e-7 (col 5917), max abs 6.13e-8, ref scale 2.33e-1 +RESULT gdn-back dg (chain) worst column rel 7.49e-7 (col 28), max abs 1.63e-7, ref scale 8.87e-1 +RESULT gdn-back dbeta (chain) worst column rel 4.28e-7 (col 39), max abs 1.79e-7, ref scale 9.67e-1 +RESULT gdn-back dact q (L2 back) worst column rel 7.44e-7 (col 1499), max abs 1.14e-7, ref scale 7.52e-1 +RESULT gdn-back dact k (L2 back) worst column rel 6.27e-7 (col 402), max abs 9.08e-8, ref scale 6.36e-1 +RESULT gdn-back dproj q (conv back) worst column rel 6.18e-7 (col 380), max abs 4.92e-8, ref scale 2.30e-1 +RESULT gdn-back dproj k (conv back) worst column rel 7.92e-7 (col 1801), max abs 4.46e-8, ref scale 2.12e-1 +RESULT gdn-back dproj v (conv back) worst column rel 9.56e-7 (col 6112), max abs 2.74e-8, ref scale 9.35e-2 +RESULT gdn-back dproj b (beta back) worst column rel 3.80e-7 (col 3), max abs 2.42e-8, ref scale 1.22e-1 +RESULT gdn-back dproj a (g back) worst column rel 6.33e-7 (col 28), max abs 3.23e-8, ref scale 8.40e-2 +RESULT gdn-back dx (in_proj back) worst column rel 6.78e-3 (col 3513), max abs 2.64e-4, ref scale 1.13e-1 +RESULT gdn-back dx, bf16 operands worst column rel 4.36e-5 (col 4893), max abs 2.75e-6, ref scale 1.13e-1 +--- gdn-back: 37 rows, after a prefix (conv history and state) +RESULT gdn-back time 37 rows: layer backward 4.596 ms (forward for training, chain and gates 0.603 ms, the rest the two projections' dequant and GEMM); scratch 218.3 MiB (checkpoints 30.0, replay 12.0, dequantized projection 160.9) +RESULT gdn-back steps 37 rows (ms): front 0.011, scan_fwd 0.167, norm_back 0.024, scan_back 0.384, qk_back 0.024, gates_back 0.004, conv_back 0.018 +RESULT gdn-back forward act 1 of 378880 bf16 values differ from float64's rounding, at most 1 ulp +RESULT gdn-back forward beta 0 of 1776 bf16 values differ from float64's rounding, at most 0 ulp +RESULT gdn-back forward y 28 of 227328 bf16 values differ from float64's rounding, at most 26 ulp +RESULT gdn-back forward q worst column rel 1.68e-7 (col 1069), max abs 7.32e-9, ref scale 5.34e-2 +RESULT gdn-back forward k worst column rel 1.54e-7 (col 1790), max abs 5.85e-8, ref scale 6.13e-1 +RESULT gdn-back forward g worst column rel 1.10e-7 (col 47), max abs 1.09e-7, ref scale 9.94e-1 +RESULT gdn-back forward y worst column rel 1.19e-3 (col 6101), max abs 1.53e-5, ref scale 5.57e-2 +RESULT gdn-back dout (o_proj back) worst column rel 1.27e-2 (col 5249), max abs 1.02e-4, ref scale 4.46e-2 +RESULT gdn-back dout, bf16 operands worst column rel 1.90e-5 (col 5249), max abs 3.17e-7, ref scale 4.47e-2 +RESULT gdn-back dy (norm back) worst column rel 2.51e-7 (col 455), max abs 5.20e-6, ref scale 4.33e1 +RESULT gdn-back dz worst column rel 1.23e-6 (col 1276), max abs 4.89e-9, ref scale 3.61e-2 +RESULT gdn-back dq (chain) worst column rel 8.41e-7 (col 1089), max abs 2.93e-6, ref scale 1.54e1 +RESULT gdn-back dk (chain) worst column rel 1.08e-6 (col 1755), max abs 2.55e-7, ref scale 1.50e0 +RESULT gdn-back dv (chain) worst column rel 7.96e-7 (col 3048), max abs 4.30e-8, ref scale 1.82e-1 +RESULT gdn-back dg (chain) worst column rel 8.78e-7 (col 41), max abs 2.06e-7, ref scale 1.39e0 +RESULT gdn-back dbeta (chain) worst column rel 6.15e-7 (col 10), max abs 1.42e-7, ref scale 7.35e-1 +RESULT gdn-back dact q (L2 back) worst column rel 7.59e-7 (col 1089), max abs 8.93e-8, ref scale 3.96e-1 +RESULT gdn-back dact k (L2 back) worst column rel 1.15e-6 (col 1755), max abs 1.30e-7, ref scale 4.26e-1 +RESULT gdn-back dproj q (conv back) worst column rel 1.08e-6 (col 1527), max abs 2.46e-8, ref scale 1.24e-1 +RESULT gdn-back dproj k (conv back) worst column rel 1.12e-6 (col 1755), max abs 3.79e-8, ref scale 1.11e-1 +RESULT gdn-back dproj v (conv back) worst column rel 9.21e-7 (col 3048), max abs 2.27e-8, ref scale 9.31e-2 +RESULT gdn-back dproj b (beta back) worst column rel 5.12e-7 (col 10), max abs 1.78e-8, ref scale 8.92e-2 +RESULT gdn-back dproj a (g back) worst column rel 1.00e-6 (col 39), max abs 3.63e-8, ref scale 1.02e-1 +RESULT gdn-back dx (in_proj back) worst column rel 9.84e-3 (col 3358), max abs 1.99e-4, ref scale 8.87e-2 +RESULT gdn-back dx, bf16 operands worst column rel 5.18e-5 (col 103), max abs 2.52e-6, ref scale 8.87e-2 +PASS gdn-back: forward for training and every input gradient within tolerance of float64 +qwen35 tf_q35_gdn_slots_1 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_1 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_1 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_1 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_1 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_2 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_4 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_8 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_slots_16 on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_chain on sm_121: no compiled usage recorded +qwen35 tf_q35_rope on sm_80: no compiled usage recorded +qwen35 tf_q35_rope on sm_86: no compiled usage recorded +qwen35 tf_q35_rope on sm_89: no compiled usage recorded +qwen35 tf_q35_rope on sm_120: no compiled usage recorded +qwen35 tf_q35_rope on sm_121: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_80: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_86: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_89: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_120: no compiled usage recorded +qwen35 tf_q35_attn_partial on sm_121: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_80: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_86: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_89: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_120: no compiled usage recorded +qwen35 tf_q35_attn_merge on sm_121: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_80: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_86: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_89: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_120: no compiled usage recorded +qwen35_kv8 tf_q35_rope8 on sm_121: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_80: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_86: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_89: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_120: no compiled usage recorded +qwen35_kv8 tf_q35_attn_partial8 on sm_121: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_80: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_86: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_89: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_120: no compiled usage recorded +qwen35_keep tf_q35k_apply on sm_121: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_80: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_86: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_89: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_120: no compiled usage recorded +qwen35_keep tf_q35k_compact on sm_121: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_80: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_86: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_89: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_120: no compiled usage recorded +qwen35_kv8 tf_q35k_compact8 on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_gather_rows on sm_121: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_80: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_86: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_89: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_120: no compiled usage recorded +qwen35 tf_q35_gdn_pre on sm_121: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_80: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_86: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_89: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_120: no compiled usage recorded +qwen35 tf_q35_conv_commit on sm_121: no compiled usage recorded +qwen35 tf_q35_tap on sm_80: no compiled usage recorded +qwen35 tf_q35_tap on sm_86: no compiled usage recorded +qwen35 tf_q35_tap on sm_89: no compiled usage recorded +qwen35 tf_q35_tap on sm_120: no compiled usage recorded +qwen35 tf_q35_tap on sm_121: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_80: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_86: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_89: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_120: no compiled usage recorded +qwen35_keep tf_q35k_conv on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_dyn_conv_batch on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_norm_rope_batch on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_mask_rows on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_attn_partial on sm_121: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_80: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_86: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_89: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_120: no compiled usage recorded +qwen35_draft tf_q35d_attn_merge on sm_121: no compiled usage recorded +tf_topk_slices on sm_80: no compiled usage recorded +tf_topk_slices on sm_86: no compiled usage recorded +tf_topk_slices on sm_89: no compiled usage recorded +tf_topk_slices on sm_120: no compiled usage recorded +tf_topk_slices on sm_121: no compiled usage recorded +tf_topk_merge on sm_80: no compiled usage recorded +tf_topk_merge on sm_86: no compiled usage recorded +tf_topk_merge on sm_89: no compiled usage recorded +tf_topk_merge on sm_120: no compiled usage recorded +tf_topk_merge on sm_121: no compiled usage recorded +tf_q35_norm on sm_80: no compiled usage recorded +tf_q35_norm on sm_86: no compiled usage recorded +tf_q35_norm on sm_89: no compiled usage recorded +tf_q35_norm on sm_120: no compiled usage recorded +tf_q35_norm on sm_121: no compiled usage recorded +tf_q35_sums on sm_80: no compiled usage recorded +tf_q35_sums on sm_86: no compiled usage recorded +tf_q35_sums on sm_89: no compiled usage recorded +tf_q35_sums on sm_120: no compiled usage recorded +tf_q35_sums on sm_121: no compiled usage recorded +tf_q35_argmax_slices on sm_80: no compiled usage recorded +tf_q35_argmax_slices on sm_86: no compiled usage recorded +tf_q35_argmax_slices on sm_89: no compiled usage recorded +tf_q35_argmax_slices on sm_120: no compiled usage recorded +tf_q35_argmax_slices on sm_121: no compiled usage recorded +tf_q35_argmax_merge on sm_80: no compiled usage recorded +tf_q35_argmax_merge on sm_86: no compiled usage recorded +tf_q35_argmax_merge on sm_89: no compiled usage recorded +tf_q35_argmax_merge on sm_120: no compiled usage recorded +tf_q35_argmax_merge on sm_121: no compiled usage recorded +sample tf_draw on sm_80: no compiled usage recorded +sample tf_draw on sm_86: no compiled usage recorded +sample tf_draw on sm_89: no compiled usage recorded +sample tf_draw on sm_120: no compiled usage recorded +sample tf_draw on sm_121: no compiled usage recorded +sample tf_logprob_rows on sm_80: no compiled usage recorded +sample tf_logprob_rows on sm_86: no compiled usage recorded +sample tf_logprob_rows on sm_89: no compiled usage recorded +sample tf_logprob_rows on sm_120: no compiled usage recorded +sample tf_logprob_rows on sm_121: no compiled usage recorded +sample tf_grammar_mask on sm_80: no compiled usage recorded +sample tf_grammar_mask on sm_86: no compiled usage recorded +sample tf_grammar_mask on sm_89: no compiled usage recorded +sample tf_grammar_mask on sm_120: no compiled usage recorded +sample tf_grammar_mask on sm_121: no compiled usage recorded +glm_glue tf_glm_top1 on sm_80: no compiled usage recorded +glm_glue tf_glm_top1 on sm_86: no compiled usage recorded +glm_glue tf_glm_top1 on sm_89: no compiled usage recorded +glm_glue tf_glm_top1 on sm_120: no compiled usage recorded +glm_glue tf_glm_top1 on sm_121: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_80: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_86: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_89: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_120: no compiled usage recorded +glm_glue tf_glm_lat_write_fp8 on sm_121: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_80: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_86: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_89: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_120: no compiled usage recorded +glm_glue tf_glm_chunks_16_fp8 on sm_121: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_80: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_86: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_89: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_120: no compiled usage recorded +glm_glue tf_glm_chunks_32_fp8 on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10 on sm_121: no compiled usage recorded +dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12 on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii on sm_121: no compiled usage recorded +dsv41_router_glue router_group_rot on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii on sm_121: no compiled usage recorded +dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +N/A dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: optional shared-memory limit +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_ on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii on sm_121: no compiled usage recorded +dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii on sm_121: no compiled usage recorded +dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE on sm_121: no compiled usage recorded +dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE on sm_121: no compiled usage recorded +dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi on sm_121: no compiled usage recorded +dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi on sm_121: no compiled usage recorded +dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm on sm_121: no compiled usage recorded +dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij on sm_121: no compiled usage recorded +dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij on sm_121: no compiled usage recorded +dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx on sm_121: no compiled usage recorded +dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif on sm_121: no compiled usage recorded +dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx on sm_121: no compiled usage recorded +dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE on sm_121: no compiled usage recorded +dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_ on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_ on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii on sm_121: no compiled usage recorded +dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf on sm_121: no compiled usage recorded +dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx on sm_121: no compiled usage recorded +dsv41_kv_glue kv_norm_store on sm_121: no compiled usage recorded +dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd on sm_121: no compiled usage recorded +dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf on sm_121: no compiled usage recorded +dsv41_sampling dsv41_sampling_stats_max on sm_121: no compiled usage recorded +dsv41_sampling dsv41_sampling_stats_sum on sm_121: no compiled usage recorded +dsv41_sampling dsv41_sampling_stats_fin on sm_121: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_80: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_86: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_89: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_120: no compiled usage recorded +tree_round tf_round_bf16_argmax on sm_121: no compiled usage recorded +tree_round tf_tree_round_match on sm_80: no compiled usage recorded +tree_round tf_tree_round_match on sm_86: no compiled usage recorded +tree_round tf_tree_round_match on sm_89: no compiled usage recorded +tree_round tf_tree_round_match on sm_120: no compiled usage recorded +tree_round tf_tree_round_match on sm_121: no compiled usage recorded +tree_round tf_round_keep_rows on sm_80: no compiled usage recorded +tree_round tf_round_keep_rows on sm_86: no compiled usage recorded +tree_round tf_round_keep_rows on sm_89: no compiled usage recorded +tree_round tf_round_keep_rows on sm_120: no compiled usage recorded +tree_round tf_round_keep_rows on sm_121: no compiled usage recorded +tree_round tf_round_gather_rows on sm_80: no compiled usage recorded +tree_round tf_round_gather_rows on sm_86: no compiled usage recorded +tree_round tf_round_gather_rows on sm_89: no compiled usage recorded +tree_round tf_round_gather_rows on sm_120: no compiled usage recorded +tree_round tf_round_gather_rows on sm_121: no compiled usage recorded +tree_round tf_round_keep_head on sm_80: no compiled usage recorded +tree_round tf_round_keep_head on sm_86: no compiled usage recorded +tree_round tf_round_keep_head on sm_89: no compiled usage recorded +tree_round tf_round_keep_head on sm_120: no compiled usage recorded +tree_round tf_round_keep_head on sm_121: no compiled usage recorded +act_rows tf_swiglu_rows on sm_80: no compiled usage recorded +act_rows tf_swiglu_rows on sm_86: no compiled usage recorded +act_rows tf_swiglu_rows on sm_89: no compiled usage recorded +act_rows tf_swiglu_rows on sm_120: no compiled usage recorded +act_rows tf_swiglu_rows on sm_121: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_80: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_86: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_89: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_120: no compiled usage recorded +act_rows tf_swiglu_rows, 16 rows or fewer on sm_121: no compiled usage recorded +act_rows tf_gate_rows on sm_80: no compiled usage recorded +act_rows tf_gate_rows on sm_86: no compiled usage recorded +act_rows tf_gate_rows on sm_89: no compiled usage recorded +act_rows tf_gate_rows on sm_120: no compiled usage recorded +act_rows tf_gate_rows on sm_121: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_80: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_86: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_89: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_120: no compiled usage recorded +act_rows tf_gate_rows, 16 rows or fewer on sm_121: no compiled usage recorded +act_rows tf_group_sums_rows on sm_80: no compiled usage recorded +act_rows tf_group_sums_rows on sm_86: no compiled usage recorded +act_rows tf_group_sums_rows on sm_89: no compiled usage recorded +act_rows tf_group_sums_rows on sm_120: no compiled usage recorded +act_rows tf_group_sums_rows on sm_121: no compiled usage recorded +act_rows tf_dconv_rows on sm_80: no compiled usage recorded +act_rows tf_dconv_rows on sm_86: no compiled usage recorded +act_rows tf_dconv_rows on sm_89: no compiled usage recorded +act_rows tf_dconv_rows on sm_120: no compiled usage recorded +act_rows tf_dconv_rows on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_64 on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_64, 16 rows or fewer on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_128 on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_128, 16 rows or fewer on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_256 on sm_121: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_80: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_86: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_89: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_120: no compiled usage recorded +gdn_prep tf_gdn_prep_256, 16 rows or fewer on sm_121: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_80: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_86: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_89: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_120: no compiled usage recorded +qk_rows tf_qk_rows at dim 64 on sm_121: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_80: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_86: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_89: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_120: no compiled usage recorded +qk_rows tf_qk_rows at dim 256 on sm_121: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_80: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_86: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_89: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_120: no compiled usage recorded +qk_rows tf_qk_rows_w on sm_121: no compiled usage recorded +FAIL limits: 1167 registered CUDA launches against sm_80/86/89/120/121, 885 problems +PASS limits live sm_121 most shared bytes a block (opt-in): the driver says 101376, the table 101376 +PASS limits live sm_121 threads a block: the driver says 1024, the table 1024 +INFO limits live: images built by nvcc 13.3, checked against nvcc 13.3's records +FAIL limits live flashnext_gdn flashnext_gdn_chain: 1024 threads launched, the driver allows 1024; 61 registers, 2060 + 0 shared bytes (recorded 64, 2060) +PASS limits live flashnext_hc fn_add_streams: 256 threads launched, the driver allows 1024; 35 registers, 0 + 0 shared bytes (recorded 35, 0) +PASS limits live flashnext_attn fn_attn_chunks: 128 threads launched, the driver allows 128; 72 registers, 640 + 43520 shared bytes (recorded 72, 640) +PASS limits live flashnext_attn fn_attn_chunks_ahead: 128 threads launched, the driver allows 128; 72 registers, 768 + 77312 shared bytes (recorded 72, 768) +PASS limits live flashnext_attn fn_attn_gate: 256 threads launched, the driver allows 256; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +FAIL limits live flashnext_attn fn_attn_merge: 256 threads launched, the driver allows 256; 25 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live flashnext_attn fn_attn_prep: 256 threads launched, the driver allows 256; 30 registers, 128 + 0 shared bytes (recorded 30, 128) +PASS limits live flashnext_batch fn_gdn_front: 256 threads launched, the driver allows 256; 30 registers, 1024 + 0 shared bytes (recorded 30, 1024) +PASS limits live flashnext_batch fn_gdn_norm: 128 threads launched, the driver allows 128; 24 registers, 528 + 0 shared bytes (recorded 24, 528) +PASS limits live flashnext_batch fn_gdn_rows: 128 threads launched, the driver allows 128; 64 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live flashnext_batch fn_gdn_rows_to: 128 threads launched, the driver allows 128; 64 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live flashnext_hc fn_hc_dense_mix: 32 threads launched, the driver allows 1024; 36 registers, 128 + 0 shared bytes (recorded 36, 128) +PASS limits live flashnext_hc fn_hc_down: 128 threads launched, the driver allows 128; 64 registers, 64 + 11136 shared bytes (recorded 64, 64) +PASS limits live flashnext_hc fn_hc_normalize: 256 threads launched, the driver allows 1024; 40 registers, 16 + 0 shared bytes (recorded 40, 16) +PASS limits live flashnext_hc fn_hc_reduce_act: 352 threads launched, the driver allows 1024; 60 registers, 4096 + 0 shared bytes (recorded 60, 4096) +PASS limits live flashnext_hc fn_hc_upmix: 256 threads launched, the driver allows 256; 62 registers, 2048 + 18688 shared bytes (recorded 62, 2048) +PASS limits live flashnext_hc fn_hc_writeback0: 256 threads launched, the driver allows 256; 37 registers, 256 + 0 shared bytes (recorded 37, 256) +PASS limits live flashnext_hc fn_hc_writeback_moe: 256 threads launched, the driver allows 256; 47 registers, 256 + 0 shared bytes (recorded 47, 256) +PASS limits live flashnext_hc fn_hc_writeback_slices: 256 threads launched, the driver allows 256; 40 registers, 256 + 0 shared bytes (recorded 40, 256) +PASS limits live flashnext_hc fn_ple_conv: 512 threads launched, the driver allows 512; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live flashnext_hc fn_ple_gate: 512 threads launched, the driver allows 512; 33 registers, 128 + 0 shared bytes (recorded 33, 128) +PASS limits live flashnext_attn fn_qsa_pool: 128 threads launched, the driver allows 128; 39 registers, 1152 + 0 shared bytes (recorded 39, 1152) +PASS limits live flashnext_attn fn_qsa_scores: 256 threads launched, the driver allows 256; 48 registers, 4096 + 0 shared bytes (recorded 48, 4096) +PASS limits live flashnext_batch fn_qsa_select: 1024 threads launched, the driver allows 1024; 44 registers, 1168 + 0 shared bytes (recorded 44, 1168) +PASS limits live flashnext_mtp fn_row_best: 1024 threads launched, the driver allows 1024; 21 registers, 392 + 0 shared bytes (recorded 21, 392) +PASS limits live flashnext_hc fn_shared_scatter: 256 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live flashnext_hc fn_shared_swiglu: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live flashnext_batch fn_shift_window: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live flashnext_batch fn_sum_slices: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live embed_rows tf_embed_rows_bf16_copies: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live norms tf_rms_rows_sums: 1024 threads launched, the driver allows 1024; 34 registers, 132 + 0 shared bytes (recorded 34, 132) +PASS limits live route_rows tf_route_logits: 128 threads launched, the driver allows 128; 35 registers, 0 + 26112 shared bytes (recorded 35, 0) +PASS limits live route_rows tf_route_logits_few: 128 threads launched, the driver allows 128; 34 registers, 0 + 76032 shared bytes (recorded 34, 0) +PASS limits live route_rows tf_route_topk: 128 threads launched, the driver allows 128; 36 registers, 64 + 0 shared bytes (recorded 36, 64) +PASS limits live norms tf_rms_rows_bf16: 256 threads launched, the driver allows 256; 80 registers, 32 + 0 shared bytes (recorded 80, 32) +PASS limits live norms tf_rms_rows: 1024 threads launched, the driver allows 1024; 43 registers, 132 + 0 shared bytes (recorded 43, 132) +PASS limits live router tf_nemo_router: 256 threads launched, the driver allows 256; 40 registers, 0 + 28800 shared bytes (recorded 40, 0) +PASS limits live router tf_nemo_topk: 32 threads launched, the driver allows 32; 96 registers, 0 + 0 shared bytes (recorded 96, 0) +FAIL limits live router tf_nemo_topk_wide: 32 threads launched, the driver allows 32; 161 registers, 0 + 0 shared bytes (recorded 160, 0) +PASS limits live latent tf_super_topk_few: 1024 threads launched, the driver allows 1024; 32 registers, 4352 + 0 shared bytes (recorded 32, 4352) +PASS limits live qmm_group cluster of 8: 128 threads launched, the driver allows 128; 48 registers, 0 + 35328 shared bytes (recorded 48, 0) +PASS limits live qmm_group one CTA a tile: 128 threads launched, the driver allows 128; 48 registers, 0 + 35328 shared bytes (recorded 48, 0) +PASS limits live lane_gemv: 128 threads launched, the driver allows 128; 66 registers, 0 + 35328 shared bytes (recorded 66, 0) +PASS limits live lane_gemv split-K: 128 threads launched, the driver allows 128; 72 registers, 128 + 17664 shared bytes (recorded 72, 128) +PASS limits live prompt16 FP4: 128 threads launched, the driver allows 128; 212 registers, 0 + 41984 shared bytes (recorded 212, 0) +PASS limits live prompt16 FP8: 128 threads launched, the driver allows 128; 212 registers, 0 + 49152 shared bytes (recorded 212, 0) +PASS limits live prompt16 FP4 fp32 out: 128 threads launched, the driver allows 128; 212 registers, 0 + 41984 shared bytes (recorded 212, 0) +PASS limits live qmmf FP4 fp32 out 16-row tile: 128 threads launched, the driver allows 128; 56 registers, 0 + 17408 shared bytes (recorded 56, 0) +PASS limits live qmmf FP4 fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 48 registers, 0 + 17408 shared bytes (recorded 48, 0) +PASS limits live qmmf FP4 fp32 out 32-row tile: 128 threads launched, the driver allows 128; 64 registers, 0 + 25600 shared bytes (recorded 64, 0) +PASS limits live qmmf FP4 fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 61 registers, 0 + 25600 shared bytes (recorded 61, 0) +PASS limits live qmmf FP4 fp32 out 64-row tile: 128 threads launched, the driver allows 128; 78 registers, 0 + 41984 shared bytes (recorded 78, 0) +PASS limits live qmmf FP4 fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 78 registers, 0 + 41984 shared bytes (recorded 78, 0) +PASS limits live qmmf FP4 fp32 out fused slices: 128 threads launched, the driver allows 128; 168 registers, 0 + 41984 shared bytes (recorded 168, 0) +PASS limits live qmmf MXFP8 16-row tile: 128 threads launched, the driver allows 128; 60 registers, 0 + 25088 shared bytes (recorded 60, 0) +PASS limits live qmmf MXFP8 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 56 registers, 0 + 25088 shared bytes (recorded 56, 0) +PASS limits live qmmf MXFP8 32-row tile: 128 threads launched, the driver allows 128; 57 registers, 0 + 33280 shared bytes (recorded 57, 0) +PASS limits live qmmf MXFP8 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 66 registers, 0 + 33280 shared bytes (recorded 66, 0) +PASS limits live qmmf MXFP8 64-row tile: 128 threads launched, the driver allows 128; 108 registers, 0 + 49664 shared bytes (recorded 108, 0) +PASS limits live qmmf MXFP8 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 103 registers, 0 + 49664 shared bytes (recorded 103, 0) +PASS limits live qmmf MXFP8 64-row tile, fused slices: 128 threads launched, the driver allows 128; 158 registers, 0 + 49664 shared bytes (recorded 158, 0) +PASS limits live qmmf reduce_kernel: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qmmf FP8 fp32 out 16-row tile: 128 threads launched, the driver allows 128; 50 registers, 0 + 24576 shared bytes (recorded 50, 0) +PASS limits live qmmf FP8 fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 40 registers, 0 + 24576 shared bytes (recorded 40, 0) +PASS limits live qmmf FP8 fp32 out 32-row tile: 128 threads launched, the driver allows 128; 56 registers, 0 + 32768 shared bytes (recorded 56, 0) +PASS limits live qmmf FP8 fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 48 registers, 0 + 32768 shared bytes (recorded 48, 0) +PASS limits live qmmf FP8 fp32 out 64-row tile: 128 threads launched, the driver allows 128; 72 registers, 0 + 49152 shared bytes (recorded 72, 0) +PASS limits live qmmf FP8 fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 64 registers, 0 + 49152 shared bytes (recorded 64, 0) +PASS limits live qmmf FP8 fp32 out 64-row tile, fused slices: 128 threads launched, the driver allows 128; 140 registers, 0 + 49152 shared bytes (recorded 140, 0) +PASS limits live qmmf MXFP8 fp32 out 16-row tile: 128 threads launched, the driver allows 128; 60 registers, 0 + 25088 shared bytes (recorded 60, 0) +PASS limits live qmmf MXFP8 fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 56 registers, 0 + 25088 shared bytes (recorded 56, 0) +PASS limits live qmmf MXFP8 fp32 out 32-row tile: 128 threads launched, the driver allows 128; 57 registers, 0 + 33280 shared bytes (recorded 57, 0) +PASS limits live qmmf MXFP8 fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 66 registers, 0 + 33280 shared bytes (recorded 66, 0) +PASS limits live qmmf MXFP8 fp32 out 64-row tile: 128 threads launched, the driver allows 128; 108 registers, 0 + 49664 shared bytes (recorded 108, 0) +PASS limits live qmmf MXFP8 fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 103 registers, 0 + 49664 shared bytes (recorded 103, 0) +PASS limits live qmmf MXFP8 fp32 out 64-row tile, fused slices: 128 threads launched, the driver allows 128; 158 registers, 0 + 49664 shared bytes (recorded 158, 0) +PASS limits live qmmf FP8G fp32 out 16-row tile: 128 threads launched, the driver allows 128; 58 registers, 0 + 25600 shared bytes (recorded 58, 0) +PASS limits live qmmf FP8G fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 56 registers, 0 + 25600 shared bytes (recorded 56, 0) +PASS limits live qmmf FP8G fp32 out 32-row tile: 128 threads launched, the driver allows 128; 56 registers, 0 + 33792 shared bytes (recorded 56, 0) +PASS limits live qmmf FP8G fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 57 registers, 0 + 33792 shared bytes (recorded 57, 0) +PASS limits live qmmf FP8G fp32 out 64-row tile: 128 threads launched, the driver allows 128; 108 registers, 0 + 50176 shared bytes (recorded 108, 0) +PASS limits live qmmf FP8G fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 122 registers, 0 + 50176 shared bytes (recorded 122, 0) +PASS limits live qmmf FP8G fp32 out 64-row tile, fused slices: 128 threads launched, the driver allows 128; 155 registers, 0 + 50176 shared bytes (recorded 155, 0) +PASS limits live qmmf BF16 fp32 out 16-row tile: 128 threads launched, the driver allows 128; 48 registers, 0 + 40960 shared bytes (recorded 48, 0) +PASS limits live qmmf BF16 fp32 out 16-row tile, cluster of 8: 128 threads launched, the driver allows 128; 40 registers, 0 + 40960 shared bytes (recorded 40, 0) +PASS limits live qmmf BF16 fp32 out 32-row tile: 128 threads launched, the driver allows 128; 55 registers, 0 + 49152 shared bytes (recorded 55, 0) +PASS limits live qmmf BF16 fp32 out 32-row tile, cluster of 8: 128 threads launched, the driver allows 128; 48 registers, 0 + 49152 shared bytes (recorded 48, 0) +PASS limits live qmmf BF16 fp32 out 64-row tile: 128 threads launched, the driver allows 128; 72 registers, 0 + 65536 shared bytes (recorded 72, 0) +PASS limits live qmmf BF16 fp32 out 64-row tile, cluster of 8: 128 threads launched, the driver allows 128; 64 registers, 0 + 65536 shared bytes (recorded 64, 0) +PASS limits live qmmf BF16 fp32 out 64-row tile, fused slices: 128 threads launched, the driver allows 128; 140 registers, 0 + 65536 shared bytes (recorded 140, 0) +PASS limits live qmmf reduce_kernel: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live ck tf_ck4_up_relu2_few: 128 threads launched, the driver allows 128; 124 registers, 0 + 0 shared bytes (recorded 124, 0) +PASS limits live ck tf_ck4_down_f32_few: 128 threads launched, the driver allows 128; 128 registers, 0 + 0 shared bytes (recorded 128, 0) +FAIL limits live mamba tf_nemo_scan: 128 threads launched, the driver allows 128; 115 registers, 41088 + 0 shared bytes (recorded 118, 41088) +PASS limits live rank_sum tf_rank_sum: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live vit vision_kernel<80, 8> (RADIO): 256 threads launched, the driver allows 256; 122 registers, 0 + 45056 shared bytes (recorded 122, 0) +PASS limits live vit full_kernel<72> (SigLIP2): 256 threads launched, the driver allows 256; 169 registers, 0 + 65536 shared bytes (recorded 169, 0) +PASS limits live vit tf_vit_layer_norm: 256 threads launched, the driver allows 256; 40 registers, 32 + 0 shared bytes (recorded 40, 32) +PASS limits live vit tf_vit_pointwise: 256 threads launched, the driver allows 256; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live vit tf_vit_bias: 256 threads launched, the driver allows 256; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live vit tf_vit_unshuffle: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live qwen_vision tf_qv_embed: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen_vision tf_qv_layernorm: 256 threads launched, the driver allows 256; 23 registers, 132 + 0 shared bytes (recorded 23, 132) +PASS limits live qwen_vision tf_qv_qkv_rope: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen_vision tf_qv_attention72: 64 threads launched, the driver allows 64; 186 registers, 36864 + 0 shared bytes (recorded 186, 36864) +PASS limits live qwen_vision tf_qv_attention64: 64 threads launched, the driver allows 64; 162 registers, 32768 + 0 shared bytes (recorded 162, 32768) +PASS limits live qwen_vision tf_qv_bias_gelu: 256 threads launched, the driver allows 256; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live qwen_vision tf_qv_bias_residual: 256 threads launched, the driver allows 256; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live qwen_vision tf_qv_bias_cast: 256 threads launched, the driver allows 256; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live qwen_vision tf_qv_splice: 256 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live flashnext_attn fn_attn_prep_media: 256 threads launched, the driver allows 256; 30 registers, 128 + 0 shared bytes (recorded 30, 128) +PASS limits live flashnext_attn fn_qsa_pool_media: 128 threads launched, the driver allows 128; 39 registers, 1152 + 0 shared bytes (recorded 39, 1152) +PASS limits live exl3_linear tf_exl3_rot_in: 128 threads launched, the driver allows 128; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live exl3_linear tf_exl3_linear_2_2_4: 128 threads launched, the driver allows 128; 154 registers, 16 + 16384 shared bytes (recorded 154, 16) +PASS limits live exl3_linear tf_exl3_linear_4_2_4: 128 threads launched, the driver allows 128; 154 registers, 16 + 16384 shared bytes (recorded 154, 16) +PASS limits live exl3_linear tf_exl3_linear_6_2_4: 128 threads launched, the driver allows 128; 159 registers, 16 + 16384 shared bytes (recorded 159, 16) +PASS limits live exl3_linear tf_exl3_linear_8_2_4: 128 threads launched, the driver allows 128; 156 registers, 16 + 16384 shared bytes (recorded 156, 16) +PASS limits live exl3_linear tf_exl3_linear_10_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_12_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_14_2_4: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_2_4: 128 threads launched, the driver allows 128; 125 registers, 16 + 16384 shared bytes (recorded 125, 16) +PASS limits live exl3_linear tf_exl3_linear_3_2_4: 128 threads launched, the driver allows 128; 156 registers, 16 + 16384 shared bytes (recorded 156, 16) +PASS limits live exl3_linear tf_exl3_linear_5_2_4: 128 threads launched, the driver allows 128; 157 registers, 16 + 16384 shared bytes (recorded 157, 16) +PASS limits live exl3_linear tf_exl3_linear_7_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_9_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_11_2_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_13_2_4: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_2_1_4: 128 threads launched, the driver allows 128; 158 registers, 16 + 16384 shared bytes (recorded 158, 16) +PASS limits live exl3_linear tf_exl3_linear_4_1_4: 128 threads launched, the driver allows 128; 158 registers, 16 + 16384 shared bytes (recorded 158, 16) +PASS limits live exl3_linear tf_exl3_linear_6_1_4: 128 threads launched, the driver allows 128; 162 registers, 16 + 16384 shared bytes (recorded 162, 16) +PASS limits live exl3_linear tf_exl3_linear_8_1_4: 128 threads launched, the driver allows 128; 160 registers, 16 + 16384 shared bytes (recorded 160, 16) +PASS limits live exl3_linear tf_exl3_linear_10_1_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_12_1_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_14_1_4: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_1_4: 128 threads launched, the driver allows 128; 123 registers, 16 + 16384 shared bytes (recorded 123, 16) +PASS limits live exl3_linear tf_exl3_linear_2_0_4: 128 threads launched, the driver allows 128; 156 registers, 16 + 16384 shared bytes (recorded 156, 16) +PASS limits live exl3_linear tf_exl3_linear_4_0_4: 128 threads launched, the driver allows 128; 156 registers, 16 + 16384 shared bytes (recorded 156, 16) +PASS limits live exl3_linear tf_exl3_linear_6_0_4: 128 threads launched, the driver allows 128; 160 registers, 16 + 16384 shared bytes (recorded 160, 16) +PASS limits live exl3_linear tf_exl3_linear_8_0_4: 128 threads launched, the driver allows 128; 162 registers, 16 + 16384 shared bytes (recorded 162, 16) +PASS limits live exl3_linear tf_exl3_linear_10_0_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_12_0_4: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live exl3_linear tf_exl3_linear_14_0_4: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_0_4: 128 threads launched, the driver allows 128; 125 registers, 16 + 16384 shared bytes (recorded 125, 16) +PASS limits live exl3_linear tf_exl3_linear_2_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_4_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_6_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_8_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_10_2_8: 256 threads launched, the driver allows 256; 228 registers, 16 + 32768 shared bytes (recorded 228, 16) +PASS limits live exl3_linear tf_exl3_linear_12_2_8: 256 threads launched, the driver allows 256; 226 registers, 16 + 32768 shared bytes (recorded 226, 16) +PASS limits live exl3_linear tf_exl3_linear_14_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_3_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_5_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_7_2_8: 256 threads launched, the driver allows 256; 236 registers, 16 + 32768 shared bytes (recorded 236, 16) +PASS limits live exl3_linear tf_exl3_linear_9_2_8: 256 threads launched, the driver allows 256; 224 registers, 16 + 32768 shared bytes (recorded 224, 16) +PASS limits live exl3_linear tf_exl3_linear_11_2_8: 256 threads launched, the driver allows 256; 226 registers, 16 + 32768 shared bytes (recorded 226, 16) +PASS limits live exl3_linear tf_exl3_linear_13_2_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_2_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_4_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_6_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_8_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_10_1_8: 256 threads launched, the driver allows 256; 241 registers, 16 + 32768 shared bytes (recorded 241, 16) +PASS limits live exl3_linear tf_exl3_linear_12_1_8: 256 threads launched, the driver allows 256; 240 registers, 16 + 32768 shared bytes (recorded 240, 16) +PASS limits live exl3_linear tf_exl3_linear_14_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_1_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_2_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_4_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_6_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_8_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_10_0_8: 256 threads launched, the driver allows 256; 231 registers, 16 + 32768 shared bytes (recorded 231, 16) +PASS limits live exl3_linear tf_exl3_linear_12_0_8: 256 threads launched, the driver allows 256; 231 registers, 16 + 32768 shared bytes (recorded 231, 16) +PASS limits live exl3_linear tf_exl3_linear_14_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_linear tf_exl3_linear_16_0_8: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live exl3_experts tf_exl3_group: 1024 threads launched, the driver allows 1024; 42 registers, 128 + 49152 shared bytes (recorded 42, 128) +PASS limits live exl3_experts tf_exl3_experts_rot_in: 32 threads launched, the driver allows 1024; 19 registers, 0 + 0 shared bytes (recorded 19, 0) +PASS limits live exl3_experts tf_exl3_grouped_0: 128 threads launched, the driver allows 512; 127 registers, 32832 + 0 shared bytes (recorded 127, 32832) +PASS limits live exl3_experts tf_exl3_grouped_1: 128 threads launched, the driver allows 512; 128 registers, 32832 + 0 shared bytes (recorded 128, 32832) +PASS limits live exl3_experts tf_exl3_grouped_2: 128 threads launched, the driver allows 512; 128 registers, 32832 + 0 shared bytes (recorded 128, 32832) +PASS limits live exl3_experts tf_exl3_gateup_epilogue: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live exl3_experts tf_exl3_down_combine: 1024 threads launched, the driver allows 1024; 42 registers, 16384 + 0 shared bytes (recorded 42, 16384) +PASS limits live exl3_experts tf_exl3_plan_rotate: 32 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +FAIL limits live exl3_experts tf_exl3_plan_finish: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 42, 0) +PASS limits live exl3_experts tf_exl3_prompt_rotate: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +FAIL limits live exl3_experts tf_exl3_prompt_gu: 256 threads launched, the driver allows 256; 128 registers, 43264 + 0 shared bytes (recorded 127, 43264) +PASS limits live exl3_experts tf_exl3_prompt_down: 256 threads launched, the driver allows 256; 127 registers, 43264 + 0 shared bytes (recorded 127, 43264) +PASS limits live exl3_candidate tf_exl3_tc_0: 256 threads launched, the driver allows 256; 128 registers, 8196 + 0 shared bytes (recorded 128, 8196) +PASS limits live exl3_candidate tf_exl3_tc_1: 256 threads launched, the driver allows 256; 128 registers, 8196 + 0 shared bytes (recorded 128, 8196) +PASS limits live exl3_candidate tf_exl3_tc_2: 256 threads launched, the driver allows 256; 128 registers, 8196 + 0 shared bytes (recorded 128, 8196) +PASS limits live exl3_candidate tf_exl3_prompt_0: 256 threads launched, the driver allows 256; 75 registers, 32768 + 0 shared bytes (recorded 75, 32768) +PASS limits live exl3_candidate tf_exl3_prompt_1: 256 threads launched, the driver allows 256; 77 registers, 32768 + 0 shared bytes (recorded 77, 32768) +FAIL limits live exl3_candidate tf_exl3_prompt_2: 256 threads launched, the driver allows 256; 79 registers, 32768 + 0 shared bytes (recorded 80, 32768) +PASS limits live exl3_candidate tf_exl3_rotate_f: 32 threads launched, the driver allows 32; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live exl3_candidate tf_exl3_rotate_h: 32 threads launched, the driver allows 32; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live exl3_candidate tf_exl3_codebook: 256 threads launched, the driver allows 1024; 8 registers, 0 + 0 shared bytes (recorded 8, 0) +PASS limits live exl3_candidate tf_exl3_unpack: 32 threads launched, the driver allows 32; 21 registers, 0 + 0 shared bytes (recorded 21, 0) +PASS limits live experts3 decode pair: 128 threads launched, the driver allows 128; 255 registers, 0 + 0 shared bytes (recorded 255, 0) +PASS limits live experts3 decode down: 128 threads launched, the driver allows 128; 194 registers, 0 + 0 shared bytes (recorded 194, 0) +PASS limits live experts3_prefill pair: 256 threads launched, the driver allows 256; 150 registers, 0 + 46080 shared bytes (recorded 150, 0) +PASS limits live experts3_prefill down: 256 threads launched, the driver allows 256; 98 registers, 0 + 47104 shared bytes (recorded 98, 0) +PASS limits live experts3 tf_experts3_widen: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live experts3 tf_experts3_planes: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live h3 h3_norm: 256 threads launched, the driver allows 256; 32 registers, 128 + 0 shared bytes (recorded 32, 128) +PASS limits live h3 h3_gate_add: 256 threads launched, the driver allows 256; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live h3 h3_gate_mix: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live h3 h3_swiglu: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live h3 h3_heads: 256 threads launched, the driver allows 256; 255 registers, 0 + 0 shared bytes (recorded 255, 0) +PASS limits live h3 h3_pool: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live h3 h3_tile_scores: 256 threads launched, the driver allows 256; 21 registers, 0 + 0 shared bytes (recorded 21, 0) +PASS limits live h3 h3_coarse: 32 threads launched, the driver allows 1024; 39 registers, 0 + 0 shared bytes (recorded 39, 0) +PASS limits live h3 h3_attention: 128 threads launched, the driver allows 128; 208 registers, 0 + 49152 shared bytes (recorded 208, 0) +PASS limits live h3 h3_topk: 32 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live drama3 d3_gemv: 256 threads launched, the driver allows 256; 64 registers, 0 + 12288 shared bytes (recorded 64, 0) +PASS limits live drama3 d3_rmsnorm: 256 threads launched, the driver allows 1024; 18 registers, 144 + 0 shared bytes (recorded 18, 144) +PASS limits live drama3 d3_add_rmsnorm: 256 threads launched, the driver allows 1024; 24 registers, 144 + 0 shared bytes (recorded 24, 144) +PASS limits live drama3 d3_embed_backbone: 256 threads launched, the driver allows 1024; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live drama3 d3_conv_silu: 128 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live drama3 d3_gdn: 32 threads launched, the driver allows 32; 255 registers, 1024 + 0 shared bytes (recorded 255, 1024) +PASS limits live drama3 d3_gated_norm: 256 threads launched, the driver allows 1024; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live drama3 d3_qk_rope: 256 threads launched, the driver allows 1024; 26 registers, 1168 + 0 shared bytes (recorded 26, 1168) +PASS limits live drama3 d3_attn_decode: 256 threads launched, the driver allows 256; 40 registers, 8256 + 0 shared bytes (recorded 40, 8256) +PASS limits live drama3 d3_attn_partial: 256 threads launched, the driver allows 256; 40 registers, 8256 + 0 shared bytes (recorded 40, 8256) +PASS limits live drama3 d3_attn_merge: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live drama3 d3_q_group: 256 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live drama3 d3_causal_softmax: 256 threads launched, the driver allows 1024; 20 registers, 272 + 0 shared bytes (recorded 20, 272) +PASS limits live drama3 d3_attn_gate: 256 threads launched, the driver allows 1024; 19 registers, 0 + 0 shared bytes (recorded 19, 0) +PASS limits live drama3 d3_bf16_f32: 256 threads launched, the driver allows 1024; 16 registers, 0 + 0 shared bytes (recorded 16, 0) +PASS limits live drama3 d3_router: 256 threads launched, the driver allows 1024; 26 registers, 592 + 0 shared bytes (recorded 26, 592) +PASS limits live drama3 d3_swiglu: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live drama3 d3_moe_combine: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +FAIL limits live drama3 d3_act_quant: 128 threads launched, the driver allows 1024; 22 registers, 144 + 0 shared bytes (recorded 21, 144) +PASS limits live drama3 d3_tower_mix: 256 threads launched, the driver allows 1024; 36 registers, 8336 + 0 shared bytes (recorded 36, 8336) +PASS limits live drama3 d3_sample_main: 1024 threads launched, the driver allows 1024; 47 registers, 400 + 65536 shared bytes (recorded 47, 400) +PASS limits live drama3 d3_sample_main_fast: 1024 threads launched, the driver allows 1024; 37 registers, 3616 + 0 shared bytes (recorded 37, 3616) +PASS limits live drama3 d3_sample_code: 1024 threads launched, the driver allows 1024; 39 registers, 400 + 32768 shared bytes (recorded 39, 400) +PASS limits live drama3 d3_sample_code_fast: 1024 threads launched, the driver allows 1024; 25 registers, 3616 + 0 shared bytes (recorded 25, 3616) +PASS limits live drama3 d3_argmax_main: 1024 threads launched, the driver allows 1024; 13 registers, 256 + 0 shared bytes (recorded 13, 256) +PASS limits live drama3 d3_finish_frame: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live drama3 d3_next_input: 32 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live drama3 d3_copy_i32: 64 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live drama3_codec c_vq_gather: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live drama3_codec c_vq_encode: 256 threads launched, the driver allows 1024; 39 registers, 424 + 0 shared bytes (recorded 39, 424) +PASS limits live drama3_codec c_rmsnorm: 256 threads launched, the driver allows 1024; 18 registers, 132 + 0 shared bytes (recorded 18, 132) +FAIL limits live drama3_codec c_layernorm: 256 threads launched, the driver allows 1024; 25 registers, 132 + 0 shared bytes (recorded 24, 132) +PASS limits live drama3_codec c_rope_kv: 512 threads launched, the driver allows 1024; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live drama3_codec c_attn: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live drama3_codec c_add_gamma: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live drama3_codec c_silu_mul: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live drama3_codec c_gelu: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live drama3_codec c_bias: 256 threads launched, the driver allows 1024; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live drama3_codec c_snake: 256 threads launched, the driver allows 1024; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live drama3_codec c_dwconv7: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live drama3_codec c_copy: 256 threads launched, the driver allows 1024; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live drama3_fp8 decode gate-up: 128 threads launched, the driver allows 128; 163 registers, 0 + 0 shared bytes (recorded 163, 0) +PASS limits live drama3_fp8 decode down: 128 threads launched, the driver allows 128; 118 registers, 0 + 0 shared bytes (recorded 118, 0) +PASS limits live drama3_fp8 prompt gate-up: 128 threads launched, the driver allows 128; 212 registers, 32768 + 0 shared bytes (recorded 212, 32768) +PASS limits live drama3_fp8 prompt down: 128 threads launched, the driver allows 128; 213 registers, 32768 + 0 shared bytes (recorded 213, 32768) +FAIL limits live router tf_nemo_router_tiles: 256 threads launched, the driver allows 256; 183 registers, 0 + 40960 shared bytes (recorded 177, 0) +PASS limits live latent tf_super_combine_sums: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live latent tf_super_moe_out_sums: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +FAIL limits live ck tf_ck4_up_nvfp4_prompt: 256 threads launched, the driver allows 256; 115 registers, 46080 + 0 shared bytes (recorded 114, 46080) +PASS limits live ck tf_ck4_down_f32_prompt: 256 threads launched, the driver allows 256; 126 registers, 36864 + 0 shared bytes (recorded 126, 36864) +PASS limits live ck tf_ck4_down_sum_prompt: 256 threads launched, the driver allows 256; 224 registers, 36864 + 0 shared bytes (recorded 224, 36864) +PASS limits live ck tf_ck8_128: 256 threads launched, the driver allows 256; 76 registers, 0 + 49152 shared bytes (recorded 76, 0) +PASS limits live ck tf_ck8_128c cluster of 8: 256 threads launched, the driver allows 256; 104 registers, 0 + 65536 shared bytes (recorded 104, 0) +PASS limits live glm_mm16_16: 128 threads launched, the driver allows 128; 56 registers, 0 + 31104 shared bytes (recorded 56, 0) +PASS limits live glm_mm16_32: 128 threads launched, the driver allows 128; 64 registers, 0 + 37248 shared bytes (recorded 64, 0) +PASS limits live glm_mm16_64: 256 threads launched, the driver allows 256; 64 registers, 0 + 49536 shared bytes (recorded 64, 0) +PASS limits live glm_mm16_128: 256 threads launched, the driver allows 256; 100 registers, 0 + 74112 shared bytes (recorded 100, 0) +PASS limits live glm_mm_tile16: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live glm_mm4_16: 128 threads launched, the driver allows 128; 50 registers, 0 + 12672 shared bytes (recorded 50, 0) +PASS limits live glm_mm4_32: 128 threads launched, the driver allows 128; 65 registers, 0 + 18816 shared bytes (recorded 65, 0) +PASS limits live glm_mm4_64: 256 threads launched, the driver allows 256; 61 registers, 0 + 31104 shared bytes (recorded 61, 0) +PASS limits live glm_mm4_128: 256 threads launched, the driver allows 256; 106 registers, 0 + 55680 shared bytes (recorded 106, 0) +PASS limits live glm_mm_tile4: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live glm_mm6_16: 128 threads launched, the driver allows 128; 54 registers, 0 + 15744 shared bytes (recorded 54, 0) +PASS limits live glm_mm6_32: 128 threads launched, the driver allows 128; 70 registers, 0 + 21888 shared bytes (recorded 70, 0) +PASS limits live glm_mm6_64: 256 threads launched, the driver allows 256; 64 registers, 0 + 34176 shared bytes (recorded 64, 0) +PASS limits live glm_mm6_128: 256 threads launched, the driver allows 256; 106 registers, 0 + 58752 shared bytes (recorded 106, 0) +PASS limits live glm_mm_tile6: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live glm_mm8_16: 128 threads launched, the driver allows 128; 58 registers, 0 + 18816 shared bytes (recorded 58, 0) +PASS limits live glm_mm8_32: 128 threads launched, the driver allows 128; 74 registers, 0 + 24960 shared bytes (recorded 74, 0) +PASS limits live glm_mm8_64: 256 threads launched, the driver allows 256; 63 registers, 0 + 37248 shared bytes (recorded 63, 0) +PASS limits live glm_mm8_128: 256 threads launched, the driver allows 256; 106 registers, 0 + 61824 shared bytes (recorded 106, 0) +PASS limits live glm_mm_tile8: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live glm_mm16_rows: 256 threads launched, the driver allows 256; 99 registers, 0 + 74112 shared bytes (recorded 99, 0) +PASS limits live glm_mm_reduce: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live df_tap: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live df_block_qkv: 64 threads launched, the driver allows 1024; 32 registers, 8 + 0 shared bytes (recorded 32, 8) +PASS limits live df_ctx_kv: 64 threads launched, the driver allows 1024; 28 registers, 8 + 0 shared bytes (recorded 28, 8) +PASS limits live df_conv: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live df_swiglu: 256 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live df_attn_part: 256 threads launched, the driver allows 256; 75 registers, 0 + 0 shared bytes (recorded 75, 0) +PASS limits live df_attn_merge: 128 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live df_sum2: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live df_top16_part: 256 threads launched, the driver allows 256; 30 registers, 32864 + 0 shared bytes (recorded 30, 32864) +PASS limits live df_top16_merge: 256 threads launched, the driver allows 256; 28 registers, 8416 + 0 shared bytes (recorded 28, 8416) +PASS limits live df_gsum: 128 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live glm_glue tf_glm_lat_write: 128 threads launched, the driver allows 1024; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live glm_glue tf_glm_pool_keys: 32 threads launched, the driver allows 32; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live glm_glue tf_glm_scores: 128 threads launched, the driver allows 128; 157 registers, 32768 + 0 shared bytes (recorded 157, 32768) +PASS limits live glm_glue tf_glm_chunks_16: 256 threads launched, the driver allows 256; 123 registers, 0 + 83328 shared bytes (recorded 123, 0) +PASS limits live glm_glue tf_glm_chunks_32: 256 threads launched, the driver allows 256; 128 registers, 0 + 101120 shared bytes (recorded 128, 0) +PASS limits live glm_ops tf_glm_dsa_tokens: 256 threads launched, the driver allows 1024; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +FAIL limits live tree attention tf_tree_paths: 128 threads launched, the driver allows 1024; 22 registers, 0 + 0 shared bytes (recorded 30, 0) +FAIL limits live tree attention tf_tree_shared: 128 threads launched, the driver allows 128; 213 registers, 0 + 65536 shared bytes (recorded 212, 0) +FAIL limits live tree attention tf_tree_shared_w1: 32 threads launched, the driver allows 32; 168 registers, 0 + 32768 shared bytes (recorded 195, 0) +FAIL limits live tree attention tf_tree_shared_w1s3: 32 threads launched, the driver allows 32; 168 registers, 0 + 49152 shared bytes (recorded 198, 0) +PASS limits live tree attention tf_tree_part: 128 threads launched, the driver allows 128; 228 registers, 0 + 65536 shared bytes (recorded 228, 0) +PASS limits live tree attention tf_tree_part_w1: 32 threads launched, the driver allows 32; 168 registers, 0 + 32768 shared bytes (recorded 168, 0) +FAIL limits live tree attention tf_tree_part_w1s3: 32 threads launched, the driver allows 32; 168 registers, 0 + 49152 shared bytes (recorded 202, 0) +PASS limits live tree attention tf_tree_shared_256: 128 threads launched, the driver allows 128; 255 registers, 0 + 73728 shared bytes (recorded 255, 0) +FAIL limits live tree attention tf_tree_shared_256_w1: 32 threads launched, the driver allows 32; 253 registers, 0 + 40960 shared bytes (recorded 249, 0) +FAIL limits live tree attention tf_tree_shared_256_w1s3: 32 threads launched, the driver allows 32; 252 registers, 0 + 57344 shared bytes (recorded 250, 0) +PASS limits live tree attention tf_tree_part_256: 128 threads launched, the driver allows 128; 255 registers, 0 + 73728 shared bytes (recorded 255, 0) +FAIL limits live tree attention tf_tree_part_256_w1: 32 threads launched, the driver allows 32; 255 registers, 0 + 40960 shared bytes (recorded 253, 0) +PASS limits live tree attention tf_tree_part_256_w1s3: 32 threads launched, the driver allows 32; 251 registers, 0 + 57344 shared bytes (recorded 251, 0) +FAIL limits live tree attention tf_tree_merge: 256 threads launched, the driver allows 256; 110 registers, 0 + 0 shared bytes (recorded 128, 0) +FAIL limits live block attention tf_block_attention (up to 8 query heads a key head): 256 threads launched, the driver allows 256; 236 registers, 0 + 16384 shared bytes (recorded 228, 0) +FAIL limits live window attention tf_window_part: 256 threads launched, the driver allows 256; 54 registers, 45824 + 0 shared bytes (recorded 56, 45824) +FAIL limits live window attention tf_window_merge: 256 threads launched, the driver allows 256; 38 registers, 0 + 0 shared bytes (recorded 48, 0) +FAIL limits live window attention tf_window_decode_tc: 128 threads launched, the driver allows 128; 182 registers, 0 + 65536 shared bytes (recorded 178, 0) +FAIL limits live window attention wattn_kernel<128, 8, 4, 8>: 256 threads launched, the driver allows 256; 218 registers, 0 + 65536 shared bytes (recorded 222, 0) +FAIL limits live window attention wattn_kernel<128, 8, 8, 8>: 256 threads launched, the driver allows 256; 218 registers, 0 + 65536 shared bytes (recorded 222, 0) +PASS limits live moe tf_moe_topk_sigmoid: 256 threads launched, the driver allows 256; 48 registers, 16704 + 0 shared bytes (recorded 48, 16704) +FAIL limits live moe tf_moe_topk_sigmoid_warp: 256 threads launched, the driver allows 256; 63 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live moe tf_moe_combine: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dense rows tf_dense_rows_1: 256 threads launched, the driver allows 256; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +FAIL limits live dense rows tf_dense_rows_4: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 45, 0) +FAIL limits live dense rows tf_dense_rows_16: 256 threads launched, the driver allows 256; 96 registers, 0 + 0 shared bytes (recorded 98, 0) +PASS limits live dense rows tf_dense_rows_32: 256 threads launched, the driver allows 256; 180 registers, 0 + 0 shared bytes (recorded 180, 0) +PASS limits live act_rows tf_geglu_rows 256 threads: 256 threads launched, the driver allows 256; 33 registers, 0 + 0 shared bytes (recorded 33, 0) +PASS limits live act_rows tf_geglu_rows 128 threads: 128 threads launched, the driver allows 256; 33 registers, 0 + 0 shared bytes (recorded 33, 0) +PASS limits live act_rows tf_swiglu_rows_round 256 threads: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live act_rows tf_swiglu_rows_round 128 threads: 128 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live embed_rows tf_embed_rows_affine: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qk_rows tf_qk_rows_once 512 threads: 512 threads launched, the driver allows 512; 34 registers, 64 + 0 shared bytes (recorded 34, 64) +PASS limits live qk_rows tf_qk_rows_round 512 threads: 512 threads launched, the driver allows 512; 34 registers, 64 + 0 shared bytes (recorded 34, 64) +PASS limits live norms tf_add_rms_rows_bf16h_tail: 256 threads launched, the driver allows 256; 147 registers, 64 + 0 shared bytes (recorded 147, 64) +PASS limits live norms tf_add_rms_rows_bf16h_tail_round: 256 threads launched, the driver allows 256; 147 registers, 64 + 0 shared bytes (recorded 147, 64) +PASS limits live flashnext_fact factored prompt matmul 64x128: 256 threads launched, the driver allows 256; 128 registers, 0 + 41472 shared bytes (recorded 128, 0) +PASS limits live flashnext_fact factored prompt matmul 64x128 folded: 256 threads launched, the driver allows 256; 128 registers, 0 + 41472 shared bytes (recorded 128, 0) +PASS limits live flashnext_hcf hyper-connection down 32x192 folded: 256 threads launched, the driver allows 256; 126 registers, 0 + 49664 shared bytes (recorded 126, 0) +PASS limits live flashnext_hcf hyper-connection up 64x128: 256 threads launched, the driver allows 256; 128 registers, 0 + 43264 shared bytes (recorded 128, 0) +PASS limits live flashnext_fexp factored experts gate and up: 256 threads launched, the driver allows 256; 116 registers, 0 + 41472 shared bytes (recorded 116, 0) +PASS limits live flashnext_fexp factored experts down: 256 threads launched, the driver allows 256; 120 registers, 0 + 41472 shared bytes (recorded 120, 0) +PASS limits live flashnext_batch fn_gdn_chain_tpr: 256 threads launched, the driver allows 256; 152 registers, 0 + 49664 shared bytes (recorded 152, 0) +PASS limits live flashnext_attn qsa_tile scores: 256 threads launched, the driver allows 256; 224 registers, 16384 + 0 shared bytes (recorded 224, 16384) +PASS limits live flashnext_attn fn_attn_fused: 128 threads launched, the driver allows 128; 122 registers, 640 + 26624 shared bytes (recorded 122, 640) +PASS limits live flashnext_attn fn_attn_chunks_one: 128 threads launched, the driver allows 128; 96 registers, 640 + 26624 shared bytes (recorded 96, 640) +PASS limits live flashnext_hc fn_hc_rinv: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live flashnext_hc fn_hc_mix_h: 256 threads launched, the driver allows 256; 38 registers, 1024 + 0 shared bytes (recorded 38, 1024) +PASS limits live route_rows tf_route_logits_wide: 256 threads launched, the driver allows 256; 58 registers, 0 + 55296 shared bytes (recorded 58, 0) +PASS limits live gdn_train tf_gdn_front threads 128: 128 threads launched, the driver allows 128; 40 registers, 1024 + 0 shared bytes (recorded 40, 1024) +PASS limits live gdn_train tf_gdn_scan_fwd threads 256: 256 threads launched, the driver allows 256; 60 registers, 1152 + 0 shared bytes (recorded 60, 1152) +PASS limits live gdn_train tf_gdn_norm_back threads 128: 128 threads launched, the driver allows 128; 24 registers, 644 + 0 shared bytes (recorded 24, 644) +FAIL limits live gdn_train tf_gdn_scan_back threads 256: 256 threads launched, the driver allows 256; 102 registers, 9536 + 0 shared bytes (recorded 100, 9536) +PASS limits live gdn_train tf_gdn_qk_back threads 128: 128 threads launched, the driver allows 128; 40 registers, 128 + 0 shared bytes (recorded 40, 128) +PASS limits live gdn_train tf_gdn_conv_back threads 256: 256 threads launched, the driver allows 1024; 35 registers, 0 + 0 shared bytes (recorded 35, 0) +PASS limits live gdn_train tf_gdn_dequant threads 256: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 32: 32 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 64: 64 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 96: 96 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 128: 128 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 160: 160 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 192: 192 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 224: 224 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 256: 256 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 288: 288 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 320: 320 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 352: 352 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 384: 384 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 416: 416 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 448: 448 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 480: 480 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 512: 512 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 544: 544 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 576: 576 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 608: 608 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 640: 640 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 672: 672 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 704: 704 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 736: 736 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 768: 768 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 800: 800 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 832: 832 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 864: 864 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 896: 896 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 928: 928 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 960: 960 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 992: 992 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live gdn_train tf_gdn_gates_back threads 1024: 1024 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live train_q4 tf_q4_back threads 128: 128 threads launched, the driver allows 128; 60 registers, 10240 + 0 shared bytes (recorded 60, 10240) +PASS limits live train_q4 tf_q4_back_bf threads 128: 128 threads launched, the driver allows 128; 56 registers, 10240 + 0 shared bytes (recorded 56, 10240) +PASS limits live train_q4 tf_q4_sum_parts threads 256: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_q4 tf_q4_experts_mm threads 64: 64 threads launched, the driver allows 64; 79 registers, 10304 + 0 shared bytes (recorded 79, 10304) +PASS limits live train_q4 tf_q4_experts_nt threads 64: 64 threads launched, the driver allows 64; 78 registers, 10816 + 0 shared bytes (recorded 78, 10816) +PASS limits live train_q4 tf_q4_swiglu_back threads 128: 128 threads launched, the driver allows 128; 46 registers, 128 + 0 shared bytes (recorded 46, 128) +PASS limits live train_q4 tf_q4_experts_nt_packed threads 64: 64 threads launched, the driver allows 64; 74 registers, 10816 + 0 shared bytes (recorded 74, 10816) +PASS limits live train_q4 tf_q4_experts_mm_packed threads 64: 64 threads launched, the driver allows 64; 80 registers, 10304 + 0 shared bytes (recorded 80, 10304) +PASS limits live train_q4 tf_q4_route_back threads 256: 256 threads launched, the driver allows 256; 40 registers, 136 + 0 shared bytes (recorded 40, 136) +PASS limits live train_q4 tf_q4_axpy threads 256: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live train_attn tf_gattn_q threads 256: 256 threads launched, the driver allows 256; 48 registers, 18560 + 0 shared bytes (recorded 48, 18560) +PASS limits live train_attn tf_gattn_gate_back threads 256: 256 threads launched, the driver allows 1024; 17 registers, 0 + 0 shared bytes (recorded 17, 0) +PASS limits live train_attn tf_gattn_prep threads 32: 32 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 32: 32 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 32: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 64: 64 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 64: 64 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 64: 64 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 96: 96 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 96: 96 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 96: 96 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 128: 128 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 128: 128 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 128: 128 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 160: 160 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 160: 160 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 160: 160 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 192: 192 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 192: 192 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 192: 192 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 224: 224 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 224: 224 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 224: 224 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train_attn tf_gattn_prep threads 256: 256 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train_attn tf_gattn_prep_back threads 256: 256 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live train tf_train_attn_kv threads 256: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live train tf_train_softmax threads 1024: 1024 threads launched, the driver allows 1024; 41 registers, 128 + 0 shared bytes (recorded 41, 128) +PASS limits live train tf_train_logits threads 256: 256 threads launched, the driver allows 256; 33 registers, 0 + 0 shared bytes (recorded 33, 0) +PASS limits live train tf_train_sort_pairs threads 256: 256 threads launched, the driver allows 256; 40 registers, 4096 + 0 shared bytes (recorded 40, 4096) +PASS limits live train tf_train_experts_tiles threads 1: 1 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live train tf_train_pairs_out threads 256: 256 threads launched, the driver allows 1024; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live train tf_train_zero threads 256: 256 threads launched, the driver allows 1024; 8 registers, 0 + 0 shared bytes (recorded 8, 0) +PASS limits live train tf_train_narrow threads 256: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live train tf_train_gemm threads 128: 128 threads launched, the driver allows 128; 64 registers, 10240 + 0 shared bytes (recorded 64, 10240) +PASS limits live flashnext_train tf_fn_hc_branch_back threads 256: 256 threads launched, the driver allows 256; 48 registers, 128 + 0 shared bytes (recorded 48, 128) +PASS limits live flashnext_train tf_fn_hc_mix_back threads 256: 256 threads launched, the driver allows 1024; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live flashnext_train tf_fn_hc_act_back threads 256: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live flashnext_train tf_fn_hc_norm_back threads 256: 256 threads launched, the driver allows 256; 38 registers, 128 + 0 shared bytes (recorded 38, 128) +PASS limits live flashnext_train tf_fn_pocket_in threads 128: 128 threads launched, the driver allows 128; 31 registers, 0 + 0 shared bytes (recorded 31, 0) +PASS limits live flashnext_train tf_fn_pocket_in_rows threads 256: 256 threads launched, the driver allows 256; 64 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live flashnext_train tf_fn_pocket_out threads 128: 128 threads launched, the driver allows 128; 64 registers, 512 + 0 shared bytes (recorded 64, 512) +PASS limits live flashnext_train tf_fn_pocket_out_rows threads 128: 128 threads launched, the driver allows 128; 72 registers, 2048 + 0 shared bytes (recorded 72, 2048) +PASS limits live flashnext_train tf_fn_pocket_back threads 256: 256 threads launched, the driver allows 256; 40 registers, 1536 + 0 shared bytes (recorded 40, 1536) +PASS limits live flashnext_train tf_fn_pocket_grad threads 256: 256 threads launched, the driver allows 256; 37 registers, 128 + 0 shared bytes (recorded 37, 128) +PASS limits live flashnext_batch fn_moe_sum threads 256: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live flashnext_hc fn_hc_upmix_up threads 256: 256 threads launched, the driver allows 256; 62 registers, 2048 + 0 shared bytes (recorded 62, 2048) +PASS limits live 27B tf_train_zero: 256 threads launched, the driver allows 1024; 8 registers, 0 + 0 shared bytes (recorded 8, 0) +PASS limits live 27B tf_q27_gemm_q4w_bf: 256 threads launched, the driver allows 256; 128 registers, 37888 + 0 shared bytes (recorded 128, 37888) +PASS limits live 27B tf_q27_gemm_q4w: 256 threads launched, the driver allows 256; 126 registers, 37888 + 0 shared bytes (recorded 126, 37888) +PASS limits live 27B tf_q27_gemm_q4_bf: 128 threads launched, the driver allows 128; 56 registers, 10240 + 0 shared bytes (recorded 56, 10240) +PASS limits live 27B tf_q27_gemm_q4: 128 threads launched, the driver allows 128; 56 registers, 10240 + 0 shared bytes (recorded 56, 10240) +PASS limits live 27B tf_sum_parts: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live 27B tf_q27_fit_step: 256 threads launched, the driver allows 256; 39 registers, 5120 + 0 shared bytes (recorded 39, 5120) +PASS limits live 27B tf_q27_zero_rows: 256 threads launched, the driver allows 1024; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live 27B tf_train_softmax: 1024 threads launched, the driver allows 1024; 41 registers, 128 + 0 shared bytes (recorded 41, 128) +PASS limits live 27B tf_train_rms_back: 256 threads launched, the driver allows 256; 37 registers, 128 + 0 shared bytes (recorded 37, 128) +PASS limits live 27B tf_axpy: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live 27B tf_q27_add_bf16: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live 27B tf_q27_cols: 256 threads launched, the driver allows 1024; 16 registers, 0 + 0 shared bytes (recorded 16, 0) +PASS limits live 27B tf_q27_pocket_rows: 256 threads launched, the driver allows 256; 40 registers, 512 + 0 shared bytes (recorded 40, 512) +PASS limits live 27B tf_q27_pocket_back: 256 threads launched, the driver allows 256; 40 registers, 1024 + 0 shared bytes (recorded 40, 1024) +PASS limits live 27B tf_q27_pocket_grad: 256 threads launched, the driver allows 256; 37 registers, 128 + 0 shared bytes (recorded 37, 128) +PASS limits live 27B tf_q27_swiglu_back: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live 27B tf_attn_prep: 256 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live 27B tf_attn_gate_back: 256 threads launched, the driver allows 1024; 17 registers, 0 + 0 shared bytes (recorded 17, 0) +PASS limits live 27B tf_attn_q: 256 threads launched, the driver allows 256; 48 registers, 18560 + 0 shared bytes (recorded 48, 18560) +PASS limits live 27B tf_train_attn_kv: 256 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live 27B tf_attn_prep_back: 256 threads launched, the driver allows 256; 32 registers, 1152 + 0 shared bytes (recorded 32, 1152) +PASS limits live 27B tf_q27_fit_dots: 256 threads launched, the driver allows 256; 40 registers, 0 + 20480 shared bytes (recorded 40, 0) +PASS limits live 27B tf_q27_fit_dots_tree: 256 threads launched, the driver allows 256; 40 registers, 0 + 20480 shared bytes (recorded 40, 0) +PASS limits live 27B tf_gdn_front: 128 threads launched, the driver allows 128; 40 registers, 1024 + 0 shared bytes (recorded 40, 1024) +PASS limits live 27B tf_gdn_scan_fwd: 256 threads launched, the driver allows 256; 60 registers, 1152 + 0 shared bytes (recorded 60, 1152) +PASS limits live 27B tf_gdn_norm_back: 128 threads launched, the driver allows 128; 24 registers, 644 + 0 shared bytes (recorded 24, 644) +FAIL limits live 27B tf_gdn_scan_back: 256 threads launched, the driver allows 256; 102 registers, 9536 + 0 shared bytes (recorded 100, 9536) +PASS limits live 27B tf_gdn_qk_back: 128 threads launched, the driver allows 128; 40 registers, 128 + 0 shared bytes (recorded 40, 128) +PASS limits live 27B tf_gdn_gates_back: 64 threads launched, the driver allows 1024; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live 27B tf_gdn_conv_back: 256 threads launched, the driver allows 1024; 35 registers, 0 + 0 shared bytes (recorded 35, 0) +PASS limits live 27B tf_gdn_dequant: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live 27B tf_train_narrow: 256 threads launched, the driver allows 1024; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +PASS limits live 27B tf_train_gemm: 128 threads launched, the driver allows 128; 64 registers, 10240 + 0 shared bytes (recorded 64, 10240) +PASS limits live paging tf_nemo_kv_write: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live paging tf_nemo_kv_write_segs: 256 threads launched, the driver allows 1024; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live paging tf_nemo_attn_chunk: 256 threads launched, the driver allows 256; 118 registers, 45568 + 0 shared bytes (recorded 118, 45568) +PASS limits live paging tf_nemo_attn_chunk_pf: 256 threads launched, the driver allows 256; 128 registers, 45568 + 0 shared bytes (recorded 128, 45568) +PASS limits live paging tf_nemo_attn_chunk_segs: 256 threads launched, the driver allows 256; 118 registers, 45568 + 0 shared bytes (recorded 118, 45568) +PASS limits live paging pattn128: 256 threads launched, the driver allows 256; 231 registers, 0 + 65536 shared bytes (recorded 231, 0) +PASS limits live tf_moe_router: 256 threads launched, the driver allows 256; 48 registers, 16512 + 0 shared bytes (recorded 48, 16512) +PASS limits live tf_moe_topk: 32 threads launched, the driver allows 32; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live tf_moe_combine: 256 threads launched, the driver allows 256; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +FAIL limits live experts plan_rows: 1024 threads launched, the driver allows 1024; 34 registers, 12672 + 0 shared bytes (recorded 32, 12672) +PASS limits live experts plan_rank_rows: 1024 threads launched, the driver allows 1024; 42 registers, 4096 + 0 shared bytes (recorded 42, 4096) +PASS limits live experts plan_scatter_rows: 256 threads launched, the driver allows 1024; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live tf_q36m_norm_rope: 32 threads launched, the driver allows 32; 42 registers, 0 + 0 shared bytes (recorded 42, 0) +PASS limits live tf_q36m_attn: 32 threads launched, the driver allows 32; 52 registers, 0 + 0 shared bytes (recorded 52, 0) +PASS limits live tf_q36m_attn_tree: 32 threads launched, the driver allows 32; 57 registers, 0 + 0 shared bytes (recorded 57, 0) +PASS limits live qwen35 tf_q35_gdn_slots_1: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen35 tf_q35_gdn_slots_2: 128 threads launched, the driver allows 128; 48 registers, 0 + 0 shared bytes (recorded 48, 0) +PASS limits live qwen35 tf_q35_gdn_slots_4: 128 threads launched, the driver allows 128; 56 registers, 0 + 0 shared bytes (recorded 56, 0) +PASS limits live qwen35 tf_q35_gdn_slots_8: 128 threads launched, the driver allows 128; 72 registers, 0 + 0 shared bytes (recorded 72, 0) +PASS limits live qwen35 tf_q35_gdn_slots_16: 128 threads launched, the driver allows 128; 112 registers, 0 + 0 shared bytes (recorded 112, 0) +PASS limits live qwen35 tf_q35_gdn_chain: 128 threads launched, the driver allows 128; 56 registers, 0 + 0 shared bytes (recorded 56, 0) +PASS limits live qwen35 tf_q35_rope: 256 threads launched, the driver allows 256; 34 registers, 16384 + 0 shared bytes (recorded 34, 16384) +PASS limits live qwen35 tf_q35_attn_partial: 1024 threads launched, the driver allows 1024; 39 registers, 0 + 0 shared bytes (recorded 39, 0) +PASS limits live qwen35 tf_q35_attn_merge: 32 threads launched, the driver allows 32; 42 registers, 0 + 0 shared bytes (recorded 42, 0) +PASS limits live qwen35_kv8 tf_q35_rope8: 256 threads launched, the driver allows 256; 41 registers, 16416 + 0 shared bytes (recorded 41, 16416) +PASS limits live qwen35_kv8 tf_q35_attn_partial8: 1024 threads launched, the driver allows 1024; 37 registers, 0 + 0 shared bytes (recorded 37, 0) +PASS limits live qwen35_keep tf_q35k_apply: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen35_keep tf_q35k_compact: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live qwen35_kv8 tf_q35k_compact8: 256 threads launched, the driver allows 256; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live qwen35_draft tf_q35d_gather_rows: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live qwen35 tf_q35_gdn_pre: 32 threads launched, the driver allows 32; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live qwen35 tf_q35_conv_commit: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live qwen35 tf_q35_tap: 256 threads launched, the driver allows 256; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live qwen35_keep tf_q35k_conv: 256 threads launched, the driver allows 1024; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live qwen35_draft tf_q35d_dyn_conv_batch: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live qwen35_draft tf_q35d_norm_rope_batch: 32 threads launched, the driver allows 32; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live qwen35_draft tf_q35d_mask_rows: 256 threads launched, the driver allows 1024; 16 registers, 0 + 0 shared bytes (recorded 16, 0) +PASS limits live qwen35_draft tf_q35d_attn_partial: 32 threads launched, the driver allows 32; 50 registers, 0 + 0 shared bytes (recorded 50, 0) +PASS limits live qwen35_draft tf_q35d_attn_merge: 32 threads launched, the driver allows 32; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live tf_topk_slices: 256 threads launched, the driver allows 256; 28 registers, 16448 + 0 shared bytes (recorded 28, 16448) +FAIL limits live tf_topk_merge: 256 threads launched, the driver allows 256; 38 registers, 64 + 0 shared bytes (recorded 36, 64) +PASS limits live tf_q35_norm: 256 threads launched, the driver allows 256; 48 registers, 16416 + 0 shared bytes (recorded 48, 16416) +PASS limits live tf_q35_sums: 64 threads launched, the driver allows 64; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live tf_q35_argmax_slices: 256 threads launched, the driver allows 256; 27 registers, 64 + 0 shared bytes (recorded 27, 64) +PASS limits live tf_q35_argmax_merge: 32 threads launched, the driver allows 32; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live sample tf_draw: 1024 threads launched, the driver allows 1024; 64 registers, 9752 + 0 shared bytes (recorded 64, 9752) +PASS limits live sample tf_logprob_rows: 1024 threads launched, the driver allows 1024; 40 registers, 520 + 0 shared bytes (recorded 40, 520) +PASS limits live sample tf_grammar_mask: 256 threads launched, the driver allows 256; 17 registers, 0 + 0 shared bytes (recorded 17, 0) +PASS limits live glm_glue tf_glm_top1: 1024 threads launched, the driver allows 1024; 36 registers, 8192 + 0 shared bytes (recorded 36, 8192) +FAIL limits live glm_glue tf_glm_lat_write_fp8: 512 threads launched, the driver allows 512; 25 registers, 64 + 0 shared bytes (recorded 0, 0) +FAIL limits live glm_glue tf_glm_chunks_16_fp8: 256 threads launched, the driver allows 256; 64 registers, 0 + 83328 shared bytes (recorded 0, 0) +FAIL limits live glm_glue tf_glm_chunks_32_fp8: 256 threads launched, the driver allows 256; 130 registers, 0 + 101120 shared bytes (recorded 0, 0) +PASS limits live sample tf_draw_pitched: 1024 threads launched, the driver allows 1024; 64 registers, 9752 + 0 shared bytes (recorded 64, 9752) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 121 registers, 32832 + 0 shared bytes (recorded 121, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 72 registers, 32832 + 0 shared bytes (recorded 72, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 167 registers, 32832 + 0 shared bytes (recorded 167, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 72 registers, 32832 + 0 shared bytes (recorded 72, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 168 registers, 32832 + 0 shared bytes (recorded 168, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi1ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 72 registers, 32832 + 0 shared bytes (recorded 72, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 142 registers, 32832 + 0 shared bytes (recorded 142, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi8ELi8ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 80 registers, 32832 + 0 shared bytes (recorded 80, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 164 registers, 32832 + 0 shared bytes (recorded 164, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi10ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 99 registers, 32832 + 0 shared bytes (recorded 99, 32832) +FAIL limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 166 registers, 32832 + 0 shared bytes (recorded 167, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi8ELi2ELi2ELi12ELi3EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 99 registers, 32832 + 0 shared bytes (recorded 99, 32832) +PASS limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi8ELi8ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 96 registers, 16448 + 0 shared bytes (recorded 96, 16448) +FAIL limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi10ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 124 registers, 16448 + 0 shared bytes (recorded 128, 16448) +FAIL limits live dsv41_x3ld _ZN10dsv41_x3ld9ld_kernelILi2ELi4ELi2ELi2ELi12ELi0EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 126 registers, 16448 + 0 shared bytes (recorded 128, 16448) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 186 registers, 16640 + 0 shared bytes (recorded 186, 16640) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 238 registers, 16640 + 0 shared bytes (recorded 238, 16640) +FAIL limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi4ELi4ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 240 registers, 16640 + 0 shared bytes (recorded 241, 16640) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 185 registers, 32896 + 0 shared bytes (recorded 185, 32896) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 250 registers, 32896 + 0 shared bytes (recorded 250, 32896) +PASS limits live dsv41_x3pf _ZN10dsv41_x3pf9pf_kernelILi2ELi8ELi2ELi2ELi12EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 250 registers, 32896 + 0 shared bytes (recorded 250, 32896) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_8_8: 128 threads launched, the driver allows 128; 125 registers, 32848 + 0 shared bytes (recorded 125, 32848) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_10: 128 threads launched, the driver allows 128; 167 registers, 32848 + 0 shared bytes (recorded 168, 32848) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_1_2_12: 128 threads launched, the driver allows 128; 167 registers, 32848 + 0 shared bytes (recorded 167, 32848) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_8_8: 128 threads launched, the driver allows 128; 146 registers, 32848 + 0 shared bytes (recorded 146, 32848) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_10: 128 threads launched, the driver allows 128; 163 registers, 32848 + 0 shared bytes (recorded 168, 32848) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_gu_2_2_12: 128 threads launched, the driver allows 128; 168 registers, 32848 + 0 shared bytes (recorded 166, 32848) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_8_8: 128 threads launched, the driver allows 128; 121 registers, 32896 + 0 shared bytes (recorded 121, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_10: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 167, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_1_2_12: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 167, 32896) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_8_8: 128 threads launched, the driver allows 128; 140 registers, 32896 + 0 shared bytes (recorded 140, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_10: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 164, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dn_2_2_12: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 164, 32896) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_8_8: 128 threads launched, the driver allows 128; 121 registers, 32896 + 0 shared bytes (recorded 121, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_10: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 167, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_1_2_12: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 167, 32896) +PASS limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_8_8: 128 threads launched, the driver allows 128; 140 registers, 32896 + 0 shared bytes (recorded 140, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_10: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 164, 32896) +FAIL limits live dsv41_x3ld_epi dsv41_x3ld_epi_dnb_2_2_12: 128 threads launched, the driver allows 128; 168 registers, 32896 + 0 shared bytes (recorded 164, 32896) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 118 registers, 32832 + 0 shared bytes (recorded 118, 32832) +FAIL limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 167 registers, 32832 + 0 shared bytes (recorded 166, 32832) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi1ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 168 registers, 32832 + 0 shared bytes (recorded 168, 32832) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 134 registers, 32832 + 0 shared bytes (recorded 134, 32832) +FAIL limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 168 registers, 32832 + 0 shared bytes (recorded 167, 32832) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi8ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 168 registers, 32832 + 0 shared bytes (recorded 168, 32832) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi8ELi8EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 89 registers, 16448 + 0 shared bytes (recorded 89, 16448) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi10EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 96 registers, 16448 + 0 shared bytes (recorded 96, 16448) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14grouped_kernelILi2ELi4ELi4ELi2ELi2ELi16EEEvPK6__halfS3_PKlS5_PKiS7_S7_S7_S7_Pfiiiiii: 128 threads launched, the driver allows 128; 128 registers, 16448 + 0 shared bytes (recorded 128, 16448) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi2EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi3EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi4EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi5EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi6EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi7EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 23 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi8EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 23 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi9EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi10EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi11EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi12EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi13EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi14EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi15EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN8tf_exl3x14dequant_kernelILi2ELi16EEEvPKjP6__halfii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts12group_kernelEPKiPiS2_S2_iiii: 1024 threads launched, the driver allows 1024; 42 registers, 128 + 49152 shared bytes (recorded 42, 128) +PASS limits live dsv41_router_glue router_group_rot: 1024 threads launched, the driver allows 1024; 25 registers, 6528 + 0 shared bytes (recorded 25, 6528) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI13__nv_bfloat16EEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts13rot_in_kernelI6__halfEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 32 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts22gateup_epilogue_kernelEPKfPKiPK6__halfS6_S6_PS4_iiiifi: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts20down_epilogue_kernelEPKfPKiPK6__halfPfiiii: 32 threads launched, the driver allows 1024; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts14combine_kernelEPKfS1_Pfii: 256 threads launched, the driver allows 1024; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelIfEEvPKfPKiPK6__halfPfS2_PT_iiiii: 1024 threads launched, the driver allows 1024; 40 registers, 16384 + 0 shared bytes (recorded 40, 16384) +PASS limits live dsv41_exl3_experts _ZN15tf_exl3_experts19down_combine_kernelI13__nv_bfloat16EEvPKfPKiPK6__halfPfS3_PT_iiiii: 1024 threads launched, the driver allows 1024; 40 registers, 16384 + 0 shared bytes (recorded 40, 16384) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 38912 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 58368 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 33792 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 33792 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 162 registers, 272 + 58368 shared bytes (recorded 162, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 38912 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 71680 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 238 registers, 528 + 53760 shared bytes (recorded 238, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi3ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 272 + 58368 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 161 registers, 272 + 38912 shared bytes (recorded 161, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 124 registers, 272 + 38912 shared bytes (recorded 124, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 166 registers, 528 + 71680 shared bytes (recorded 166, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 166 registers, 528 + 53760 shared bytes (recorded 166, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi3ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 243 registers, 528 + 67584 shared bytes (recorded 243, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi3EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 26 registers, 48 + 0 shared bytes (recorded 26, 48) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 36864 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 33792 shared bytes (recorded 128, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 40960 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 61440 shared bytes (recorded 128, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 33792 shared bytes (recorded 238, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 36864 shared bytes (recorded 128, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 162 registers, 272 + 61440 shared bytes (recorded 161, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 40960 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 127 registers, 528 + 73728 shared bytes (recorded 127, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 55296 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi4ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 272 + 61440 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 159 registers, 272 + 40960 shared bytes (recorded 159, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 124 registers, 272 + 40960 shared bytes (recorded 124, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 164 registers, 528 + 73728 shared bytes (recorded 164, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 164 registers, 528 + 55296 shared bytes (recorded 164, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi4ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 242 registers, 528 + 67584 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi4EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 20 registers, 64 + 0 shared bytes (recorded 20, 64) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 39936 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 43008 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 64512 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 33792 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 39936 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 151 registers, 272 + 64512 shared bytes (recorded 151, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 43008 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 75776 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 56832 shared bytes (recorded 240, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi5ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 174 registers, 272 + 64512 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 143 registers, 272 + 43008 shared bytes (recorded 143, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 43008 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 528 + 75776 shared bytes (recorded 168, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 167 registers, 528 + 56832 shared bytes (recorded 167, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi5ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 242 registers, 528 + 67584 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi5EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 26 registers, 80 + 0 shared bytes (recorded 26, 80) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 43008 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 45056 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 67584 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 33792 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 43008 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 151 registers, 272 + 67584 shared bytes (recorded 151, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 45056 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 77824 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 58368 shared bytes (recorded 240, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi6ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 174 registers, 272 + 67584 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 143 registers, 272 + 45056 shared bytes (recorded 143, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 45056 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 528 + 77824 shared bytes (recorded 168, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 167 registers, 528 + 58368 shared bytes (recorded 167, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi6ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 242 registers, 528 + 67584 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi6EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 26 registers, 96 + 0 shared bytes (recorded 26, 96) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 46080 shared bytes (recorded 128, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 47104 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 70656 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 35328 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 46080 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 155 registers, 272 + 70656 shared bytes (recorded 155, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 125 registers, 272 + 47104 shared bytes (recorded 125, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 127 registers, 528 + 79872 shared bytes (recorded 127, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 528 + 59904 shared bytes (recorded 242, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi7ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 182 registers, 272 + 70656 shared bytes (recorded 178, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 149 registers, 272 + 47104 shared bytes (recorded 149, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 123 registers, 272 + 47104 shared bytes (recorded 123, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 528 + 79872 shared bytes (recorded 168, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 160 registers, 528 + 59904 shared bytes (recorded 160, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi7ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 240 registers, 528 + 67584 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi7EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 22 registers, 112 + 0 shared bytes (recorded 22, 112) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 49152 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 33792 shared bytes (recorded 127, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 49152 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 73728 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 36864 shared bytes (recorded 240, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 49152 shared bytes (recorded 128, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 162 registers, 272 + 73728 shared bytes (recorded 162, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 127 registers, 272 + 49152 shared bytes (recorded 127, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 126 registers, 528 + 81920 shared bytes (recorded 125, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 238 registers, 528 + 61440 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi8ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 168 registers, 272 + 73728 shared bytes (recorded 168, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 149 registers, 272 + 49152 shared bytes (recorded 149, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 125 registers, 272 + 49152 shared bytes (recorded 125, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 164 registers, 528 + 81920 shared bytes (recorded 164, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 167 registers, 528 + 61440 shared bytes (recorded 167, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi8ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 242 registers, 528 + 67584 shared bytes (recorded 242, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi8EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 20 registers, 128 + 0 shared bytes (recorded 20, 128) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 157 registers, 272 + 55296 shared bytes (recorded 157, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 272 + 36864 shared bytes (recorded 126, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 53248 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi1ELi16ELi4ELi3ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 79872 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 528 + 39936 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi1ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 155 registers, 272 + 55296 shared bytes (recorded 155, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 155 registers, 272 + 79872 shared bytes (recorded 155, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 158 registers, 272 + 53248 shared bytes (recorded 156, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 127 registers, 528 + 86016 shared bytes (recorded 127, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi16ELi2ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 238 registers, 528 + 64512 shared bytes (recorded 238, 528) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi2ELi2ELi10ELi2ELi8ELi2ELi6ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 158 registers, 272 + 79872 shared bytes (recorded 156, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 149 registers, 272 + 53248 shared bytes (recorded 149, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 119 registers, 272 + 53248 shared bytes (recorded 119, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi4ELi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 152 registers, 528 + 86016 shared bytes (recorded 152, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi1ELi16ELi2ELi6ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 160 registers, 528 + 64512 shared bytes (recorded 160, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm9gm_kernelILi1ELi1ELi10ELi2ELi16ELi4ELi3ELi16EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 240 registers, 528 + 67584 shared bytes (recorded 240, 528) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm14dequant_kernelILi10EEEvPKjP6__halfi: 32 threads launched, the driver allows 32; 21 registers, 160 + 0 shared bytes (recorded 21, 160) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 49152 shared bytes (recorded 128, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi1ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 36864 shared bytes (recorded 128, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi4ELi3ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 252 registers, 272 + 79872 shared bytes (recorded 248, 272) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 272 + 49152 shared bytes (recorded 128, 272) +FAIL limits live dsv41_x3gm _ZN10dsv41_x3gm10gm2_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 253 registers, 272 + 49152 shared bytes (recorded 251, 272) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm2_kernelILi2ELi2ELi1ELi16ELi2ELi4ELi4EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 528 + 86016 shared bytes (recorded 128, 528) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi2ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi2ELi4ELi4ELi1EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 544 + 53248 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi2ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 544 + 79872 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi2ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi2ELi1ELi1ELi8ELi4ELi3ELi4ELi1EEEvNS_4ArgsE: 512 threads launched, the driver allows 512; 128 registers, 544 + 55296 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 250 registers, 544 + 49152 shared bytes (recorded 250, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi2ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 255 registers, 544 + 49152 shared bytes (recorded 255, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi4ELi4ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 49152 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm3 _ZN10dsv41_x3gm10gm3_kernelILi1ELi1ELi1ELi8ELi2ELi6ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 544 + 39936 shared bytes (recorded 128, 544) +PASS limits live dsv41_x3gm_plan _ZN15dsv41_x3gm_plan11plan_kernelEPKiiiiPiS2_S2_S2_S2_: 512 threads launched, the driver allows 512; 60 registers, 38980 + 0 shared bytes (recorded 60, 38980) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li1EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI13__nv_bfloat16Li2EEEvPKT_iPKiPK6__halfS9_PS7_SA_iii: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi1EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_x3gm _ZN10dsv41_x3gm10rot_kernelI6__halfLi2EEEvPKT_iPKiPKS1_S8_PS1_S9_iii: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13rot_in_kernelEPKviPK6__halfPS2_i: 128 threads launched, the driver allows 128; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 127 registers, 16 + 8192 shared bytes (recorded 127, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 115 registers, 16 + 16384 shared bytes (recorded 115, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi2ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 122 registers, 16 + 32768 shared bytes (recorded 122, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi2ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 161 registers, 16 + 8192 shared bytes (recorded 161, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 161 registers, 16 + 16384 shared bytes (recorded 161, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi3ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 153 registers, 16 + 32768 shared bytes (recorded 153, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi3ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 126 registers, 16 + 8192 shared bytes (recorded 126, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 117 registers, 16 + 16384 shared bytes (recorded 117, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi4ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 126 registers, 16 + 32768 shared bytes (recorded 126, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi4ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 162 registers, 16 + 8192 shared bytes (recorded 162, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 162 registers, 16 + 16384 shared bytes (recorded 162, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi5ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 154 registers, 16 + 32768 shared bytes (recorded 154, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi5ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 165 registers, 16 + 8192 shared bytes (recorded 165, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 165 registers, 16 + 16384 shared bytes (recorded 165, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi6ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 154 registers, 16 + 32768 shared bytes (recorded 154, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi6ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 166 registers, 16 + 8192 shared bytes (recorded 166, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi7ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 228 registers, 16 + 32768 shared bytes (recorded 228, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi7ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 156 registers, 16 + 8192 shared bytes (recorded 156, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 154 registers, 16 + 16384 shared bytes (recorded 154, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi8ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi8ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 167 registers, 16 + 16384 shared bytes (recorded 167, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi10ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 220 registers, 16 + 32768 shared bytes (recorded 220, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi10ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 167 registers, 16 + 16384 shared bytes (recorded 167, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi12ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 218 registers, 16 + 32768 shared bytes (recorded 218, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi12ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 23 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 127 registers, 16 + 8192 shared bytes (recorded 127, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 127 registers, 16 + 16384 shared bytes (recorded 127, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi14ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi14ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 21 registers, 0 + 0 shared bytes (recorded 21, 0) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi2EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 126 registers, 16 + 8192 shared bytes (recorded 126, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi4EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 122 registers, 16 + 16384 shared bytes (recorded 122, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13linear_kernelILi16ELi2ELi8EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 127 registers, 16 + 32768 shared bytes (recorded 127, 16) +PASS limits live dsv41_linear _ZN14tf_exl3_linear13unpack_kernelILi16ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 22 registers, 0 + 0 shared bytes (recorded 22, 0) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_rot_in_kernelENS_8RotTableE: 128 threads launched, the driver allows 128; 38 registers, 32 + 0 shared bytes (recorded 38, 32) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 183 registers, 16 + 16384 shared bytes (recorded 183, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi4ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 178 registers, 16 + 32768 shared bytes (recorded 178, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 238 registers, 16 + 16384 shared bytes (recorded 238, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi6ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 233 registers, 16 + 32768 shared bytes (recorded 233, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 205 registers, 16 + 16384 shared bytes (recorded 205, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi8ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 201 registers, 16 + 32768 shared bytes (recorded 201, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 255 registers, 16 + 16384 shared bytes (recorded 255, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi10ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 255 registers, 16 + 32768 shared bytes (recorded 255, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 255 registers, 16 + 16384 shared bytes (recorded 255, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi12ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 255 registers, 16 + 32768 shared bytes (recorded 255, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi3EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi4ELi2EEEvNS_8LinTableEii: 128 threads launched, the driver allows 128; 172 registers, 16 + 16384 shared bytes (recorded 172, 16) +PASS limits live dsv41_x3seg _ZN11dsv41_x3seg17seg_linear_kernelILi16ELi2ELi8ELi1EEEvNS_8LinTableEii: 256 threads launched, the driver allows 256; 168 registers, 16 + 32768 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 192 registers, 16 + 32768 shared bytes (recorded 192, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi4ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi4ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 196 registers, 16 + 32768 shared bytes (recorded 196, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi6ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi6ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 31 registers, 0 + 0 shared bytes (recorded 31, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 206 registers, 16 + 32768 shared bytes (recorded 206, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi8ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi8ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 29 registers, 0 + 0 shared bytes (recorded 29, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 212 registers, 16 + 32768 shared bytes (recorded 212, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi10ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi10ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 37 registers, 0 + 0 shared bytes (recorded 37, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi3EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi4ELi4EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 128 registers, 16 + 16384 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 222 registers, 16 + 32768 shared bytes (recorded 222, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_linear_kernelILi12ELi2ELi8ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 128 registers, 16 + 32768 shared bytes (recorded 128, 16) +PASS limits live dsv41_dense3 _ZN12dsv41_dense319lanes_unpack_kernelILi12ELi2EEEvPKjP6__halfill: 32 threads launched, the driver allows 32; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense315to_lanes_kernelEPKjPjxii: 256 threads launched, the driver allows 1024; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live dsv41_dense3 _ZN12dsv41_dense316to_strips_kernelEPKjPjxii: 256 threads launched, the driver allows 1024; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 246 registers, 16 + 16384 shared bytes (recorded 246, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi4ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 239 registers, 16 + 32768 shared bytes (recorded 239, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 244 registers, 16 + 16384 shared bytes (recorded 244, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi6ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 245 registers, 16 + 32768 shared bytes (recorded 245, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 240 registers, 16 + 16384 shared bytes (recorded 240, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi8ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 242 registers, 16 + 32768 shared bytes (recorded 242, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 248 registers, 16 + 16384 shared bytes (recorded 248, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi10ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 248 registers, 16 + 32768 shared bytes (recorded 248, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi4ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 243 registers, 16 + 16384 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanes2_kernelILi12ELi2ELi8ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 242 registers, 16 + 32768 shared bytes (recorded 242, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 192 registers, 16 + 32768 shared bytes (recorded 192, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 226 registers, 16 + 8192 shared bytes (recorded 226, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 226 registers, 16 + 16384 shared bytes (recorded 226, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 226 registers, 16 + 32768 shared bytes (recorded 226, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 194 registers, 16 + 32768 shared bytes (recorded 194, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 232 registers, 16 + 8192 shared bytes (recorded 232, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 232 registers, 16 + 16384 shared bytes (recorded 232, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 232 registers, 16 + 32768 shared bytes (recorded 232, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 230 registers, 16 + 8192 shared bytes (recorded 230, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 230 registers, 16 + 16384 shared bytes (recorded 230, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 230 registers, 16 + 32768 shared bytes (recorded 230, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 243 registers, 16 + 8192 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 243 registers, 16 + 16384 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 243 registers, 16 + 32768 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 212 registers, 16 + 8192 shared bytes (recorded 212, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 212 registers, 16 + 16384 shared bytes (recorded 212, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 212 registers, 16 + 32768 shared bytes (recorded 212, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 243 registers, 16 + 8192 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 243 registers, 16 + 16384 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 243 registers, 16 + 32768 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 243 registers, 16 + 8192 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 243 registers, 16 + 16384 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 243 registers, 16 + 32768 shared bytes (recorded 243, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 173 registers, 16 + 32768 shared bytes (recorded 173, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb1EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 136 registers, 0 + 24576 shared bytes (recorded 136, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 136 registers, 0 + 40960 shared bytes (recorded 136, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 241 registers, 0 + 32768 shared bytes (recorded 241, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi4ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 241 registers, 0 + 49152 shared bytes (recorded 241, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 147 registers, 0 + 24576 shared bytes (recorded 147, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 147 registers, 0 + 40960 shared bytes (recorded 147, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 241 registers, 0 + 32768 shared bytes (recorded 241, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi6ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 241 registers, 0 + 49152 shared bytes (recorded 241, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 145 registers, 0 + 24576 shared bytes (recorded 145, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 145 registers, 0 + 40960 shared bytes (recorded 145, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 239 registers, 0 + 32768 shared bytes (recorded 239, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi8ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 239 registers, 0 + 49152 shared bytes (recorded 239, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 159 registers, 0 + 24576 shared bytes (recorded 159, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 164 registers, 0 + 40960 shared bytes (recorded 164, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 248 registers, 0 + 32768 shared bytes (recorded 248, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi10ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 248 registers, 0 + 49152 shared bytes (recorded 248, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi3ELi1EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 156 registers, 0 + 24576 shared bytes (recorded 156, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi1EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 160 registers, 0 + 40960 shared bytes (recorded 160, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi4ELi2ELi2EEEvN11dsv41_x3seg8LinTableEii: 128 threads launched, the driver allows 128; 245 registers, 0 + 32768 shared bytes (recorded 245, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin213lanesc_kernelILi12ELi2ELi8ELi1ELi2EEEvN11dsv41_x3seg8LinTableEii: 256 threads launched, the driver allows 256; 245 registers, 0 + 49152 shared bytes (recorded 245, 0) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 167 registers, 16 + 8192 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi2ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 173 registers, 16 + 32768 shared bytes (recorded 173, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 167 registers, 16 + 8192 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi3ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 171 registers, 16 + 32768 shared bytes (recorded 171, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 167 registers, 16 + 8192 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 166 registers, 16 + 16384 shared bytes (recorded 166, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi4ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 173 registers, 16 + 32768 shared bytes (recorded 173, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 167 registers, 16 + 16384 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi5ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 167 registers, 16 + 16384 shared bytes (recorded 167, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi6ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi7ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 174 registers, 16 + 32768 shared bytes (recorded 174, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 166 registers, 16 + 8192 shared bytes (recorded 166, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi8ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi10ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 174 registers, 16 + 32768 shared bytes (recorded 174, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi12ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 174 registers, 16 + 32768 shared bytes (recorded 174, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi14ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 173 registers, 16 + 32768 shared bytes (recorded 173, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi2ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 64 threads launched, the driver allows 64; 168 registers, 16 + 8192 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi4ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 128 threads launched, the driver allows 128; 168 registers, 16 + 16384 shared bytes (recorded 168, 16) +PASS limits live dsv41_lin2 _ZN10dsv41_lin214linear2_kernelILi16ELi2ELi8ELb0EEEvPK6__halfPKjxxS3_S3_PviPfPiiiiiPKi: 256 threads launched, the driver allows 256; 170 registers, 16 + 32768 shared bytes (recorded 170, 16) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 114 registers, 0 + 98816 shared bytes (recorded 117, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb0ELb1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 114 registers, 0 + 98816 shared bytes (recorded 117, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 114 registers, 0 + 98816 shared bytes (recorded 117, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi2ELb1ELb1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 114 registers, 0 + 98816 shared bytes (recorded 117, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 111 registers, 0 + 98816 shared bytes (recorded 113, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb0ELb1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 111 registers, 0 + 98816 shared bytes (recorded 113, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb0EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 111 registers, 0 + 98816 shared bytes (recorded 113, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn11attn_kernelILi4ELb1ELb1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 111 registers, 0 + 98816 shared bytes (recorded 113, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 224 registers, 0 + 98816 shared bytes (recorded 238, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb0ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb0ELb1ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb0ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 90, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi2ELb1ELb1ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 90, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb0ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb0ELb1ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb0ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_attn _ZN10dsv41_attn12split_kernelILi4ELb1ELb1ELi4EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 0 + 100864 shared bytes (recorded 88, 0) +FAIL limits live dsv41_topk _ZN10dsv41_topk11topk_kernelENS_4ArgsE: 512 threads launched, the driver allows 512; 55 registers, 3232 + 63488 shared bytes (recorded 54, 3232) +PASS limits live dsv41_topk_b _ZN10dsv41_topk13topk_b_kernelENS_4ArgsE: 512 threads launched, the driver allows 512; 55 registers, 3232 + 63488 shared bytes (recorded 55, 3232) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 113 registers, 20496 + 2048 shared bytes (recorded 113, 20496) +FAIL limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 125 registers, 21392 + 2048 shared bytes (recorded 124, 21392) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 20496 + 2048 shared bytes (recorded 80, 20496) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 62 registers, 21392 + 2048 shared bytes (recorded 62, 21392) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 20496 + 2048 shared bytes (recorded 80, 20496) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 89 registers, 21392 + 2048 shared bytes (recorded 89, 21392) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 96 registers, 20496 + 2048 shared bytes (recorded 96, 20496) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 113 registers, 20656 + 2048 shared bytes (recorded 113, 20656) +FAIL limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 21552 + 2048 shared bytes (recorded 126, 21552) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 20656 + 2048 shared bytes (recorded 80, 20656) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi1ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 64 registers, 21552 + 2048 shared bytes (recorded 64, 21552) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 20656 + 2048 shared bytes (recorded 80, 20656) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb0ELi2ELb1ELb0ELb1ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 91 registers, 21552 + 2048 shared bytes (recorded 91, 21552) +PASS limits live dsv41_mhc _ZN9dsv41_mhc15boundary_kernelILb1ELi2ELb0ELb1ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 94 registers, 20656 + 2048 shared bytes (recorded 94, 20656) +PASS limits live dsv41_mhc _ZN9dsv41_mhc11coef_kernelENS_4ArgsE: 128 threads launched, the driver allows 128; 22 registers, 18048 + 0 shared bytes (recorded 22, 18048) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 141 registers, 0 + 47872 shared bytes (recorded 141, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb0ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 143 registers, 0 + 47872 shared bytes (recorded 143, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 141 registers, 0 + 72448 shared bytes (recorded 141, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb1ELi2ELb1ELb1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 143 registers, 0 + 72448 shared bytes (recorded 143, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 47872 shared bytes (recorded 88, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi1ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 72448 shared bytes (recorded 88, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb0ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 150 registers, 0 + 47872 shared bytes (recorded 150, 0) +PASS limits live dsv41_mhc_pf _ZN12dsv41_mhc_pf11site_kernelILb0ELi2ELb1ELb0EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 148 registers, 0 + 72448 shared bytes (recorded 148, 0) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 84 registers, 2052 + 0 shared bytes (recorded 84, 2052) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 84 registers, 2052 + 0 shared bytes (recorded 84, 2052) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 79 registers, 2052 + 0 shared bytes (recorded 79, 2052) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi1ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 79 registers, 2052 + 0 shared bytes (recorded 79, 2052) +FAIL limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 89 registers, 10244 + 0 shared bytes (recorded 92, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 100 registers, 20484 + 0 shared bytes (recorded 100, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 100 registers, 40964 + 0 shared bytes (recorded 100, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi1ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 101 registers, 40964 + 0 shared bytes (recorded 101, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 80 registers, 4100 + 0 shared bytes (recorded 80, 4100) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 80 registers, 4100 + 0 shared bytes (recorded 80, 4100) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 86 registers, 4100 + 0 shared bytes (recorded 86, 4100) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi2ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 86 registers, 4100 + 0 shared bytes (recorded 86, 4100) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 110 registers, 10244 + 0 shared bytes (recorded 110, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 111 registers, 20484 + 0 shared bytes (recorded 111, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 111 registers, 40964 + 0 shared bytes (recorded 111, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi2ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 118 registers, 40964 + 0 shared bytes (recorded 118, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 84 registers, 8196 + 0 shared bytes (recorded 84, 8196) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 8196 + 0 shared bytes (recorded 88, 8196) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 80 registers, 8196 + 0 shared bytes (recorded 80, 8196) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi4ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 90 registers, 8196 + 0 shared bytes (recorded 90, 8196) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 191 registers, 10244 + 0 shared bytes (recorded 191, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 192 registers, 20484 + 0 shared bytes (recorded 192, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 205 registers, 40964 + 0 shared bytes (recorded 205, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi4ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 40964 + 0 shared bytes (recorded 128, 40964) +FAIL limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 104 registers, 16388 + 0 shared bytes (recorded 91, 16388) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 109 registers, 16388 + 0 shared bytes (recorded 109, 16388) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 105 registers, 16388 + 0 shared bytes (recorded 105, 16388) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi8ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 116 registers, 16388 + 0 shared bytes (recorded 116, 16388) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 142 registers, 5636 + 0 shared bytes (recorded 142, 5636) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 140 registers, 10244 + 0 shared bytes (recorded 140, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 140 registers, 20484 + 0 shared bytes (recorded 140, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi8ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 142 registers, 40964 + 0 shared bytes (recorded 142, 40964) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi1EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 122 registers, 32772 + 0 shared bytes (recorded 122, 32772) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi1EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 199 registers, 32772 + 0 shared bytes (recorded 199, 32772) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi8ELi2EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 177 registers, 32772 + 0 shared bytes (recorded 177, 32772) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg11gemv_kernelILi16ELi4ELi2EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 216 registers, 32772 + 0 shared bytes (recorded 216, 32772) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi1EEEvNS_4ArgsE: 32 threads launched, the driver allows 32; 210 registers, 5636 + 0 shared bytes (recorded 210, 5636) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi2EEEvNS_4ArgsE: 64 threads launched, the driver allows 64; 158 registers, 10244 + 0 shared bytes (recorded 158, 10244) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi4EEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 158 registers, 20484 + 0 shared bytes (recorded 158, 20484) +PASS limits live dsv41_router_gemv _ZN8dsv41_rg13narrow_kernelILi16ELi8EEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 170 registers, 40964 + 0 shared bytes (recorded 170, 40964) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 47104 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 132 registers, 0 + 57344 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 47104 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 40960 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 132 registers, 0 + 40960 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 36864 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 32768 shared bytes (recorded 88, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 0 + 71680 shared bytes (recorded 242, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb0EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 26 registers, 10240 + 0 shared bytes (recorded 26, 10240) +FAIL limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 47104 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 134 registers, 0 + 57344 shared bytes (recorded 134, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 47104 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 40960 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 132 registers, 0 + 40960 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 36864 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 32768 shared bytes (recorded 88, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd10pfd_kernelILi8ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 0 + 71680 shared bytes (recorded 240, 0) +PASS limits live dsv41_pfdense_k8 _ZN9dsv41_pfd18pfd_dequant_kernelILi8ELb1EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 30 registers, 10240 + 0 shared bytes (recorded 30, 10240) +FAIL limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 48640 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 133 registers, 0 + 59392 shared bytes (recorded 133, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 48640 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 43008 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 43008 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 37888 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 87 registers, 0 + 34816 shared bytes (recorded 87, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 244 registers, 0 + 73216 shared bytes (recorded 244, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb0EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 31 registers, 10752 + 0 shared bytes (recorded 31, 10752) +FAIL limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 48640 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 136 registers, 0 + 59392 shared bytes (recorded 136, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 48640 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 132 registers, 0 + 43008 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 133 registers, 0 + 43008 shared bytes (recorded 133, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 37888 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 91 registers, 0 + 34816 shared bytes (recorded 91, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd10pfd_kernelILi10ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 0 + 73216 shared bytes (recorded 242, 0) +PASS limits live dsv41_pfdense_k10 _ZN9dsv41_pfd18pfd_dequant_kernelILi10ELb1EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 33 registers, 10752 + 0 shared bytes (recorded 33, 10752) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 50176 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 133 registers, 0 + 61440 shared bytes (recorded 133, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 50176 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 45056 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 45056 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 38912 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 87 registers, 0 + 36864 shared bytes (recorded 87, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 244 registers, 0 + 74752 shared bytes (recorded 244, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb0EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 31 registers, 11264 + 0 shared bytes (recorded 31, 11264) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 50176 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 134 registers, 0 + 61440 shared bytes (recorded 134, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 50176 shared bytes (recorded 128, 0) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 132 registers, 0 + 45056 shared bytes (recorded 128, 0) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 133 registers, 0 + 45056 shared bytes (recorded 134, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 38912 shared bytes (recorded 126, 0) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 91 registers, 0 + 36864 shared bytes (recorded 90, 0) +PASS limits live dsv41_pfdense_k12 _ZN9dsv41_pfd10pfd_kernelILi12ELb1ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 242 registers, 0 + 74752 shared bytes (recorded 242, 0) +FAIL limits live dsv41_pfdense_k12 _ZN9dsv41_pfd18pfd_dequant_kernelILi12ELb1EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 31 registers, 11264 + 0 shared bytes (recorded 32, 11264) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 132 registers, 0 + 53248 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi4EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 132 registers, 0 + 65536 shared bytes (recorded 132, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 128 registers, 0 + 53248 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi2ELi2ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 49152 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi64ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 128 registers, 0 + 49152 shared bytes (recorded 128, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi128ELi2ELi4ELi2ELi2EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 126 registers, 0 + 40960 shared bytes (recorded 126, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi32ELi1ELi4ELi2ELi4EEEEEvNS_4ArgsE: 128 threads launched, the driver allows 128; 88 registers, 0 + 40960 shared bytes (recorded 88, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd10pfd_kernelILi16ELb0ENS_3CfgILi256ELi4ELi2ELi2ELi3EEEEEvNS_4ArgsE: 256 threads launched, the driver allows 256; 240 registers, 0 + 77824 shared bytes (recorded 240, 0) +PASS limits live dsv41_pfdense_k16 _ZN9dsv41_pfd18pfd_dequant_kernelILi16ELb0EEEvPKjxxP6__halfi: 128 threads launched, the driver allows 128; 27 registers, 12288 + 0 shared bytes (recorded 27, 12288) +PASS limits live dsv41_engram_gate _ZN17dsv41_engram_gate16gate_copy_kernelEPxiiPKhxPhxxi: 256 threads launched, the driver allows 1024; 48 registers, 8 + 0 shared bytes (recorded 48, 8) +PASS limits live dsv41_l2pace _ZN12dsv41_l2pace12paced_kernelEPKlijmm: 32 threads launched, the driver allows 32; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_l2pf _ZN6tfl2pf15segments_kernelILb0EEEvPKlij: 256 threads launched, the driver allows 256; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live dsv41_l2pf _ZN6tfl2pf15segments_kernelILb1EEEvPKlij: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_topk_keys _ZN15dsv41_topk_keys11topk_kernelEPKfxiiPfPx: 1024 threads launched, the driver allows 1024; 64 registers, 9232 + 0 shared bytes (recorded 64, 9232) +PASS limits live dsv41_pointwise _ZN15dsv41_pointwise13swiglu_kernelEPKfxS1_xPfxif: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_pointwise _ZN15dsv41_pointwise10add_kernelEPfPKfx: 256 threads launched, the driver allows 256; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_pointwise _ZN15dsv41_pointwise15add_bf16_kernelEPKfS1_P13__nv_bfloat16x: 256 threads launched, the driver allows 256; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit12dense_kernelENS_9DenseArgsE: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 34, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit13gather_kernelEPKhjPKjjPh: 256 threads launched, the driver allows 256; 30 registers, 0 + 0 shared bytes (recorded 30, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit11pack_kernelENS_8PackArgsE: 256 threads launched, the driver allows 256; 36 registers, 2336 + 0 shared bytes (recorded 36, 2336) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_mark_kernelENS_9DedupArgsE: 256 threads launched, the driver allows 256; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit18dedup_flags_kernelENS_9DedupArgsE: 256 threads launched, the driver allows 256; 24 registers, 32 + 0 shared bytes (recorded 24, 32) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit17dedup_pack_kernelENS_9DedupArgsE: 256 threads launched, the driver allows 256; 37 registers, 2336 + 0 shared bytes (recorded 37, 2336) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit16dedup_tok_kernelENS_9DedupArgsE: 256 threads launched, the driver allows 256; 35 registers, 0 + 0 shared bytes (recorded 35, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit18hist_gather_kernelENS_8HistArgsE: 256 threads launched, the driver allows 256; 38 registers, 0 + 0 shared bytes (recorded 38, 0) +PASS limits live dsv41_kvsplit _ZN13dsv41_kvsplit15hist_tok_kernelENS_8HistArgsE: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue17embed_send_kernelEPKxiPKtxiiPt: 256 threads launched, the driver allows 256; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16embed_sum_kernelEPKtiiiPt: 256 threads launched, the driver allows 256; 64 registers, 0 + 0 shared bytes (recorded 64, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16cast_bf16_kernelEPKfPtx: 256 threads launched, the driver allows 256; 23 registers, 0 + 0 shared bytes (recorded 23, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue18kit_weights_kernelEPKfPfx: 256 threads launched, the driver allows 256; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue17kit_logits_kernelEPfx: 256 threads launched, the driver allows 256; 24 registers, 0 + 0 shared bytes (recorded 24, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16positions_kernelEiiPiPxS0_: 256 threads launched, the driver allows 256; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue18gather_cols_kernelEPKtiiiPt: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue12carry_kernelEPKtxPKiiiPf: 256 threads launched, the driver allows 256; 16 registers, 0 + 0 shared bytes (recorded 16, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue20positions_dev_kernelEPKiiPiPxS2_: 256 threads launched, the driver allows 256; 12 registers, 0 + 0 shared bytes (recorded 12, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16widen_f32_kernelEPKtPfx: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue15ds_cands_kernelEPKfxiiiPf: 1024 threads launched, the driver allows 1024; 42 registers, 2080 + 0 shared bytes (recorded 42, 2080) +PASS limits live dsv41_glue _ZN10dsv41_glue16pick_pack_kernelEPKfPKxixPd: 256 threads launched, the driver allows 256; 14 registers, 0 + 0 shared bytes (recorded 14, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue17pick_merge_kernelEPKdiiPKxxPx: 1024 threads launched, the driver allows 1024; 26 registers, 0 + 0 shared bytes (recorded 26, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue15ds_stage_kernelEPKxiPxPiiiixiiiiiii: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue21ds_accept_rows_kernelEPKxS1_S1_iPx: 32 threads launched, the driver allows 32; 20 registers, 0 + 0 shared bytes (recorded 20, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue20ds_stage_rows_kernelEPKxS1_iPxPiiiiiiiiiii: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_glue _ZN10dsv41_glue16vs_choose_kernelEPKfiiiiyxidddPx: 256 threads launched, the driver allows 256; 60 registers, 24576 + 0 shared bytes (recorded 60, 24576) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue20top_positions_kernelEPKxxiiPi: 1024 threads launched, the driver allows 1024; 48 registers, 9244 + 0 shared bytes (recorded 48, 9244) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue16cand_keys_kernelEPKixiiiPi: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue13counts_kernelEPKiiPKxiiPi: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue15gm_picks_kernelEPKiPKfiiiPiPf: 256 threads launched, the driver allows 256; 32 registers, 0 + 0 shared bytes (recorded 32, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue17width_mask_kernelEPKixS1_iiPi: 256 threads launched, the driver allows 256; 27 registers, 0 + 0 shared bytes (recorded 27, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue19f64_bf16_f32_kernelEPKdPfx: 256 threads launched, the driver allows 256; 28 registers, 0 + 0 shared bytes (recorded 28, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue16widen_cat_kernelEPKtS1_iiPf: 256 threads launched, the driver allows 256; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +PASS limits live dsv41_pfglue _ZN12dsv41_pfglue16ring_copy_kernelEPKhiPhixxx: 256 threads launched, the driver allows 256; 36 registers, 0 + 0 shared bytes (recorded 36, 0) +PASS limits live dsv41_kv_glue kv_norm_store: 128 threads launched, the driver allows 128; 26 registers, 1056 + 0 shared bytes (recorded 26, 1056) +PASS limits live dsv41_sampling _ZN14dsv41_sampling12stats_kernelEPKfxidPd: 1024 threads launched, the driver allows 1024; 43 registers, 264 + 0 shared bytes (recorded 43, 264) +PASS limits live dsv41_sampling _ZN14dsv41_sampling11pack_kernelEPKfPKxiiPf: 256 threads launched, the driver allows 256; 18 registers, 0 + 0 shared bytes (recorded 18, 0) +PASS limits live dsv41_sampling dsv41_sampling_stats_max: 256 threads launched, the driver allows 256; 39 registers, 32 + 0 shared bytes (recorded 39, 32) +PASS limits live dsv41_sampling dsv41_sampling_stats_sum: 256 threads launched, the driver allows 256; 38 registers, 64 + 0 shared bytes (recorded 38, 64) +PASS limits live dsv41_sampling dsv41_sampling_stats_fin: 32 threads launched, the driver allows 32; 40 registers, 0 + 0 shared bytes (recorded 40, 0) +FAIL limits live tree_round tf_round_bf16_argmax: 256 threads launched, the driver allows 256; 23 registers, 3072 + 0 shared bytes (recorded 0, 0) +FAIL limits live tree_round tf_tree_round_match: 64 threads launched, the driver allows 1024; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live tree_round tf_round_keep_rows: 16 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live tree_round tf_round_gather_rows: 256 threads launched, the driver allows 1024; 18 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live tree_round tf_round_keep_head: 256 threads launched, the driver allows 1024; 20 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_swiglu_rows: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_swiglu_rows, 16 rows or fewer: 128 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_gate_rows: 256 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_gate_rows, 16 rows or fewer: 64 threads launched, the driver allows 256; 34 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_group_sums_rows: 256 threads launched, the driver allows 256; 16 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live act_rows tf_dconv_rows: 256 threads launched, the driver allows 256; 37 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_64: 128 threads launched, the driver allows 128; 46 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_64, 16 rows or fewer: 64 threads launched, the driver allows 128; 46 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_128: 128 threads launched, the driver allows 128; 77 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_128, 16 rows or fewer: 64 threads launched, the driver allows 128; 77 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_256: 128 threads launched, the driver allows 128; 145 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live gdn_prep tf_gdn_prep_256, 16 rows or fewer: 64 threads launched, the driver allows 128; 145 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live qk_rows tf_qk_rows at dim 64: 64 threads launched, the driver allows 256; 29 registers, 32 + 0 shared bytes (recorded 0, 0) +FAIL limits live qk_rows tf_qk_rows at dim 256: 256 threads launched, the driver allows 256; 29 registers, 32 + 0 shared bytes (recorded 0, 0) +FAIL limits live qk_rows tf_qk_rows_w: 128 threads launched, the driver allows 128; 40 registers, 0 + 0 shared bytes (recorded 0, 0) +FAIL limits live affine4 verify32 one CTA a tile: 128 threads launched, the driver allows 128; 72 registers, 0 + 32640 shared bytes (recorded 92, 0) +FAIL limits live affine4 verify32 cluster of 8: 128 threads launched, the driver allows 128; 72 registers, 0 + 32640 shared bytes (recorded 92, 0) +FAIL limits live affine4 verify64 one CTA a tile: 128 threads launched, the driver allows 128; 122 registers, 0 + 43008 shared bytes (recorded 124, 0) +FAIL limits live affine4 verify64 cluster of 8: 128 threads launched, the driver allows 128; 122 registers, 0 + 43008 shared bytes (recorded 124, 0) +FAIL limits live affine4 verify128 one CTA a tile: 128 threads launched, the driver allows 128; 252 registers, 0 + 57600 shared bytes (recorded 240, 0) +FAIL limits live affine4 verify128 cluster of 8: 128 threads launched, the driver allows 128; 252 registers, 0 + 57600 shared bytes (recorded 240, 0) +RED limits live control: past its bounds: 512 threads launched, the driver allows 256; 10 registers, 0 + 0 shared bytes (recorded 10, 0) +RED limits live control: heavy at 1,024 threads: 1024 threads launched, the driver allows 384; 166 registers, 0 + 0 shared bytes (recorded 166, 0) +SUMMARY 148 kernels: 148 pass gates 1-4, 0 fail, 1 known findings; 32 controls: 32 red as planted, 0 missed; 1018 limit problems; gates 883131 ms, limits 76092 ms +FAIL kernel-contract: 0 gate failures, 0 controls missed, 1018 limit problems +FAIL kernel-contract: TestFailed +kernel-contract ++- run exe tf-cuda-test failure +error: process exited with code 1 +failed command: ./.zig-cache/o/131e3ff6c7e622ff0b9271fc971ea66e/tf-cuda-test kernel-contract + +Build Summary: 134/136 steps succeeded (1 failed) +kernel-contract transitive failure ++- run exe tf-cuda-test failure diff --git a/docs/receipts/glm-kv-pages-0b094b74/single/reg/glm-lanes-concurrent.log b/docs/receipts/glm-kv-pages-0b094b74/single/reg/glm-lanes-concurrent.log new file mode 100644 index 0000000000..cd8eb49304 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single/reg/glm-lanes-concurrent.log @@ -0,0 +1,39 @@ +info: checkpoint digest: 0.0 GiB read in 0.0 s, 1/1 cached files +PASS all at once: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS all at once: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS all at once: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS all at once: 3 streams at most in one round, 127 rounds with two or more +PASS staggered by 5 rounds: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS staggered by 5 rounds: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS staggered by 5 rounds: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS staggered by 5 rounds: 3 streams at most in one round, 127 rounds with two or more +PASS in reverse: stream 0 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS in reverse: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS in reverse: stream 2 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS in reverse: 3 streams at most in one round, 127 rounds with two or more +PASS the same prompt twice: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS the same prompt twice: stream 1 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS the same prompt twice: 2 streams at most in one round, 127 rounds with two or more +PASS sampled: stream 0 (prompt 0, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled: stream 1 (prompt 1, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled: stream 2 (prompt 2, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled: 3 streams at most in one round, 127 rounds with two or more +PASS the second discarded after 6 rounds: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS the second discarded after 6 rounds: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS the second discarded after 6 rounds: 3 streams at most in one round, 127 rounds with two or more +PASS prompts a chunk at a time while the others decode: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS prompts a chunk at a time while the others decode: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS prompts a chunk at a time while the others decode: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS prompts a chunk at a time while the others decode: 3 streams at most in one round, 127 rounds with two or more +PASS prompts a chunk at a time while the others decode: 2 rounds of the others between a prompt's chunks +PASS every prompt a chunk at a time: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS every prompt a chunk at a time: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS every prompt a chunk at a time: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS every prompt a chunk at a time: 3 streams at most in one round, 127 rounds with two or more +PASS every prompt a chunk at a time: 2 rounds of the others between a prompt's chunks +PASS sampled, prompts a chunk at a time: stream 0 (prompt 0, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled, prompts a chunk at a time: stream 1 (prompt 1, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled, prompts a chunk at a time: stream 2 (prompt 2, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled, prompts a chunk at a time: 3 streams at most in one round, 127 rounds with two or more +PASS sampled, prompts a chunk at a time: 2 rounds of the others between a prompt's chunks +PASS 1143 forwards held several streams' windows (Engine.verifyShared) diff --git a/docs/receipts/glm-kv-pages-0b094b74/single/reg/glm-lanes-cover.log b/docs/receipts/glm-kv-pages-0b094b74/single/reg/glm-lanes-cover.log new file mode 100644 index 0000000000..2b133b692a --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single/reg/glm-lanes-cover.log @@ -0,0 +1,47 @@ +info: checkpoint digest: 2.0 GiB read in 2.0 s, 0/1 cached files +PASS copy drafts, greedy: 256 ids equal, finished by length (120 rounds) +PASS copy drafts: the proposer judged 139 copied rows (135 kept) +PASS copy drafts, sampled: 256 ids equal, finished by length (254 rounds) +PASS copy drafts, sampled, thin model: the proposer judged 0 copied rows (0 kept) +PASS a cancel at ask 0 of the prompt pass ended it with error.Cancelled +PASS the cancel was heard at ask 0 (1 asks) +PASS the request after a cancel at prompt ask 0: 64 ids equal, finished by length (63 rounds) +PASS a cancel at ask 1 of the prompt pass ended it with error.Cancelled +PASS the cancel was heard at ask 1 (2 asks) +PASS the request after a cancel at prompt ask 1: 64 ids equal, finished by length (63 rounds) +PASS a cancel at ask 3 of the prompt pass ended it with error.Cancelled +PASS the cancel was heard at ask 3 (4 asks) +PASS the request after a cancel at prompt ask 3: 64 ids equal, finished by length (63 rounds) +PASS a stream was discarded mid-decode (after 1 rounds) +PASS the request after a discard after 1 rounds: 64 ids equal, finished by length (63 rounds) +PASS a stream was discarded mid-decode (after 8 rounds) +PASS the request after a discard after 8 rounds: 64 ids equal, finished by length (63 rounds) +PASS a stream was discarded mid-decode (after a rejected draft) +PASS the request after a discard after a rejected draft: 64 ids equal, finished by length (63 rounds) +PASS eos at token 0 with drafts: 1 ids equal, finished by stop (0 rounds) +PASS eos at token 0: 1 ids equal, finished by stop (0 rounds) +PASS eos at token 9 with drafts: 10 ids equal, finished by stop (9 rounds) +PASS eos at token 9: 10 ids equal, finished by stop (9 rounds) +PASS eos at token 33 with drafts: 34 ids equal, finished by stop (33 rounds) +PASS eos at token 33: 34 ids equal, finished by stop (33 rounds) +PASS eos at token 100 with drafts: 101 ids equal, finished by stop (100 rounds) +PASS eos at token 100: 101 ids equal, finished by stop (100 rounds) +PASS max_new 1 with drafts: 1 ids equal, finished by length (0 rounds) +PASS max_new 2 with drafts: 2 ids equal, finished by length (1 rounds) +PASS max_new 3 with drafts: 3 ids equal, finished by length (2 rounds) +PASS max_new 4 with drafts: 4 ids equal, finished by length (3 rounds) +PASS max_new 5 with drafts: 5 ids equal, finished by length (4 rounds) +PASS the pass kept 2 states at its marks (2 asked) +PASS the pass resumed at 2048 (2048 restored) +PASS resumed at 2048, against the fresh pass: 64 ids equal, finished by length (63 rounds) +PASS cached drafted against cached plain: 64 ids equal, finished by length (63 rounds) +PASS cached concurrent equals fresh solo at 2048 +PASS cached concurrent equals fresh solo at 2048 +PASS the pass resumed at 4096 (4096 restored) +PASS resumed at 4096, against the fresh pass: 64 ids equal, finished by length (63 rounds) +PASS cached drafted against cached plain: 64 ids equal, finished by length (63 rounds) +PASS cached concurrent equals fresh solo at 4096 +PASS cached concurrent equals fresh solo at 4096 +PASS another tail resumed at 4096 (4096 restored) +PASS another tail after the kept 4096, against its fresh pass: 64 ids equal, finished by length (63 rounds) +PASS the last request, all of W with drafts: 512 ids equal, finished by length (511 rounds) diff --git a/docs/receipts/glm-kv-pages-0b094b74/single/reg/glm-shared-check.log b/docs/receipts/glm-kv-pages-0b094b74/single/reg/glm-shared-check.log new file mode 100644 index 0000000000..0aaef6b747 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single/reg/glm-shared-check.log @@ -0,0 +1,2 @@ +info: checkpoint digest: 0.0 GiB read in 0.0 s, 1/1 cached files +PASS shared windows at TP=1: 108 rows of 5 streams over 12 rounds (2 to 5 streams a forward, 1 to 4 rows each, greedy and sampled) equal their lone twins' logits, tokens and drafts byte for byte, and every state its twin's diff --git a/docs/receipts/glm-kv-pages-0b094b74/single/reg/paged.log b/docs/receipts/glm-kv-pages-0b094b74/single/reg/paged.log new file mode 100644 index 0000000000..ce304f8ed1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single/reg/paged.log @@ -0,0 +1,9 @@ +info: checkpoint digest: 0.0 GiB read in 0.0 s, 1/1 cached files +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +PASS pages at TP=1: 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after diff --git a/docs/receipts/glm-kv-pages-0b094b74/single/reg/ref.log b/docs/receipts/glm-kv-pages-0b094b74/single/reg/ref.log new file mode 100644 index 0000000000..6b607222b1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single/reg/ref.log @@ -0,0 +1,30 @@ +RESULT b16 matmul 300 rows x [512, 4096] (K in 8 slices), bf16 out: worst column 93 at 5.502e-1 of its tolerance +RESULT b16 matmul 300 rows x [512, 4096] (K in 8 slices), fp32 out: worst column 165 at 2.086e-4 of its tolerance +PASS b16 matmul [512, 4096] (8 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal +RESULT b16 matmul 300 rows x [1280, 1536] (K in 4 slices), bf16 out: worst column 257 at 7.984e-1 of its tolerance +RESULT b16 matmul 300 rows x [1280, 1536] (K in 4 slices), fp32 out: worst column 1227 at 7.001e-4 of its tolerance +PASS b16 matmul [1280, 1536] (4 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal +RESULT b16 matmul 300 rows x [1024, 128] (K in 1 slices), bf16 out: worst column 182 at 9.820e-1 of its tolerance +RESULT b16 matmul 300 rows x [1024, 128] (K in 1 slices), fp32 out: worst column 839 at 1.471e-2 of its tolerance +PASS b16 matmul [1024, 128] (1 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal +RESULT router 300 rows x [256, 4096] (K in 8 slices): worst column 57 at 1.681e-4 of its tolerance +PASS router [256, 4096]: within float64 tolerance, router_tile and router_part bits equal at every row count +RESULT select 300 rows, top 8 of 256: weights at 1.807e-2 of 2^-16 relative, 0 near-tie swaps +PASS select top-8 of 256: picks follow the float64 ranking (near ties aside), weights within float64 tolerance +RESULT latent attention 70 rows at 1430, 32 heads, 32-head blocks: worst column 306 at 1.809e-1 of its tolerance +PASS latent attention over 1500 keys: within float64 tolerance, both head blocks and one-row windows equal +RESULT latent sparse attention 70 rows, up to 1100 of 6000 keys: worst column 349 at 4.737e-1 of its tolerance +PASS latent sparse attention over up to 1100 tokens: within float64 tolerance, one-row windows equal, count 0 left alone +RESULT GLM FP8 reconstructed attention vs FP64: worst column 5 at 5.919e-1 of its tolerance +RESULT GLM FP8 reconstructed attention vs FP64: worst column 5 at 5.919e-1 of its tolerance +RESULT GLM FP8 reconstructed attention vs FP64: worst column 171 at 2.159e-1 of its tolerance +RESULT GLM FP8 reconstructed attention vs FP64: worst column 171 at 2.159e-1 of its tolerance +PASS GLM FP8 synthetic format, masking, row/chunk identity and reconstructed BF16/FP64 attention +RESULT hc mixing 70 rows x 16 K blocks: 24 dots and the squares (column 24): worst column 24 at 2.226e-3 of its tolerance +PASS hc mixing: dots and squares within float64 tolerance, decode windows' partials and outputs equal the prompt rows' +RESULT KDA 70 rows x 32 heads, fused chain against float64 (column = head x 128 + t): worst column 1041 at 4.563e-1 of its tolerance +RESULT KDA 3 rows x 32 heads, fused chain against float64 (column = head x 128 + t): worst column 476 at 3.576e-1 of its tolerance +PASS KDA 32 heads: within float64 tolerance, fused and three-kernel chains equal at 70 and 3 rows, replays rebuild the state +RESULT index scores 96 rows at 6000, 32 heads x 128, 1536 pools: worst column 862 at 1.974e-3 of its tolerance +PASS DSA indexer 96 rows at 6000: scores within float64 tolerance, top-512 pools equal the host's rule exactly +PASS glm-ref: every projection within 9.820e-1 of its float64 tolerance diff --git a/docs/receipts/glm-kv-pages-0b094b74/single/reg/summary.log b/docs/receipts/glm-kv-pages-0b094b74/single/reg/summary.log new file mode 100644 index 0000000000..873554667c --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single/reg/summary.log @@ -0,0 +1,8 @@ +-- run 2026-10-23 01:19:40 0b094b74 glm cuda: a stream takes its pages before its sequence is bound, so a s +2026-10-23 01:20:30 ref rc=0 50s: PASS b16 matmul [512, 4096] (8 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal PASS b16 matmul [1280, 1536] (4 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal PASS b16 matmul [1024, 128] (1 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal +2026-10-23 01:20:36 synth rc=0 6s: PASS glm-synth: 330 tensors, 2095 MiB, config 2 layers 16 experts vocabulary 8192, dc.json in /work/k2/synth +2026-10-23 01:21:12 glm-lanes-cover rc=0 36s: PASS copy drafts, greedy: 256 ids equal, finished by length (120 rounds) PASS copy drafts: the proposer judged 139 copied rows (135 kept) PASS copy drafts, sampled: 256 ids equal, finished by length (254 rounds) +2026-10-23 01:21:58 glm-lanes-concurrent rc=0 46s: PASS all at once: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) PASS all at once: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) PASS all at once: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +2026-10-23 01:22:07 glm-shared-check rc=0 9s: PASS shared windows at TP=1: 108 rows of 5 streams over 12 rounds (2 to 5 streams a forward, 1 to 4 rows each, greedy and sampled) equal their lone twins' logits, tokens and drafts byte for byte, and every state its twin's +2026-10-23 01:22:17 paged rc=0 10s: PASS pages at TP=1: 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after +2026-10-23 01:22:17 === kvp-reg k2 done 0 failed diff --git a/docs/receipts/glm-kv-pages-0b094b74/single/reg/synth.log b/docs/receipts/glm-kv-pages-0b094b74/single/reg/synth.log new file mode 100644 index 0000000000..1d54c5527f --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/single/reg/synth.log @@ -0,0 +1 @@ +PASS glm-synth: 330 tensors, 2095 MiB, config 2 layers 16 experts vocabulary 8192, dc.json in /work/k2/synth diff --git a/docs/receipts/glm-kv-pages-0b094b74/tools/kvp_admit.py b/docs/receipts/glm-kv-pages-0b094b74/tools/kvp_admit.py new file mode 100644 index 0000000000..1ad3e03675 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tools/kvp_admit.py @@ -0,0 +1,105 @@ +"""Admission on pages: each prompt alone, then all at once (an optional long one first), from a cold-prompt file. + +Usage: kvp_admit.py BASE MODEL PROMPTS.json OUT.json (--length L | --fit LOG) --n N [--long LONG] [--tokens T] [--tools DIR] +PROMPTS.json is reuse_served.py cold's file. The N prompts of length L run alone, one after another, then all at once; +with --fit, L is the longest length of which one request fits the pool rank 0's LOG reports and two do not; +with --long, the first prompt of length LONG starts first and the N follow once its stream is admitted. Each reply is +T tokens (ignore_eos), seeded and sampled. The report: every request's wait (sent to first token) and end, which +requests started only after another ended (the pool held too few pages for them), and each together reply's token SHA +against its alone run. Exit 0 only when every reply equals its alone run. +""" +import argparse +import json +import re +import sys +import threading +import time + + +def fit(log: str, lengths: list, tokens: int) -> int: + """The longest of `lengths` whose request takes the pool's pages once but not twice (0 when none).""" + m = re.search(r"taking (\d+) pages of 512 rows", open(log, errors="replace").read()) + if not m: + print(f"fit: no page count in {log}", flush=True) + return 0 + pages = int(m.group(1)) + need = lambda n: -(-(n + 64 + tokens + 16) // 512) # noqa: E731 - the prompt with its template, the reply, the slack + ok = [n for n in lengths if need(n) <= pages < 2 * need(n)] + print(f"fit: {pages} pages; " + (f"{ok[-1]}-token prompts take {need(ok[-1])} each" if ok else "no length fits once and not twice"), flush=True) + return ok[-1] if ok else 0 + + +def main() -> int: + ap = argparse.ArgumentParser() + ap.add_argument("base") + ap.add_argument("model") + ap.add_argument("prompts") + ap.add_argument("out") + ap.add_argument("--length", type=int, default=0) + ap.add_argument("--fit", default="") + ap.add_argument("--n", type=int, required=True) + ap.add_argument("--long", type=int, default=0) + ap.add_argument("--tokens", type=int, default=128) + ap.add_argument("--tools", default="tools") + o = ap.parse_args() + sys.path.insert(0, o.tools) + import bench_concurrent as bench # noqa: E402 - the tree's own client + + items = json.load(open(o.prompts)) + if o.fit: + o.length = fit(o.fit, sorted({it["length"] for it in items}), o.tokens) + if not o.length: + return 2 + pick = lambda length: [it for it in items if it["length"] == length] # noqa: E731 + short = [{"name": f"p{it['length']}-{it['rep']}", "kind": "chat", "prompt": it["messages"][0]["content"]} + for it in pick(o.length)] + if len(short) < o.n: + short = [dict(s, name=f"{s['name']}-{k}") for k in range(o.n) for s in short][: o.n] + short = short[: o.n] + long = None + if o.long: + it = pick(o.long)[0] + long = {"name": f"long{o.long}", "kind": "chat", "prompt": it["messages"][0]["content"]} + alone = [] + for i, s in enumerate(short): + r = bench.stream(o.base, o.model, s, o.tokens, 1.0, 1000 + i) + alone.append(r) + print(f"alone {s['name']}: prompt {r.get('prompt_tokens')} ttft {r.get('ttft_s')} sha {r.get('token_sha256')}", flush=True) + runs: list = [None] * (len(short) + (1 if long else 0)) + threads = [] + + def go(k, item, seed): + runs[k] = bench.stream(o.base, o.model, item, o.tokens, 1.0, seed) + + t0 = time.perf_counter() + if long: + threads.append(threading.Thread(target=go, args=(len(short), long, 999))) + threads[-1].start() + time.sleep(2.0) # its stream is admitted before the others arrive + for i, s in enumerate(short): + threads.append(threading.Thread(target=go, args=(i, s, 1000 + i))) + threads[-1].start() + for t in threads: + t.join() + rows, equal = [], 0 + for k, r in enumerate(runs): + name = long["name"] if long and k == len(short) else short[k]["name"] + same = None if k == len(short) else (r.get("token_sha256") is not None and r.get("token_sha256") == alone[k].get("token_sha256")) + equal += 1 if same else 0 + rows.append({"name": name, "prompt": r.get("prompt_tokens"), "sent": r["sent"] - t0 if r.get("sent") else None, + "first": r["first"] - t0 if r.get("first") else None, "last": r["last"] - t0 if r.get("last") else None, + "error": r.get("error"), "equal_alone": same}) + ends = sorted((x["last"], x["name"]) for x in rows if x["last"] is not None) + for x in rows: + before = [n for e, n in ends if x["first"] is not None and e < x["first"] and n != x["name"]] + x["started_after"] = before + print(f"{x['name']:>12}: prompt {x['prompt']} first {x['first']:.2f} s last {x['last']:.2f} s" + f" equal_alone {x['equal_alone']} started after {before or '-'}" if x["first"] is not None else f"{x['name']}: {x['error']}", flush=True) + waited = sum(1 for x in rows if x["started_after"]) + print(f"together == alone: {equal}/{len(short)}; {waited} request(s) started only after another ended", flush=True) + json.dump({"alone": alone, "together": runs, "rows": rows}, open(o.out, "w")) + return 0 if equal == len(short) else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/docs/receipts/glm-kv-pages-0b094b74/tools/kvp_sum.py b/docs/receipts/glm-kv-pages-0b094b74/tools/kvp_sum.py new file mode 100644 index 0000000000..aa56708108 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tools/kvp_sum.py @@ -0,0 +1,87 @@ +"""KV_PAGES receipts in one line each: bench_concurrent's alone/serial counts, a record's planned count, two records' bodies, +and R3's speed (each serve's decode, 32k cold and 2k-64k cold, with the regression rule).""" +import json +import statistics +import sys + + +def conc(path): + """Cells, and the alone and serial (draft: false) equal / compared counts over every cell; false when a request + failed or a reply differed (bench_concurrent.py ends 0 even then, so a chain reads this instead).""" + rows = json.load(open(path))["cells"] + a = [sum(r.get("alone", {}).get(k, 0) for r in rows) for k in ("equal", "unequal", "failed")] + s = [sum(r.get("serial", {}).get(k, 0) for r in rows) for k in ("equal", "unequal", "failed")] + print(f"{path}: {len(rows)} cells; alone {a[0]}/{sum(a)} equal ({a[1]} unequal, {a[2]} failed); " + f"serial {s[0]}/{sum(s)} equal ({s[1]} unequal, {s[2]} failed)") + return a[1] + a[2] + s[1] + s[2] + sum(r.get("failed", 0) for r in rows) == 0 + + +def bodies(a_path, b_path): + """Whether two records sent the same bodies in the same order (so one replay serves both compares).""" + a = [json.loads(line) for line in open(a_path)] + b = [json.loads(line) for line in open(b_path)] + same = sum(1 for x, y in zip(a, b) if x["i"] == y["i"] and x["body"] == y["body"]) + shas = sum(1 for x, y in zip(a, b) if x["got"].get("sha") == y["got"].get("sha")) + cached = sum(1 for x, y in zip(a, b) if x["got"].get("cached") == y["got"].get("cached")) + print(f"{a_path} vs {b_path}: {len(a)} and {len(b)} rows; bodies equal {same}; shas equal {shas}; cached equal {cached}") + + +def measures(d, label): + """A serve's times (lower is better): greedy decode s/token per prompt, the 32k cold TTFT (median of its three + requests) and the cold TTFT per length (median of the reps); sampled decode apart, as each run decodes its own text.""" + t, own, shas = {}, {}, {} + for r in json.load(open(f"{d}/{label}-decode.json")): + key = f"decode {r['prompt']}" + if r["temperature"] == 0: + t[key] = 1 / r["decode_tps_median"] + shas[key] = {e["output_sha256"] for e in r["evidence"]} + else: + own[key] = r["decode_tps_median"] + rows = json.load(open(f"{d}/{label}-cache.json"))["rows"] + t["cold 32k chat"] = statistics.median(r["ttft_s"] for r in rows) + shas["cold 32k chat"] = {r["token_sha256"] for r in rows} + cold = {} + for r in json.load(open(f"{d}/{label}-cold.json")): + cold.setdefault(r["length"], []).append(r["ttft"]) + for n, v in sorted(cold.items()): + t[f"cold {n // 1024}k"] = statistics.median(v) + return t, own, shas + + +def speed(d, *groups): + """groups: main=L1,L2 head=L3,L4 [all=L5]. Slower is a time over 1.02 times the other's: a regression when both + head serves are slower than both main serves on one measure, or the all serve slower than both main serves. Greedy + tokens that differ between serves are a mismatch, a stop before any speed.""" + g = {k: v.split(",") for k, v in (x.split("=") for x in groups)} + m = {label: measures(d, label) for v in g.values() for label in v} + labels = [label for v in g.values() for label in v] + print("measure (s, lower is better)".ljust(30) + "".join(label.rjust(10) for label in labels)) + for key in m[labels[0]][0]: + print(key.ljust(30) + "".join(f"{m[label][0][key]:10.4f}" for label in labels)) + for key in m[labels[0]][1]: + print(f"{key} sampled tok/s (own text)".ljust(30) + "".join(f"{m[label][1][key]:10.2f}" for label in labels)) + differ = [] + for key in m[labels[0]][2]: + same = len(set().union(*(m[label][2][key] for label in labels))) == 1 + print(f"{key}: the same tokens in every serve: {same}") + if not same: + differ.append(key) + slow = [] + for key in m[labels[0]][0]: + worst_main = max(m[label][0][key] for label in g["main"]) + for k in [k for k in g if k != "main"]: + best = min(m[label][0][key] for label in g[k]) + ratio = best / worst_main + print(f"{key}: {k} best / main worst = {ratio:.4f}") + if ratio > 1.02: + slow.append(f"{key} ({k} {ratio:.4f})") + print("REGRESSION: " + "; ".join(slow) if slow else "no regression: no measure over 1.02") + if differ: + print("MISMATCH (stop: the serves' tokens differ): " + "; ".join(differ)) + + +if __name__ == "__main__": + mode, *paths = sys.argv[1:] + if mode == "conc": + sys.exit(0 if all([conc(p) for p in paths]) else 1) # every file printed + {"bodies": lambda: bodies(*paths), "speed": lambda: speed(*paths)}[mode]() diff --git a/docs/receipts/glm-kv-pages-0b094b74/tools/media_http.py b/docs/receipts/glm-kv-pages-0b094b74/tools/media_http.py new file mode 100644 index 0000000000..965824f8ce --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tools/media_http.py @@ -0,0 +1,56 @@ +"""Against `tensorfold serve --vision` of GLM-5.3-Flash on CUDA: one image request (IMAGE as a data URL, a question +about it, thinking off, greedy, ignore_eos, 128 tokens, seed 1234) through upstream's bench_concurrent stream: + - first, with drafts (a cold prompt) + - repeated with drafts: the server's cached_tokens says whether it resumed; its ids equal the first's (resumed = fresh) + - with "draft": false: its ids equal the first's (drafted = plain) + - three at once with drafts: each one's ids equal the first's (concurrent = solo) +Every row's counters and hashes to OUT; one line a request, then "exit 0|1" (1 when any ids differ or a row is +unverified). Run beside upstream's tools/bench_concurrent.py and bench_evidence.py. + + python3 media_http.py BASE MODEL IMAGE OUT +""" +import base64 +import json +import sys + +import bench_concurrent as bench +from bench_evidence import token_equal + +FIELDS = ("prompt_tokens", "cached_tokens", "tokens", "cache_state", "complete", "token_sha256", "error") + + +def main(base, model, image, out): + with open(image, "rb") as f: + url = "data:image/png;base64," + base64.b64encode(f.read()).decode() + item = {"name": "image", "kind": "chat", "prompt": [ + {"type": "image_url", "image_url": {"url": url}}, + {"type": "text", "text": "Describe this image in two sentences."}]} + rows, ok = [], True + + def keep(phase, run, ref=None): + nonlocal ok + row = {"phase": phase, **{k: run.get(k) for k in FIELDS}} + if ref is not None: + row["equal_first"] = token_equal(ref, run) + ok &= row["equal_first"] is True + rows.append(row) + print(f"{phase}: prompt {row['prompt_tokens']} cached {row['cached_tokens']} ({row['cache_state']}) tokens " + f"{row['tokens']} sha256 {str(row['token_sha256'])[:12]}" + + ("" if ref is None else f" equal_first {row['equal_first']}"), flush=True) + + first = bench.stream(base, model, item, 128, 0, 1234) + ok &= first.get("complete") is True and first.get("token_sha256") is not None + keep("first", first) + keep("repeated", bench.stream(base, model, item, 128, 0, 1234), first) + keep("draft_false", bench.stream(base, model, item, 128, 0, 1234, draft=False), first) + for run in bench.together(base, model, [(item, 1234)] * 3, 128, 0): + keep("concurrent", run, first) + with open(out, "w", encoding="utf-8") as f: + json.dump({"image_bytes": len(url), "requested_reply_tokens": 128, "temperature": 0, "seed": 1234, + "rows": rows}, f, indent=1) + print(f"exit {0 if ok else 1}") + return 0 if ok else 1 + + +if __name__ == "__main__": + sys.exit(main(*sys.argv[1:5])) diff --git a/docs/receipts/glm-kv-pages-0b094b74/tools/media_resume_http.py b/docs/receipts/glm-kv-pages-0b094b74/tools/media_resume_http.py new file mode 100644 index 0000000000..042644a14b --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tools/media_resume_http.py @@ -0,0 +1,92 @@ +"""Against `tensorfold serve --vision` of GLM-5.3-Flash on CUDA: a prompt resumed from a state after an image equals a +fresh one. media_http.py's request is one message, so nothing before it is kept and its repeat is cold; here the image +sits in the first of three messages (resume_http.py's form: the image as a data URL first, as media_http.py sends it, +then a nonce and about 5,000 tokens of notes, a fixed assistant reply, a follow-up question), thinking off, greedy, +ignore_eos, 64 tokens: + - first: cold (cached_tokens 0) + - repeated: resumed from the state the first kept (cached_tokens above 0, past the image), the same ids + - "draft": false: the same ids + - two at once: each the same ids +Every reply's counters and hashes to OUT; one line a request, then "exit 0|1". Standard library only. + + python3 media_resume_http.py PORT IMAGE OUT +""" +import base64 +import json +import sys +import threading +import time +import urllib.request + +LINE = "Line {i}: the river carries {n} stones past the mill, and the miller counts them twice before the bell. " + + +def body(nonce, url, draft=True): + notes = "".join(LINE.format(i=i, n=(i * 37) % 101) for i in range(220)) + b = {"model": "local-model", "max_tokens": 64, "temperature": 0, "ignore_eos": True, + "chat_template_kwargs": {"enable_thinking": False}, + "messages": [{"role": "user", "content": [ + {"type": "image_url", "image_url": {"url": url}}, + {"type": "text", "text": f"Reference notes {nonce}, after the picture.\n" + notes + + "\nWhat does the picture show, in one sentence?"}]}, + {"role": "assistant", "content": "The picture shows a logo on a plain background."}, + {"role": "user", "content": "Which line mentions the most stones? Answer in one sentence."}]} + if not draft: + b["draft"] = False + return b + + +def post(port, b): + req = urllib.request.Request(f"http://127.0.0.1:{port}/v1/chat/completions", data=json.dumps(b).encode(), + headers={"Content-Type": "application/json"}) + t0 = time.perf_counter() + with urllib.request.urlopen(req, timeout=900) as r: + d = json.loads(r.read()) + usage = d.get("usage") or {} + return {"seconds": round(time.perf_counter() - t0, 3), "prompt_tokens": usage.get("prompt_tokens"), + "cached_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens"), + "tokens": usage.get("completion_tokens"), + "token_sha256": (d.get("tensorfold") or {}).get("token_sha256"), + "content": d["choices"][0]["message"].get("content")} + + +def main(port, image, out): + with open(image, "rb") as f: + url = "data:image/png;base64," + base64.b64encode(f.read()).decode() + nonce = f"{time.time_ns():x}" + rows, ok = [], True + + def keep(phase, row, first=None): + nonlocal ok + row = {"phase": phase, **row} + if first is not None: + row["same_as_first"] = (row["token_sha256"] or row["content"]) == (first["token_sha256"] or first["content"]) + ok &= row["same_as_first"] + rows.append(row) + print(f"{phase}: prompt {row['prompt_tokens']} cached {row['cached_tokens']} tokens {row['tokens']} " + f"{row['seconds']} s sha256 {str(row['token_sha256'])[:12]}" + + ("" if first is None else f" same_as_first {row['same_as_first']}"), flush=True) + return row + + first = keep("first", post(port, body(nonce, url))) + ok &= first["cached_tokens"] == 0 and first["tokens"] == 64 + again = keep("repeated", post(port, body(nonce, url)), first) + ok &= (again["cached_tokens"] or 0) > 0 + keep("draft_false", post(port, body(nonce, url, draft=False)), first) + both = [None, None] + threads = [threading.Thread(target=lambda i=i: both.__setitem__(i, post(port, body(nonce, url)))) for i in range(2)] + for t in threads: + t.start() + for t in threads: + t.join() + for row in both: + keep("concurrent", row, first) + with open(out, "w", encoding="utf-8") as f: + json.dump({"nonce": nonce, "image_bytes": len(url), "requested_reply_tokens": 64, "temperature": 0, + "rows": rows}, f, indent=1) + print(f"exit {0 if ok else 1}") + return 0 if ok else 1 + + +if __name__ == "__main__": + sys.exit(main(int(sys.argv[1]), sys.argv[2], sys.argv[3])) diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/check/glm-paged-check-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/check/glm-paged-check-rank0.log new file mode 100644 index 0000000000..719788e0da --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/check/glm-paged-check-rank0.log @@ -0,0 +1,9 @@ +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +PASS pages at TP=2: 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/check/glm-paged-check-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/check/glm-paged-check-rank1.log new file mode 100644 index 0000000000..2a1f2a7088 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/check/glm-paged-check-rank1.log @@ -0,0 +1,2 @@ +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +PASS rank 1 made rank 0's pool and mapped its pages through every step diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/auto-against-all.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/auto-against-all.log new file mode 100644 index 0000000000..c589bee588 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/auto-against-all.log @@ -0,0 +1,83 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.29 s vs fresh 5.94 s ( 4.6x) + S1 t2 drafted prompt 15550 cached 6144: ttft 7.28 s vs fresh 11.91 s ( 1.6x) + S1 t2 plain prompt 15550 cached 14336: ttft 1.04 s vs fresh 11.87 s ( 11.5x) + S1 t3 drafted prompt 19460 cached 14336: ttft 4.19 s vs fresh 14.90 s ( 3.6x) + S1 t3 plain prompt 19460 cached 18432: ttft 0.95 s vs fresh 14.87 s ( 15.7x) + S1 t4 drafted prompt 21874 cached 18432: ttft 2.78 s vs fresh 16.67 s ( 6.0x) + S1 t4 plain prompt 21874 cached 20480: ttft 1.14 s vs fresh 16.63 s ( 14.6x) + S1 t5 drafted prompt 26786 cached 20480: ttft 5.17 s vs fresh 20.59 s ( 4.0x) + S1 t5 plain prompt 26786 cached 26624: ttft 0.44 s vs fresh 20.57 s ( 46.8x) + S1 t6 drafted prompt 29293 cached 26624: ttft 2.45 s vs fresh 22.44 s ( 9.1x) + S1 t6 plain prompt 29293 cached 28672: ttft 0.71 s vs fresh 22.39 s ( 31.4x) + S1 t7 drafted prompt 32216 cached 28672: ttft 2.82 s vs fresh 24.50 s ( 8.7x) + S1 t7 plain prompt 32216 cached 30720: ttft 1.19 s vs fresh 24.44 s ( 20.5x) + S1 t8 drafted prompt 35142 cached 30720: ttft 3.73 s vs fresh 26.95 s ( 7.2x) + S1 t8 plain prompt 35142 cached 34816: ttft 0.56 s vs fresh 26.93 s ( 48.2x) + S1 t9 drafted prompt 37860 cached 34816: ttft 2.56 s vs fresh 28.90 s ( 11.3x) + S1 t9 plain prompt 37860 cached 36864: ttft 0.94 s vs fresh 28.86 s ( 30.8x) + A t1 drafted prompt 7412 cached 6144: ttft 1.09 s vs fresh 5.81 s ( 5.3x) + A t1 plain prompt 7412 cached 6144: ttft 1.05 s vs fresh 5.76 s ( 5.5x) + B t1 drafted prompt 7474 cached 6144: ttft 1.12 s vs fresh 5.85 s ( 5.2x) + B t1 plain prompt 7474 cached 6144: ttft 1.08 s vs fresh 5.80 s ( 5.4x) + A t2 drafted prompt 12623 cached 6144: ttft 5.30 s vs fresh 9.88 s ( 1.9x) + A t2 plain prompt 12623 cached 12288: ttft 0.55 s vs fresh 9.86 s ( 18.0x) + B t2 drafted prompt 12720 cached 6144: ttft 5.23 s vs fresh 9.94 s ( 1.9x) + B t2 plain prompt 12720 cached 12288: ttft 0.60 s vs fresh 9.91 s ( 16.6x) + A t3 drafted prompt 18063 cached 12288: ttft 4.45 s vs fresh 13.74 s ( 3.1x) + A t3 plain prompt 18063 cached 16384: ttft 1.29 s vs fresh 13.68 s ( 10.6x) + B t3 drafted prompt 15032 cached 12288: ttft 2.32 s vs fresh 11.63 s ( 5.0x) + B t3 plain prompt 15032 cached 14336: ttft 0.74 s vs fresh 11.59 s ( 15.6x) + E t1 drafted prompt 40609 cached 6144: ttft 26.39 s vs fresh 30.89 s ( 1.2x) + E t1 plain prompt 40609 cached 38912: ttft 1.34 s vs fresh 30.82 s ( 23.0x) + T t2 plain prompt 4208 cached 4096: ttft 0.37 s vs fresh 3.57 s ( 9.6x) + C t1 plain prompt 9702 cached 8192: ttft 1.19 s vs fresh 7.42 s ( 6.2x) + C t2 drafted prompt 15152 cached 8192: ttft 5.48 s vs fresh 11.72 s ( 2.1x) + C t2 plain prompt 15152 cached 14336: ttft 0.82 s vs fresh 11.69 s ( 14.2x) + A t4 drafted prompt 20872 cached 16384: ttft 6.66 s vs fresh 30.56 s ( 4.6x) + B t4 drafted prompt 17550 cached 14336: ttft 2.61 s vs fresh 13.46 s ( 5.2x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/auto-against-on.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/auto-against-on.log new file mode 100644 index 0000000000..7eed244fe3 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/auto-against-on.log @@ -0,0 +1,83 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.24 s vs fresh 5.94 s ( 4.8x) + S1 t2 drafted prompt 15550 cached 6144: ttft 7.19 s vs fresh 11.91 s ( 1.7x) + S1 t2 plain prompt 15550 cached 14336: ttft 1.02 s vs fresh 11.87 s ( 11.6x) + S1 t3 drafted prompt 19460 cached 14336: ttft 4.04 s vs fresh 14.90 s ( 3.7x) + S1 t3 plain prompt 19460 cached 18432: ttft 0.91 s vs fresh 14.87 s ( 16.3x) + S1 t4 drafted prompt 21874 cached 18432: ttft 2.72 s vs fresh 16.67 s ( 6.1x) + S1 t4 plain prompt 21874 cached 20480: ttft 1.12 s vs fresh 16.63 s ( 14.9x) + S1 t5 drafted prompt 26786 cached 20480: ttft 5.09 s vs fresh 20.59 s ( 4.0x) + S1 t5 plain prompt 26786 cached 26624: ttft 0.42 s vs fresh 20.57 s ( 49.1x) + S1 t6 drafted prompt 29293 cached 26624: ttft 2.30 s vs fresh 22.44 s ( 9.8x) + S1 t6 plain prompt 29293 cached 28672: ttft 0.70 s vs fresh 22.39 s ( 31.9x) + S1 t7 drafted prompt 32216 cached 28672: ttft 2.81 s vs fresh 24.50 s ( 8.7x) + S1 t7 plain prompt 32216 cached 30720: ttft 1.19 s vs fresh 24.44 s ( 20.5x) + S1 t8 drafted prompt 35142 cached 30720: ttft 3.70 s vs fresh 26.95 s ( 7.3x) + S1 t8 plain prompt 35142 cached 34816: ttft 0.55 s vs fresh 26.93 s ( 48.9x) + S1 t9 drafted prompt 37860 cached 34816: ttft 2.54 s vs fresh 28.90 s ( 11.4x) + S1 t9 plain prompt 37860 cached 36864: ttft 0.93 s vs fresh 28.86 s ( 31.2x) + A t1 drafted prompt 7412 cached 6144: ttft 1.09 s vs fresh 5.81 s ( 5.3x) + A t1 plain prompt 7412 cached 6144: ttft 1.05 s vs fresh 5.76 s ( 5.5x) + B t1 drafted prompt 7474 cached 6144: ttft 1.13 s vs fresh 5.85 s ( 5.2x) + B t1 plain prompt 7474 cached 6144: ttft 1.08 s vs fresh 5.80 s ( 5.4x) + A t2 drafted prompt 12623 cached 6144: ttft 5.16 s vs fresh 9.88 s ( 1.9x) + A t2 plain prompt 12623 cached 12288: ttft 0.55 s vs fresh 9.86 s ( 18.0x) + B t2 drafted prompt 12720 cached 6144: ttft 5.22 s vs fresh 9.94 s ( 1.9x) + B t2 plain prompt 12720 cached 12288: ttft 0.59 s vs fresh 9.91 s ( 16.7x) + A t3 drafted prompt 18063 cached 12288: ttft 4.43 s vs fresh 13.74 s ( 3.1x) + A t3 plain prompt 18063 cached 16384: ttft 1.28 s vs fresh 13.68 s ( 10.7x) + B t3 drafted prompt 15032 cached 12288: ttft 2.31 s vs fresh 11.63 s ( 5.0x) + B t3 plain prompt 15032 cached 14336: ttft 0.74 s vs fresh 11.59 s ( 15.8x) + E t1 drafted prompt 40609 cached 6144: ttft 26.11 s vs fresh 30.89 s ( 1.2x) + E t1 plain prompt 40609 cached 38912: ttft 1.32 s vs fresh 30.82 s ( 23.3x) + T t2 plain prompt 4208 cached 4096: ttft 0.37 s vs fresh 3.57 s ( 9.7x) + C t1 plain prompt 9702 cached 8192: ttft 1.18 s vs fresh 7.42 s ( 6.3x) + C t2 drafted prompt 15152 cached 8192: ttft 5.46 s vs fresh 11.72 s ( 2.1x) + C t2 plain prompt 15152 cached 14336: ttft 0.82 s vs fresh 11.69 s ( 14.3x) + A t4 drafted prompt 20872 cached 16384: ttft 6.54 s vs fresh 30.56 s ( 4.7x) + B t4 drafted prompt 17550 cached 14336: ttft 2.60 s vs fresh 13.46 s ( 5.2x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/main-against-all.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/main-against-all.log new file mode 100644 index 0000000000..3ff444128f --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/main-against-all.log @@ -0,0 +1,83 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.29 s vs fresh 5.95 s ( 4.6x) + S1 t2 drafted prompt 15550 cached 6144: ttft 7.28 s vs fresh 11.92 s ( 1.6x) + S1 t2 plain prompt 15550 cached 14336: ttft 1.04 s vs fresh 11.90 s ( 11.5x) + S1 t3 drafted prompt 19460 cached 14336: ttft 4.19 s vs fresh 14.92 s ( 3.6x) + S1 t3 plain prompt 19460 cached 18432: ttft 0.95 s vs fresh 14.92 s ( 15.8x) + S1 t4 drafted prompt 21874 cached 18432: ttft 2.78 s vs fresh 16.72 s ( 6.0x) + S1 t4 plain prompt 21874 cached 20480: ttft 1.14 s vs fresh 16.67 s ( 14.7x) + S1 t5 drafted prompt 26786 cached 20480: ttft 5.17 s vs fresh 20.65 s ( 4.0x) + S1 t5 plain prompt 26786 cached 26624: ttft 0.44 s vs fresh 20.62 s ( 46.9x) + S1 t6 drafted prompt 29293 cached 26624: ttft 2.45 s vs fresh 22.50 s ( 9.2x) + S1 t6 plain prompt 29293 cached 28672: ttft 0.71 s vs fresh 22.48 s ( 31.5x) + S1 t7 drafted prompt 32216 cached 28672: ttft 2.82 s vs fresh 24.98 s ( 8.9x) + S1 t7 plain prompt 32216 cached 30720: ttft 1.19 s vs fresh 24.86 s ( 20.8x) + S1 t8 drafted prompt 35142 cached 30720: ttft 3.73 s vs fresh 27.22 s ( 7.3x) + S1 t8 plain prompt 35142 cached 34816: ttft 0.56 s vs fresh 26.99 s ( 48.3x) + S1 t9 drafted prompt 37860 cached 34816: ttft 2.56 s vs fresh 29.00 s ( 11.3x) + S1 t9 plain prompt 37860 cached 36864: ttft 0.94 s vs fresh 28.97 s ( 30.9x) + A t1 drafted prompt 7412 cached 6144: ttft 1.09 s vs fresh 5.83 s ( 5.3x) + A t1 plain prompt 7412 cached 6144: ttft 1.05 s vs fresh 5.79 s ( 5.5x) + B t1 drafted prompt 7474 cached 6144: ttft 1.12 s vs fresh 5.86 s ( 5.2x) + B t1 plain prompt 7474 cached 6144: ttft 1.08 s vs fresh 5.81 s ( 5.4x) + A t2 drafted prompt 12623 cached 6144: ttft 5.30 s vs fresh 9.93 s ( 1.9x) + A t2 plain prompt 12623 cached 12288: ttft 0.55 s vs fresh 9.90 s ( 18.0x) + B t2 drafted prompt 12720 cached 6144: ttft 5.23 s vs fresh 9.97 s ( 1.9x) + B t2 plain prompt 12720 cached 12288: ttft 0.60 s vs fresh 9.93 s ( 16.6x) + A t3 drafted prompt 18063 cached 12288: ttft 4.45 s vs fresh 13.80 s ( 3.1x) + A t3 plain prompt 18063 cached 16384: ttft 1.29 s vs fresh 13.74 s ( 10.6x) + B t3 drafted prompt 15032 cached 12288: ttft 2.32 s vs fresh 11.66 s ( 5.0x) + B t3 plain prompt 15032 cached 14336: ttft 0.74 s vs fresh 11.62 s ( 15.7x) + E t1 drafted prompt 40609 cached 6144: ttft 26.39 s vs fresh 31.01 s ( 1.2x) + E t1 plain prompt 40609 cached 38912: ttft 1.34 s vs fresh 30.94 s ( 23.0x) + T t2 plain prompt 4208 cached 4096: ttft 0.37 s vs fresh 3.58 s ( 9.6x) + C t1 plain prompt 9702 cached 8192: ttft 1.19 s vs fresh 7.43 s ( 6.2x) + C t2 drafted prompt 15152 cached 8192: ttft 5.48 s vs fresh 11.76 s ( 2.1x) + C t2 plain prompt 15152 cached 14336: ttft 0.82 s vs fresh 11.72 s ( 14.3x) + A t4 drafted prompt 20872 cached 16384: ttft 6.66 s vs fresh 30.67 s ( 4.6x) + B t4 drafted prompt 17550 cached 14336: ttft 2.61 s vs fresh 13.50 s ( 5.2x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/main-against-on.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/main-against-on.log new file mode 100644 index 0000000000..6a255231f5 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/compare/main-against-on.log @@ -0,0 +1,83 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.24 s vs fresh 5.95 s ( 4.8x) + S1 t2 drafted prompt 15550 cached 6144: ttft 7.19 s vs fresh 11.92 s ( 1.7x) + S1 t2 plain prompt 15550 cached 14336: ttft 1.02 s vs fresh 11.90 s ( 11.6x) + S1 t3 drafted prompt 19460 cached 14336: ttft 4.04 s vs fresh 14.92 s ( 3.7x) + S1 t3 plain prompt 19460 cached 18432: ttft 0.91 s vs fresh 14.92 s ( 16.3x) + S1 t4 drafted prompt 21874 cached 18432: ttft 2.72 s vs fresh 16.72 s ( 6.1x) + S1 t4 plain prompt 21874 cached 20480: ttft 1.12 s vs fresh 16.67 s ( 14.9x) + S1 t5 drafted prompt 26786 cached 20480: ttft 5.09 s vs fresh 20.65 s ( 4.1x) + S1 t5 plain prompt 26786 cached 26624: ttft 0.42 s vs fresh 20.62 s ( 49.2x) + S1 t6 drafted prompt 29293 cached 26624: ttft 2.30 s vs fresh 22.50 s ( 9.8x) + S1 t6 plain prompt 29293 cached 28672: ttft 0.70 s vs fresh 22.48 s ( 32.0x) + S1 t7 drafted prompt 32216 cached 28672: ttft 2.81 s vs fresh 24.98 s ( 8.9x) + S1 t7 plain prompt 32216 cached 30720: ttft 1.19 s vs fresh 24.86 s ( 20.9x) + S1 t8 drafted prompt 35142 cached 30720: ttft 3.70 s vs fresh 27.22 s ( 7.4x) + S1 t8 plain prompt 35142 cached 34816: ttft 0.55 s vs fresh 26.99 s ( 49.0x) + S1 t9 drafted prompt 37860 cached 34816: ttft 2.54 s vs fresh 29.00 s ( 11.4x) + S1 t9 plain prompt 37860 cached 36864: ttft 0.93 s vs fresh 28.97 s ( 31.3x) + A t1 drafted prompt 7412 cached 6144: ttft 1.09 s vs fresh 5.83 s ( 5.3x) + A t1 plain prompt 7412 cached 6144: ttft 1.05 s vs fresh 5.79 s ( 5.5x) + B t1 drafted prompt 7474 cached 6144: ttft 1.13 s vs fresh 5.86 s ( 5.2x) + B t1 plain prompt 7474 cached 6144: ttft 1.08 s vs fresh 5.81 s ( 5.4x) + A t2 drafted prompt 12623 cached 6144: ttft 5.16 s vs fresh 9.93 s ( 1.9x) + A t2 plain prompt 12623 cached 12288: ttft 0.55 s vs fresh 9.90 s ( 18.1x) + B t2 drafted prompt 12720 cached 6144: ttft 5.22 s vs fresh 9.97 s ( 1.9x) + B t2 plain prompt 12720 cached 12288: ttft 0.59 s vs fresh 9.93 s ( 16.7x) + A t3 drafted prompt 18063 cached 12288: ttft 4.43 s vs fresh 13.80 s ( 3.1x) + A t3 plain prompt 18063 cached 16384: ttft 1.28 s vs fresh 13.74 s ( 10.7x) + B t3 drafted prompt 15032 cached 12288: ttft 2.31 s vs fresh 11.66 s ( 5.0x) + B t3 plain prompt 15032 cached 14336: ttft 0.74 s vs fresh 11.62 s ( 15.8x) + E t1 drafted prompt 40609 cached 6144: ttft 26.11 s vs fresh 31.01 s ( 1.2x) + E t1 plain prompt 40609 cached 38912: ttft 1.32 s vs fresh 30.94 s ( 23.4x) + T t2 plain prompt 4208 cached 4096: ttft 0.37 s vs fresh 3.58 s ( 9.7x) + C t1 plain prompt 9702 cached 8192: ttft 1.18 s vs fresh 7.43 s ( 6.3x) + C t2 drafted prompt 15152 cached 8192: ttft 5.46 s vs fresh 11.76 s ( 2.2x) + C t2 plain prompt 15152 cached 14336: ttft 0.82 s vs fresh 11.72 s ( 14.3x) + A t4 drafted prompt 20872 cached 16384: ttft 6.54 s vs fresh 30.67 s ( 4.7x) + B t4 drafted prompt 17550 cached 14336: ttft 2.60 s vs fresh 13.50 s ( 5.2x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/client.log new file mode 100644 index 0000000000..296fda32cb --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/client.log @@ -0,0 +1,61 @@ +server up 2026-10-23 00:00:28 +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 39.1, "per_stream_tps": 39.1, "ttft_s_max": 0.15, "timing_valid": true, "end_to_end_tps": 38.38897479203177, "makespan_s": 6.668581314996118, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 44.8, "per_stream_tps": 24.7, "ttft_s_max": 0.37, "timing_valid": true, "end_to_end_tps": 44.35212894920937, "makespan_s": 11.543977980996715, "steady_tps": 48.1, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 50.2, "per_stream_tps": 20.9, "ttft_s_max": 12.46, "timing_valid": true, "end_to_end_tps": 50.02088333776209, "makespan_s": 20.471449756005313, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 29.2, "per_stream_tps": 29.2, "ttft_s_max": 0.22, "timing_valid": true, "end_to_end_tps": 28.62200487756052, "makespan_s": 8.944167296984233, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 39.9, "per_stream_tps": 20.4, "ttft_s_max": 0.51, "timing_valid": true, "end_to_end_tps": 39.424584383089815, "makespan_s": 12.986820482998155, "steady_tps": 40.8, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 39.5, "per_stream_tps": 16.2, "ttft_s_max": 16.12, "timing_valid": true, "end_to_end_tps": 39.33300974210784, "makespan_s": 26.034112485009246, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 56.1, "per_stream_tps": 56.1, "ttft_s_max": 0.15, "timing_valid": true, "end_to_end_tps": 54.495886321860844, "makespan_s": 4.6976022830058355, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 62.8, "per_stream_tps": 32.0, "ttft_s_max": 0.37, "timing_valid": true, "end_to_end_tps": 61.966239094653794, "makespan_s": 8.262563736003358, "steady_tps": 64.2, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 69.9, "per_stream_tps": 26.1, "ttft_s_max": 10.08, "timing_valid": true, "end_to_end_tps": 69.46815527665999, "makespan_s": 14.7405670399894, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 30.6, "per_stream_tps": 30.6, "ttft_s_max": 0.22, "timing_valid": true, "end_to_end_tps": 29.907326707600312, "makespan_s": 8.559775419009384, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 43.5, "per_stream_tps": 22.1, "ttft_s_max": 0.51, "timing_valid": true, "end_to_end_tps": 42.84285383649368, "makespan_s": 11.95065113902092, "steady_tps": 44.4, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 44.0, "per_stream_tps": 17.6, "ttft_s_max": 14.91, "timing_valid": true, "end_to_end_tps": 43.80047501346692, "makespan_s": 23.37874189001741, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +logs/r2all-conc.json: 12 cells; alone 28/28 equal (0 unequal, 0 failed); serial 30/30 equal (0 unequal, 0 failed) + 0 S1 t1 drafted prompt 7759 history 7757 cached 0 (allowed [0]) ttft 6.16s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok + 1 S1 t1 plain prompt 7759 history 7757 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 1.29s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok +system cuts [4563, 6099, 6611] + 2 S1 t2 drafted prompt 15550 history 15548 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 7.28s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 tool_calls ok + 3 S1 t2 plain prompt 15550 history 15548 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 1.04s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 tool_calls ok + 4 S1 t3 drafted prompt 19460 history 19458 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 4.19s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 tool_calls ok + 5 S1 t3 plain prompt 19460 history 19458 cached 18432 (allowed {'min': 11266, 'max': 19458}) ttft 0.95s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 tool_calls ok + 6 S1 t4 drafted prompt 21874 history 21872 cached 18432 (allowed {'min': 11266, 'max': 19458}) ttft 2.78s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 tool_calls ok + 7 S1 t4 plain prompt 21874 history 21872 cached 20480 (allowed {'min': 13680, 'max': 21872}) ttft 1.14s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 tool_calls ok + 8 S1 t5 drafted prompt 26786 history 26784 cached 20480 (allowed {'min': 13680, 'max': 21872}) ttft 5.17s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c length ok + 9 S1 t5 plain prompt 26786 history 26784 cached 26624 (allowed {'min': 18592, 'max': 26784}) ttft 0.44s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c length ok + 10 S1 t6 drafted prompt 29293 history 29291 cached 26624 (allowed {'min': 18592, 'max': 26784}) ttft 2.45s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 length ok + 11 S1 t6 plain prompt 29293 history 29291 cached 28672 (allowed {'min': 21099, 'max': 29291}) ttft 0.71s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 length ok + 12 S1 t7 drafted prompt 32216 history 32214 cached 28672 (allowed {'min': 21099, 'max': 29291}) ttft 2.82s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e length ok + 13 S1 t7 plain prompt 32216 history 32214 cached 30720 (allowed {'min': 24022, 'max': 32214}) ttft 1.19s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e length ok + 14 S1 t8 drafted prompt 35142 history 35140 cached 30720 (allowed {'min': 24022, 'max': 32214}) ttft 3.73s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 length ok + 15 S1 t8 plain prompt 35142 history 35140 cached 34816 (allowed {'min': 26948, 'max': 35140}) ttft 0.56s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 length ok + 16 S1 t9 drafted prompt 37860 history 37858 cached 34816 (allowed {'min': 26948, 'max': 35140}) ttft 2.56s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 17 S1 t9 plain prompt 37860 history 37858 cached 36864 (allowed {'min': 29666, 'max': 37858}) ttft 0.94s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 18 X t1 drafted prompt 2115 history 2113 cached 0 (allowed [0]) ttft 1.77s sha 9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd length ok + 19 A t1 drafted prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 1.09s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 tool_calls ok + 20 A t1 plain prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 1.05s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 tool_calls ok + 21 B t1 drafted prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 1.12s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a tool_calls ok + 22 B t1 plain prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 1.08s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a tool_calls ok + 23 A t2 drafted prompt 12623 history 12621 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 5.30s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 tool_calls ok + 24 A t2 plain prompt 12623 history 12621 cached 12288 (allowed {'min': 4429, 'max': 12621}) ttft 0.55s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 tool_calls ok + 25 B t2 drafted prompt 12720 history 12718 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 5.23s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c tool_calls ok + 26 B t2 plain prompt 12720 history 12718 cached 12288 (allowed {'min': 4526, 'max': 12718}) ttft 0.60s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c tool_calls ok + 27 A t3 drafted prompt 18063 history 18061 cached 12288 (allowed {'min': 4429, 'max': 12621}) ttft 4.45s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc length ok + 28 A t3 plain prompt 18063 history 18061 cached 16384 (allowed {'min': 9869, 'max': 18061}) ttft 1.29s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc length ok + 29 B t3 drafted prompt 15032 history 15030 cached 12288 (allowed {'min': 4526, 'max': 12718}) ttft 2.32s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa tool_calls ok + 30 B t3 plain prompt 15032 history 15030 cached 14336 (allowed {'min': 6838, 'max': 15030}) ttft 0.74s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa tool_calls ok + 31 E t1 drafted prompt 40609 history 40607 cached 6144 (allowed {'min': 0, 'max': 6611}) ttft 26.39s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 32 E t1 plain prompt 40609 history 40607 cached 38912 (allowed {'min': 32415, 'max': 40607}) ttft 1.34s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 33 T t1 drafted prompt 1520 history 1517 cached 0 (allowed [0]) ttft 1.14s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 stop ok + 34 T t1 plain prompt 1520 history 1517 cached 0 (allowed [0]) ttft 1.10s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 stop ok + 35 T t2 drafted prompt 4208 history 4205 cached 0 (allowed [0]) ttft 3.59s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 36 T t2 plain prompt 4208 history 4205 cached 4096 (allowed {'min': 1, 'max': 4205}) ttft 0.37s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 37 C t1 cancel prompt None history 0 cached None (allowed None) ttft 0.00s sha None None ok + 38 C t1 drafted prompt 9702 history 9700 cached 0 (allowed {'min': 0, 'max': 9688}) ttft 8.20s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 tool_calls ok + 39 C t1 plain prompt 9702 history 9700 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 1.19s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 tool_calls ok + 40 C t2 drafted prompt 15152 history 15150 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 5.48s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 length ok + 41 C t2 plain prompt 15152 history 15150 cached 14336 (allowed {'min': 6958, 'max': 15150}) ttft 0.82s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 length ok + 42 A t4 drafted prompt 20872 history 20870 cached 16384 (allowed {'min': 9869, 'max': 18061}) ttft 6.66s sha eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c length ok + 43 B t4 drafted prompt 17550 history 17548 cached 14336 (allowed {'min': 6838, 'max': 15030}) ttft 2.61s sha fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8 length ok +record: 44 requests, 0 not cached as planned +=== kvp-client r2all rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/conc.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/conc.json new file mode 100644 index 0000000000..9a2639dc3d --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/conc.json @@ -0,0 +1,4606 @@ +{ + "label": "", + "tokens": 256, + "mixed": false, + "cells": [ + { + "prompt": "code", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.1, + "per_stream_tps": 39.1, + "ttft_s_max": 0.15, + "timing_valid": true, + "end_to_end_tps": 38.38897479203177, + "makespan_s": 6.668581314996118, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.105028419582375, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 6.668581314996118, + "end_to_end_tps": 38.38897479203177, + "timing_valid": true, + "per_stream_tps": [ + 39.1 + ], + "ttft_s": [ + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + } + ], + "client_intervals": [ + { + "start": 132936.085449597, + "first": 132936.232959124, + "last": 132942.75385964, + "end": 132942.754030912, + "elapsed_s": 6.668581314996118, + "chunk_offsets_s": [ + 0.1475095269852318, + 0.221211615978973, + 0.29308419197332114, + 0.3686169619904831, + 0.4521313719742466, + 0.5379113479866646, + 0.617477516992949, + 0.7021766399848275, + 0.7842509589972906, + 0.8668578469951171, + 0.9488171830016654, + 1.0330567109922413, + 1.1131401149905287, + 1.1949806139746215, + 1.2683842349797487, + 1.3434369140013587, + 1.419883835973451, + 1.4986954990017693, + 1.5851420139952097, + 1.6694069509976543, + 1.7561989369860385, + 1.8382783540000673, + 1.9116700989834499, + 1.9840167840011418, + 2.0567647849966306, + 2.1339439949952066, + 2.214204442978371, + 2.3025397649907973, + 2.3877087089931592, + 2.472684970998671, + 2.55565467197448, + 2.640039336984046, + 2.723861478996696, + 2.808235065982444, + 2.8907092489826027, + 2.9754338459752034, + 3.061130313988542, + 3.1488143029855564, + 3.2329817839781754, + 3.3186882689769845, + 3.408297951973509, + 3.4980283769837115, + 3.584471872978611, + 3.66835762897972, + 3.7514524639991578, + 3.8342272639856674, + 3.918759221996879, + 4.00073806400178, + 4.082881392998388, + 4.16465208097361, + 4.246871637995355, + 4.329058457980864, + 4.4154573869891465, + 4.495786355983, + 4.567780022975057, + 4.641677654988598, + 4.717811571987113, + 4.803875279001659, + 4.893274235975696, + 4.97739700099919, + 5.060532581992447, + 5.16070139099611, + 5.270031749998452, + 5.356634266994661, + 5.440244478988461, + 5.523453150992282, + 5.611794953001663, + 5.700813891977305, + 5.788030083000194, + 5.876108874974307, + 5.9556716039951425, + 6.037754232995212, + 6.12574873198173, + 6.208019144978607, + 6.293148033990292, + 6.374058012996102, + 6.460315115982667, + 6.535491088987328, + 6.606999826995889, + 6.668410042999312 + ], + "chunk_gaps_ms": { + "count": 79, + "p50": 83.13558099325746, + "p95": 89.6096829965245, + "max": 109.33035900234245 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 44.8, + "per_stream_tps": 24.7, + "ttft_s_max": 0.37, + "timing_valid": true, + "end_to_end_tps": 44.35212894920937, + "makespan_s": 11.543977980996715, + "steady_tps": 48.1, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 44.751577684969504, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 11.543977980996715, + "end_to_end_tps": 44.35212894920937, + "timing_valid": true, + "steady_tps": 48.09061481283255, + "per_stream_tps": [ + 27.0, + 22.4 + ], + "ttft_s": [ + 0.37, + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + } + ], + "client_intervals": [ + { + "start": 132942.755819031, + "first": 132943.12097957, + "last": 132952.58027032, + "end": 132952.580381526, + "elapsed_s": 9.824562494992279, + "chunk_offsets_s": [ + 0.3651605389895849, + 0.46711716600111686, + 0.5817122150037903, + 0.6997826650040224, + 0.8188350980053656, + 0.9351214289781637, + 1.046592614002293, + 1.1668279449804686, + 1.2834311320038978, + 1.4007106389908586, + 1.5204743149806745, + 1.6405726639786735, + 1.759844123997027, + 1.8789382480026688, + 1.996263420995092, + 2.115019652992487, + 2.236044118006248, + 2.3567880259943195, + 2.477532461984083, + 2.5954348410014063, + 2.717503724998096, + 2.8370246299891733, + 2.9578651339979842, + 3.076438647985924, + 3.1989506339887157, + 3.326470750005683, + 3.449771461979253, + 3.5689639909833204, + 3.6899182380002458, + 3.811502850992838, + 3.9342938420013525, + 4.055103841994423, + 4.17179721899447, + 4.289462819986511, + 4.409249497985002, + 4.531917867978336, + 4.64900864099036, + 4.769382707978366, + 4.897439017979195, + 5.021856815990759, + 5.140614623000147, + 5.26257992797764, + 5.384027591004269, + 5.506171207991429, + 5.629196514986688, + 5.749714342004154, + 5.872622171998955, + 5.996909123990918, + 6.112889351992635, + 6.236530697002308, + 6.391461516992422, + 6.540793235995807, + 6.692269648978254, + 6.838899349997519, + 6.9637887550052255, + 7.086818420997588, + 7.2110476910020225, + 7.330221341981087, + 7.449799347989028, + 7.56490885198582, + 7.677379732980626, + 7.796227607992478, + 7.918845567997778, + 8.04197036899859, + 8.163180990988621, + 8.287577062001219, + 8.412709934986196, + 8.5318653860013, + 8.657983362994855, + 8.77606629798538, + 8.898371771996608, + 9.019842741981847, + 9.139306394004961, + 9.258466222003335, + 9.376749271003064, + 9.500319965998642, + 9.615995370986639, + 9.71996984598809, + 9.824451288994169 + ], + "chunk_gaps_ms": { + "count": 78, + "p50": 120.44594700273592, + "p95": 146.62970101926476, + "max": 154.93081999011338 + } + }, + { + "start": 132942.755533385, + "first": 132942.903175545, + "last": 132954.299421809, + "end": 132954.299511366, + "elapsed_s": 11.543977980996715, + "chunk_offsets_s": [ + 0.1476421600091271, + 0.21980239800177515, + 0.4670245859888382, + 0.5817351760051679, + 0.6997570160019677, + 0.8187275569944177, + 0.9350688660051674, + 1.0466012859833427, + 1.1668698829889763, + 1.283333222992951, + 1.4005626639991533, + 1.5205815369845368, + 1.6405565190070774, + 1.7598198190098628, + 1.8788824529910926, + 1.9962336919852532, + 2.115396214998327, + 2.2360313659883104, + 2.3566448040073738, + 2.4775719360040966, + 2.595405063999351, + 2.717378358996939, + 2.837020773993572, + 2.9577304880076554, + 3.0764183109859005, + 3.1989802829921246, + 3.326258148008492, + 3.4496079020027537, + 3.568786927004112, + 3.6899373430060223, + 3.8113749409967568, + 3.9343154759844765, + 4.055056607990991, + 4.171842021984048, + 4.289348333986709, + 4.409284572000615, + 4.531646702991566, + 4.648822663992178, + 4.769767543009948, + 4.8978779669851065, + 5.021816510008648, + 5.140647984982934, + 5.262592184997629, + 5.383971108996775, + 5.506156839983305, + 5.629142672987655, + 5.749725895002484, + 5.872515430994099, + 5.996862050000345, + 6.1127194720029365, + 6.2364835269981995, + 6.391296149988193, + 6.540613866993226, + 6.692105402005836, + 6.838808545988286, + 6.963850901986007, + 7.086552200984443, + 7.21084033799707, + 7.3300218129879795, + 7.449805396987358, + 7.564857969991863, + 7.677302241005236, + 7.796090529009234, + 7.918844047992025, + 8.04196907300502, + 8.16337200900307, + 8.287569796986645, + 8.41266150100273, + 8.531812152010389, + 8.65791009599343, + 8.775979686004575, + 8.898286599986022, + 9.019831957994029, + 9.139282088988693, + 9.258535889006453, + 9.37673666299088, + 9.500109988002805, + 9.616014188010013, + 9.719963381998241, + 9.824356356984936, + 9.903240140993148, + 9.968178887007525, + 10.030304266983876, + 10.094169183983468, + 10.158293551998213, + 10.2232668880024, + 10.29645559398341, + 10.37054704700131, + 10.442692710988922, + 10.507802981999703, + 10.574953286995878, + 10.648500217997935, + 10.722989611007506, + 10.797374421992572, + 10.86892925799475, + 10.93375841100351, + 11.000270508986432, + 11.074574002996087, + 11.151210373995127, + 11.228682463988662, + 11.30933149100747, + 11.393943522998597, + 11.47324406899861, + 11.543888423999306 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 119.17902500135824, + "p95": 128.11042397515848, + "max": 247.22218798706308 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.2, + "per_stream_tps": 20.9, + "ttft_s_max": 12.46, + "timing_valid": true, + "end_to_end_tps": 50.02088333776209, + "makespan_s": 20.471449756005313, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.185083132744964, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 20.471449756005313, + "end_to_end_tps": 50.02088333776209, + "timing_valid": true, + "per_stream_tps": [ + 21.3, + 17.7, + 31.8, + 20.5 + ], + "ttft_s": [ + 0.36, + 0.61, + 12.46, + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6c97574b6d3c", + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692" + } + ], + "client_intervals": [ + { + "start": 132954.303195478, + "first": 132954.668141146, + "last": 132966.624231712, + "end": 132966.624439385, + "elapsed_s": 12.321243907004828, + "chunk_offsets_s": [ + 0.36494566799956374, + 0.46921784000005573, + 0.7448556840245146, + 0.8912343420088291, + 1.0410760620143265, + 1.1886181520239916, + 1.3274452650221065, + 1.4783883970230818, + 1.6240195560094435, + 1.7731040780199692, + 1.9196701600158121, + 2.0713405340211466, + 2.2233222820213996, + 2.3765900600119494, + 2.5279841950105038, + 2.6748026960121933, + 2.826425762003055, + 2.977685707010096, + 3.1289629329985473, + 3.2839208460063674, + 3.435678974026814, + 3.588233679998666, + 3.7421933120058384, + 3.892457968002418, + 4.048802978009917, + 4.21163749601692, + 4.366981034021592, + 4.525996329000918, + 4.6830224500154145, + 4.8342978810251225, + 4.985772345011355, + 5.134975278022466, + 5.288531921018148, + 5.443390051019378, + 5.598926270002266, + 5.752148777013645, + 5.905396292015212, + 6.062662926007761, + 6.21756628100411, + 6.3754622410051525, + 6.529021032998571, + 6.683321982010966, + 6.8330298800137825, + 6.985828069009585, + 7.140824874019017, + 7.294029402022716, + 7.450611050007865, + 7.601180536003085, + 7.754895159014268, + 7.908294743014267, + 8.05678225401789, + 8.208702792006079, + 8.362565022020135, + 8.514455222000834, + 8.67055191702093, + 8.828222270007245, + 8.984322420001263, + 9.13331457102322, + 9.283445623004809, + 9.435018215008313, + 9.585616933007259, + 9.73629707400687, + 9.894839938002406, + 10.046067337010754, + 10.198912397027016, + 10.357118461019127, + 10.512957854021806, + 10.666549579007551, + 10.821870714018587, + 10.9690701860236, + 11.122221779020038, + 11.277193505025934, + 11.43235997701413, + 11.586480826023035, + 11.738909752020845, + 11.896956526004942, + 12.046760624012677, + 12.184757093025837, + 12.321036234003259 + ], + "chunk_gaps_ms": { + "count": 78, + "p50": 152.8216245060321, + "p95": 158.5428639955353, + "max": 275.6378440244589 + } + }, + { + "start": 132954.303704062, + "first": 132954.915743081, + "last": 132969.30525801, + "end": 132969.30533859, + "elapsed_s": 15.001634527987335, + "chunk_offsets_s": [ + 0.612039018975338, + 0.7439839949947782, + 0.8904767709900625, + 1.040197491995059, + 1.1879129659791943, + 1.32669169298606, + 1.4775746459781658, + 1.6233875099860597, + 1.7724048770032823, + 1.9189545259869192, + 2.0707517379778437, + 2.222515155997826, + 2.376008576975437, + 2.5273378449783195, + 2.6740817499812692, + 2.825587546976749, + 2.977063645987073, + 3.1282600999984425, + 3.283208491979167, + 3.4354191219899803, + 3.5876168189861346, + 3.741546416975325, + 3.8917229090002365, + 4.0481424019963015, + 4.211279830982676, + 4.366411758994218, + 4.5254263499809895, + 4.6824221489951015, + 4.833624377002707, + 4.985198509995826, + 5.134372832981171, + 5.287898594979197, + 5.442623965995153, + 5.598320433986373, + 5.751572909997776, + 5.904791719978675, + 6.0619585889799055, + 6.216954347997671, + 6.374703660985688, + 6.528412908985047, + 6.682621980988188, + 6.832279987982474, + 6.9855864889977966, + 7.140216140978737, + 7.293423180992249, + 7.4499539629905485, + 7.600570506998338, + 7.754262681002729, + 7.907677417999366, + 8.056161057000281, + 8.208133965992602, + 8.361953938001534, + 8.513835560996085, + 8.669855195999844, + 8.827508731978014, + 8.983549151977059, + 9.132588023989229, + 9.283133160002762, + 9.434843150986126, + 9.585382121993462, + 9.735999923985219, + 9.894053372991038, + 10.045744841976557, + 10.198623582982691, + 10.356770316982875, + 10.512349456985248, + 10.665897611994296, + 10.821172264986672, + 10.968750234984327, + 11.121885603002738, + 11.276642822980648, + 11.431809759000316, + 11.586014403990703, + 11.738798225997016, + 11.89640373198199, + 12.046210214000894, + 12.18420522697852, + 12.320644262974383, + 12.60315479899873, + 12.724938144994667, + 12.841600435000146, + 12.961748423986137, + 13.081872107984964, + 13.19798384400201, + 13.313174303999403, + 13.431633781990968, + 13.552759445999982, + 13.668460499000503, + 13.786741705989698, + 13.910192258976167, + 14.032381184981205, + 14.153041526995366, + 14.274385614000494, + 14.398021679982776, + 14.525518982001813, + 14.650034969992703, + 14.772966306976741, + 14.88887144299224, + 15.001553947979119 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 151.63280098931864, + "p95": 158.05344900581986, + "max": 282.51053602434695 + } + }, + { + "start": 132954.303981243, + "first": 132966.767761066, + "last": 132974.774355156, + "end": 132974.774489066, + "elapsed_s": 20.470507823018124, + "chunk_offsets_s": [ + 12.463779823010555, + 12.60333608801011, + 12.724956610007212, + 12.841767802019604, + 12.961900495021837, + 13.08188303699717, + 13.197989316016901, + 13.31296733400086, + 13.431804830004694, + 13.552738917001989, + 13.668529190006666, + 13.78679911600193, + 13.910226933017839, + 14.032441332004964, + 14.153234992001671, + 14.274558645993238, + 14.398122724000132, + 14.525493477005512, + 14.650295686005848, + 14.772953619016334, + 14.888936197996372, + 15.001650976017118, + 15.090067606011871, + 15.164841531019192, + 15.239746135019232, + 15.32315273801214, + 15.40103192901006, + 15.482894795015454, + 15.566479130007792, + 15.651144155010115, + 15.742370063002454, + 15.82178679300705, + 15.9089253419952, + 15.993925534014124, + 16.074115837021964, + 16.147537087003002, + 16.216373144998215, + 16.27875011699507, + 16.343078236008296, + 16.411151523003355, + 16.48281787399901, + 16.55631362000713, + 16.631609513016883, + 16.71161506700446, + 16.798591970000416, + 16.884032786008902, + 16.966115460993024, + 17.074488037993433, + 17.159167363017332, + 17.24154819501564, + 17.322802847018465, + 17.402844002004713, + 17.48609681200469, + 17.563095724995947, + 17.636312123999232, + 17.711742483021226, + 17.788833094993606, + 17.877667937020306, + 17.962981732998742, + 18.04497511900263, + 18.125530660996446, + 18.207319492998067, + 18.28740327002015, + 18.35950172902085, + 18.421605016017566, + 18.486814800999127, + 18.549139569018735, + 18.614155740011483, + 18.678004578017863, + 18.74461964101647, + 18.816350781999063, + 18.889427251007874, + 18.961408852017485, + 19.039248231012607, + 19.11609836301068, + 19.200268229993526, + 19.300764810002875, + 19.409446108998964, + 19.523625008005183, + 19.607971101009753, + 19.688531257997965, + 19.7571546510153, + 19.830336064012954, + 19.905265845998656, + 19.986106304015266, + 20.0940934040118, + 20.168220813997323, + 20.245328810007777, + 20.325778897997225, + 20.407705438003177, + 20.47037391300546 + ], + "chunk_gaps_ms": { + "count": 90, + "p50": 81.89470300567336, + "p95": 123.42781701590866, + "max": 139.55626499955542 + } + }, + { + "start": 132954.30303931, + "first": 132954.449590578, + "last": 132966.90734914, + "end": 132966.907456249, + "elapsed_s": 12.604416938993381, + "chunk_offsets_s": [ + 0.14655126800062135, + 0.2203438059950713, + 0.469199840008514, + 0.7448128830001224, + 0.8914773949945811, + 1.0410246519895736, + 1.1888693959917873, + 1.3274953640066087, + 1.478454480995424, + 1.6243138429999817, + 1.773432062997017, + 1.9197522290050983, + 2.0717314329813235, + 2.22343451200868, + 2.3767828379932325, + 2.5280974170018453, + 2.6749251349829137, + 2.8264761969912797, + 2.978024540003389, + 3.129202593001537, + 3.284240349981701, + 3.4362868600001093, + 3.5883512219879776, + 3.742385960998945, + 3.89256816598936, + 4.049070478999056, + 4.212307576002786, + 4.367299257981358, + 4.526315848983359, + 4.6833437609893735, + 4.834613304003142, + 4.986157164006727, + 5.135170183988521, + 5.28865048699663, + 5.4434646790032275, + 5.599117223988287, + 5.752344418986468, + 5.9055890860036016, + 6.062770307995379, + 6.217682766990038, + 6.375569461990381, + 6.5292478769843, + 6.683515798998997, + 6.8332284659845755, + 6.986162501008948, + 7.141016643989133, + 7.294221955991816, + 7.450728689000243, + 7.601295950007625, + 7.755087536002975, + 7.908486976986751, + 8.056987560994457, + 8.20889798598364, + 8.362679588986794, + 8.514656096987892, + 8.670755366998492, + 8.827841388003435, + 8.983988644991769, + 9.133550437982194, + 9.28394867098541, + 9.435315701994114, + 9.5859904710087, + 9.736832283990225, + 9.894566101982491, + 10.046518894989276, + 10.199383747996762, + 10.357520336983725, + 10.51320535398554, + 10.666652800980955, + 10.821939933986869, + 10.969319717987673, + 11.122491999994963, + 11.277410459995735, + 11.432873298006598, + 11.586575086985249, + 11.739987882989226, + 11.897301310993498, + 12.047129713988397, + 12.185109237994766, + 12.321375145984348, + 12.604309829999693 + ], + "chunk_gaps_ms": { + "count": 80, + "p50": 153.18879700498655, + "p95": 159.0165910020005, + "max": 282.9346840153448 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 29.2, + "per_stream_tps": 29.2, + "ttft_s_max": 0.22, + "timing_valid": true, + "end_to_end_tps": 28.62200487756052, + "makespan_s": 8.944167296984233, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 29.21366011063888, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 8.944167296984233, + "end_to_end_tps": 28.62200487756052, + "timing_valid": true, + "per_stream_tps": [ + 29.2 + ], + "ttft_s": [ + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + } + ], + "client_intervals": [ + { + "start": 133069.783174712, + "first": 133069.998282992, + "last": 133078.72707627, + "end": 133078.727342009, + "elapsed_s": 8.944167296984233, + "chunk_offsets_s": [ + 0.21510827998281457, + 0.29334825999103487, + 0.3669840339862276, + 0.43225181198795326, + 0.4998941840021871, + 0.572779765992891, + 0.6375017940008547, + 0.7010930230026133, + 0.7662611010018736, + 0.8310261539882049, + 0.8949599409825169, + 0.9589293939934578, + 1.0233079989848193, + 1.0883650459873024, + 1.1517266779846977, + 1.2156629569944926, + 1.2821508789784275, + 1.3578939359867945, + 1.4341855600068811, + 1.5071315889945254, + 1.5824404039885849, + 1.6715179329912644, + 1.7541597870003898, + 1.8327865159953944, + 1.8963552119967062, + 1.9603938479849603, + 2.0227124450029805, + 2.0866137449920643, + 2.1518502830003854, + 2.2279583280032966, + 2.3018874019908253, + 2.374801153986482, + 2.4492318329866976, + 2.5335253999801353, + 2.619033272989327, + 2.699605806003092, + 2.786139784991974, + 2.867227412003558, + 2.95245802498539, + 3.027362388005713, + 3.1016226719948463, + 3.1748207199852914, + 3.249468475987669, + 3.3329300209879875, + 3.415833909995854, + 3.499308366997866, + 3.5675653469807003, + 3.643809216999216, + 3.717884597979719, + 3.7926300060062204, + 3.8705970519804396, + 3.951811414997792, + 4.018390629993519, + 4.090156616002787, + 4.154128647991456, + 4.217166657006601, + 4.28377915098099, + 4.359627897007158, + 4.43263700598618, + 4.507129151985282, + 4.579994014988188, + 4.644313228985993, + 4.7123228359851055, + 4.785992227989482, + 4.858036769001046, + 4.933719716995256, + 5.00893698100117, + 5.097357202001149, + 5.1736962529830635, + 5.2396590519929305, + 5.313343880989123, + 5.376684443996055, + 5.43989084099303, + 5.504546993994154, + 5.570056526979897, + 5.634571409987984, + 5.699734431982506, + 5.763685611978872, + 5.829566264001187, + 5.8948846840066835, + 5.959584614989581, + 6.022997533000307, + 6.084991604991956, + 6.148522556992248, + 6.214216445980128, + 6.286230722995242, + 6.360584407986607, + 6.436631043005036, + 6.514989644987509, + 6.593869053001981, + 6.658456784003647, + 6.7222561620001215, + 6.78937969400431, + 6.853553535998799, + 6.919768261985155, + 6.9941602849867195, + 7.067411219002679, + 7.142993238987401, + 7.216071828006534, + 7.279350911005167, + 7.343348309979774, + 7.407776959997136, + 7.471574862982379, + 7.535593287990196, + 7.5998780759982765, + 7.665876326005673, + 7.730113286990672, + 7.794238419999601, + 7.857791769987671, + 7.923386882990599, + 7.986852981004631, + 8.05090652598301, + 8.114621110988082, + 8.183307723986218, + 8.259027394989971, + 8.333469364006305, + 8.405778382002609, + 8.48274368798593, + 8.562195062986575, + 8.626926553988596, + 8.690741414990043, + 8.754491697996855, + 8.82004093399155, + 8.883258906978881, + 8.943901557999197 + ], + "chunk_gaps_ms": { + "count": 124, + "p50": 67.3829520092113, + "p95": 83.47445700201206, + "max": 89.07752900267951 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.9, + "per_stream_tps": 20.4, + "ttft_s_max": 0.51, + "timing_valid": true, + "end_to_end_tps": 39.424584383089815, + "makespan_s": 12.986820482998155, + "steady_tps": 40.8, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.933529706523444, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 12.986820482998155, + "end_to_end_tps": 39.424584383089815, + "timing_valid": true, + "steady_tps": 40.77245963389988, + "per_stream_tps": [ + 20.4, + 20.4 + ], + "ttft_s": [ + 0.22, + 0.51 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + } + ], + "client_intervals": [ + { + "start": 133078.728977692, + "first": 133078.944007078, + "last": 133091.446980531, + "end": 133091.447353186, + "elapsed_s": 12.718375494005159, + "chunk_offsets_s": [ + 0.21502938601770438, + 0.2931522070139181, + 0.6112932430114597, + 0.7268936500186101, + 0.8480072579986881, + 0.9668904400023166, + 1.0815490960085299, + 1.1993061609973665, + 1.316839458013419, + 1.455302460002713, + 1.6098592730122618, + 1.767861044005258, + 1.9073253830138128, + 2.0312242840009276, + 2.1497925640142057, + 2.2684855660190806, + 2.3908328289981, + 2.511594572017202, + 2.63269492101972, + 2.7503374319931027, + 2.8724597110121977, + 2.992552706011338, + 3.1147605220030528, + 3.2390611880109645, + 3.360448182997061, + 3.483252450998407, + 3.606264055997599, + 3.7327298349991906, + 3.852885819011135, + 3.973335654998664, + 4.0908938530192245, + 4.21458232801524, + 4.3350637440162245, + 4.448135176993674, + 4.56353190500522, + 4.681851216999348, + 4.797648520994699, + 4.911062032013433, + 5.034153779008193, + 5.152923601010116, + 5.269578823004849, + 5.388685729005374, + 5.506578442000318, + 5.630237645993475, + 5.754532315011602, + 5.8806264660088345, + 5.999242184014292, + 6.121307961002458, + 6.2403704750176985, + 6.363264874002198, + 6.481489275000058, + 6.603175176016521, + 6.727322154009016, + 6.850571751012467, + 6.972046812006738, + 7.0971306590072345, + 7.217362415016396, + 7.340520002995618, + 7.458429969003191, + 7.578977113997098, + 7.697471456020139, + 7.818870524002705, + 7.942781710007694, + 8.063959473016439, + 8.18138541400549, + 8.298417931015138, + 8.416602319019148, + 8.534819299995434, + 8.656093557015993, + 8.777903948008316, + 8.899993086000904, + 9.026344189012889, + 9.149957993999124, + 9.272367219993612, + 9.397909035993507, + 9.523563098016893, + 9.643558720999863, + 9.766792764014099, + 9.885560674010776, + 10.003934423002647, + 10.128153176017804, + 10.24841501301853, + 10.364537993998965, + 10.482569084007991, + 10.600847880006768, + 10.72004287299933, + 10.83972358200117, + 10.95955205700011, + 11.076727128995117, + 11.193940196011681, + 11.318093817011686, + 11.437509534007404, + 11.56376055101282, + 11.686259705020348, + 11.803134486021008, + 11.925955973012606, + 12.047583634994226, + 12.164526784006739, + 12.283376951992977, + 12.403363902005367, + 12.518382987997029, + 12.617670492007164, + 12.718002839013934 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 120.35583649412729, + "p95": 126.46577900159173, + "max": 318.1410359975416 + } + }, + { + "start": 133078.728881624, + "first": 133079.234947558, + "last": 133091.715229751, + "end": 133091.715702107, + "elapsed_s": 12.986820482998155, + "chunk_offsets_s": [ + 0.5060659339942504, + 0.6109404129965696, + 0.7268970739969518, + 0.8477180149930064, + 0.9665405539853964, + 1.081142359005753, + 1.1991980450111441, + 1.316766990988981, + 1.4551724699849728, + 1.6096398560039233, + 1.767753680003807, + 1.9076599570107646, + 2.031142280990025, + 2.1498181330098305, + 2.268147839989979, + 2.390720297000371, + 2.5118385019886773, + 2.6326661519997288, + 2.7502720539923757, + 2.8724622869922314, + 2.992553201998817, + 3.1147960599919315, + 3.23904123500688, + 3.360690320987487, + 3.483180415991228, + 3.606145956000546, + 3.7326683279825374, + 3.852795126993442, + 3.9733278790081386, + 4.090929422003683, + 4.214370047004195, + 4.33505651098676, + 4.448101368005155, + 4.5635315850086045, + 4.68183724800474, + 4.797538484999677, + 4.911031534982612, + 5.034098225005437, + 5.1529100810003, + 5.2694427219976205, + 5.388565948989708, + 5.5066136269888375, + 5.630175898986636, + 5.7544052219891455, + 5.8805191020073835, + 5.999243831989588, + 6.121273127995664, + 6.240273272007471, + 6.363509315997362, + 6.4815238199953455, + 6.603062212001532, + 6.727229221985908, + 6.850403311982518, + 6.972046459995909, + 7.097128834982868, + 7.2172931320092175, + 7.34044023999013, + 7.4584650739852805, + 7.578860292996978, + 7.697458751004888, + 7.818871355993906, + 7.942790717992466, + 8.06390911899507, + 8.18137259798823, + 8.298318710993044, + 8.416524587984895, + 8.534657037002034, + 8.655923629994504, + 8.778191222983878, + 8.899961101007648, + 9.02658592700027, + 9.14982586100814, + 9.272310786007438, + 9.39815467898734, + 9.523429667984601, + 9.643736425001407, + 9.766802541009383, + 9.885826060984982, + 10.003855107992422, + 10.1281750969938, + 10.248430229985388, + 10.364803748001577, + 10.482831189990975, + 10.600908410007833, + 10.72030213198741, + 10.840045226999791, + 10.95968486298807, + 11.07699093199335, + 11.19424638498458, + 11.318407398008276, + 11.437768103991402, + 11.563892300997395, + 11.686393805983244, + 11.803264540008968, + 11.925884369993582, + 12.04771471300046, + 12.164531215996249, + 12.283509517990751, + 12.403494644007878, + 12.51852515299106, + 12.617802897992078, + 12.718139148986666, + 12.79715128900716, + 12.860819913010346, + 12.92593443498481, + 12.986348126985831 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 120.16429702634923, + "p95": 126.52237198199145, + "max": 158.1138239998836 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.5, + "per_stream_tps": 16.2, + "ttft_s_max": 16.12, + "timing_valid": true, + "end_to_end_tps": 39.33300974210784, + "makespan_s": 26.034112485009246, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.50744115250233, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 26.034112485009246, + "end_to_end_tps": 39.33300974210784, + "timing_valid": true, + "per_stream_tps": [ + 16.3, + 15.8, + 16.2, + 25.7 + ], + "ttft_s": [ + 0.22, + 0.51, + 0.82, + 16.12 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2e4cb369c515", + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8" + } + ], + "client_intervals": [ + { + "start": 133091.717826897, + "first": 133091.933647894, + "last": 133107.625201532, + "end": 133107.625312096, + "elapsed_s": 15.907485199015355, + "chunk_offsets_s": [ + 0.21582099702209234, + 0.29346259802696295, + 0.6120185750187375, + 0.9618416070006788, + 1.1040234000247438, + 1.2506946150097065, + 1.3965758470003493, + 1.537081439018948, + 1.6765800130087882, + 1.8177939690067433, + 1.963221230020281, + 2.1143283680139575, + 2.261960544012254, + 2.417388858011691, + 2.569306151010096, + 2.7205304560193326, + 2.878860151016852, + 3.0264657650259323, + 3.1785602610034402, + 3.325668873003451, + 3.4788601930195, + 3.6339874640107155, + 3.7878796290024184, + 3.9441431060258765, + 4.088525048020529, + 4.239112209004816, + 4.391384829010349, + 4.545809073024429, + 4.693133254011627, + 4.843829652003478, + 4.997775828000158, + 5.157851367024705, + 5.310236593009904, + 5.456399713002611, + 5.607255902024917, + 5.757698490022449, + 5.914325359015493, + 6.061414820025675, + 6.212349934008671, + 6.361834093026118, + 6.513042833015788, + 6.660156448022462, + 6.806902993004769, + 6.958639754011529, + 7.115861758007668, + 7.274283980019391, + 7.430083793005906, + 7.5845277910120785, + 7.736120812012814, + 7.893729449016973, + 8.046472323010676, + 8.201382847008063, + 8.360184677003417, + 8.518373874016106, + 8.674445739015937, + 8.833943436999107, + 8.986424277012702, + 9.139945510018151, + 9.287343020027038, + 9.43670475302497, + 9.58543569300673, + 9.739587313000811, + 9.894723853009054, + 10.047285378008382, + 10.190512829023646, + 10.334206137020374, + 10.48410850900109, + 10.633580303023336, + 10.78840893600136, + 10.944750637019752, + 11.098241087020142, + 11.253463152010227, + 11.410592974018073, + 11.563558021007339, + 11.72293610800989, + 11.876480458013248, + 12.028141037008027, + 12.17996968200896, + 12.32714943602332, + 12.477818224026123, + 12.63172357401345, + 12.784090638015186, + 12.930241022026166, + 13.076623576023849, + 13.227836652018595, + 13.379762828000821, + 13.53123679300188, + 13.679193579009734, + 13.826833376020659, + 13.9771132300084, + 14.130531063012313, + 14.279662145010661, + 14.435424930008594, + 14.587339626014, + 14.735440618998837, + 14.887898441025754, + 15.034748600010062, + 15.185492269025417, + 15.337657624011626, + 15.488642933021765, + 15.635822186013684, + 15.771420322998893, + 15.907374635018641 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 151.6267999977572, + "p95": 158.80182999535464, + "max": 349.8230319819413 + } + }, + { + "start": 133091.71839812, + "first": 133092.224814252, + "last": 133108.3859838, + "end": 133108.386092428, + "elapsed_s": 16.66769430800923, + "chunk_offsets_s": [ + 0.5064161320042331, + 0.6118858169938903, + 0.9617315380019136, + 1.1041246200038586, + 1.2506478300201707, + 1.3964934440155048, + 1.536956010007998, + 1.6763858290214557, + 1.8176430190214887, + 1.9627349229995161, + 2.1142551969969645, + 2.2618471320020035, + 2.417293830018025, + 2.5690186510037165, + 2.7203428799984977, + 2.8787146730173845, + 3.026375506015029, + 3.1780715369968675, + 3.3255460360087454, + 3.4786817700078245, + 3.6338939240085892, + 3.7878332270192914, + 3.9433492660173215, + 4.088182586012408, + 4.2383072639931925, + 4.391040096001234, + 4.544879787019454, + 4.692325798008824, + 4.843017987004714, + 4.997433734009974, + 5.157246159011265, + 5.309621337015415, + 5.455789800995262, + 5.606939104996854, + 5.756991902017035, + 5.91398244901211, + 6.060624803998508, + 6.211540846008575, + 6.361216356017394, + 6.512424504006049, + 6.659542968001915, + 6.806295832997421, + 6.958026177017018, + 7.1155095360009, + 7.273755735019222, + 7.429464584012749, + 7.583999641996343, + 7.735514643019997, + 7.8931957790046, + 8.045853882009396, + 8.200636913010385, + 8.35983658299665, + 8.517757466004696, + 8.673829570005182, + 8.833327636006288, + 8.98581556501449, + 9.139329069002997, + 9.286912251001922, + 9.43608783200034, + 9.584899879002478, + 9.738962680014083, + 9.894092692004051, + 10.046679562015925, + 10.189906133018667, + 10.333596993004903, + 10.483492964005563, + 10.632874931005063, + 10.787797886994667, + 10.943950061016949, + 11.097596134000923, + 11.252657775999978, + 11.409796126012225, + 11.562947852013167, + 11.72232917099609, + 11.876128683012212, + 12.02765287301736, + 12.179351608996512, + 12.32667928902083, + 12.477335293020587, + 12.631092188006733, + 12.78343125100946, + 12.9297070169996, + 13.076004207017832, + 13.227223955007503, + 13.37890196099761, + 13.530622784019215, + 13.678667397995014, + 13.82621171901701, + 13.97649046900915, + 14.130002513993531, + 14.279127275018254, + 14.434919598017586, + 14.586822868994204, + 14.734906997007784, + 14.88727779200417, + 15.034229443001095, + 15.18496863200562, + 15.337141316005727, + 15.488106303004315, + 15.635312708996935, + 15.770884686004138, + 15.90690101601649, + 16.270054950000485, + 16.425007320998702, + 16.562442821013974, + 16.66758567999932 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 151.52419000514783, + "p95": 159.19966998626478, + "max": 363.15393398399465 + } + }, + { + "start": 133091.718908316, + "first": 133092.542153091, + "last": 133108.281336289, + "end": 133108.281405011, + "elapsed_s": 16.562496694998117, + "chunk_offsets_s": [ + 0.8232447750051506, + 0.9612770570092835, + 1.10340244701365, + 1.2499583780008834, + 1.3962596589990426, + 1.5365654659981374, + 1.6760438000201248, + 1.8172496110200882, + 1.9622655599960126, + 2.113584050995996, + 2.261181313020643, + 2.41662420402281, + 2.5685542169958353, + 2.719867725012591, + 2.8780311260197777, + 3.025760905002244, + 3.1776886740117334, + 3.324945052008843, + 3.4782044080202468, + 3.633421138016274, + 3.787158241000725, + 3.9431729869975243, + 4.087766778015066, + 4.238131849997444, + 4.390792470017914, + 4.544516093010316, + 4.692346519004786, + 4.842858909018105, + 4.997062983020442, + 5.157220941997366, + 5.308921629010001, + 5.455720343015855, + 5.6066218450141605, + 5.756766709004296, + 5.9137275270186365, + 6.060402236005757, + 6.21136090302025, + 6.3609967000084, + 6.512290483020479, + 6.659335424017627, + 6.806046767014777, + 6.957791032007663, + 7.1151417450164445, + 7.273580848006532, + 7.429206878005061, + 7.583921543002361, + 7.7353754060168285, + 7.892951002024347, + 8.0458521550172, + 8.200198703998467, + 8.3594997220207, + 8.517709895997541, + 8.673469236004166, + 8.83328329899814, + 8.985666871012654, + 9.139123285014648, + 9.286540172004607, + 9.435875152004883, + 9.584826069010887, + 9.73894093601848, + 9.893837498006178, + 10.046601142996224, + 10.189910918008536, + 10.333398137008771, + 10.48343728200416, + 10.632606185012264, + 10.787551990011707, + 10.943764598021517, + 11.097548196004936, + 11.252476873021806, + 11.409606550005265, + 11.562748035998084, + 11.722191614011535, + 11.876125772017986, + 12.026967406010954, + 12.178642782004317, + 12.326335579011356, + 12.476680738996947, + 12.6308893969981, + 12.783202459017048, + 12.929495329008205, + 13.075811304006493, + 13.226998971018475, + 13.378525706008077, + 13.530290308000986, + 13.67857985800947, + 13.826039152016165, + 13.97628746900591, + 14.129765065008542, + 14.27890621102415, + 14.43461656200816, + 14.586510601016926, + 14.73469284499879, + 14.887042982998537, + 15.034105246013496, + 15.18467503701686, + 15.336383142013801, + 15.488058301009005, + 15.635203089012066, + 15.770671638019849, + 15.906810643995414, + 16.269882287015207, + 16.424866932007717, + 16.562427972996375 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 151.52673498960212, + "p95": 158.43910299008712, + "max": 363.07164301979356 + } + }, + { + "start": 133091.717738621, + "first": 133107.838198614, + "last": 133117.751569031, + "end": 133117.751851106, + "elapsed_s": 26.034112485009246, + "chunk_offsets_s": [ + 16.120459992991528, + 16.271229990001302, + 16.42620316101238, + 16.563409947993932, + 16.668745678995037, + 16.75380730899633, + 16.833096551010385, + 16.900814901018748, + 16.96994154699496, + 17.033976456004893, + 17.09769909500028, + 17.162327464000555, + 17.226945528003853, + 17.291687966004247, + 17.35699282499263, + 17.421692669013282, + 17.486495732999174, + 17.552344230003655, + 17.61661312900833, + 17.68041138499393, + 17.743725477019325, + 17.80792979701073, + 17.873228576994734, + 17.939072145993123, + 18.002886161993956, + 18.067660136992345, + 18.133143017999828, + 18.197599489009008, + 18.261780828994233, + 18.324703591992147, + 18.39110482801334, + 18.456516361009562, + 18.521993447997374, + 18.588298904011026, + 18.66345999101759, + 18.73950508100097, + 18.815569739992497, + 18.892945010011317, + 18.975272876006784, + 19.043476135993842, + 19.116947681992315, + 19.18034503699164, + 19.24681191999116, + 19.31065778899938, + 19.37767005400383, + 19.443765706993872, + 19.510467031999724, + 19.585389865998877, + 19.660291547013912, + 19.73375915500219, + 19.80882265101536, + 19.88652631500736, + 19.975284815998748, + 20.05936972401105, + 20.142117520998, + 20.225772056001006, + 20.307845777017064, + 20.380572073016083, + 20.44584573901375, + 20.51187665000907, + 20.578139538993128, + 20.643109224009095, + 20.707552904990735, + 20.770737144019222, + 20.833435444015777, + 20.899062420008704, + 20.963093747996027, + 21.02799715899164, + 21.093862672016257, + 21.17132214299636, + 21.25324369399459, + 21.334490659006406, + 21.41979320801329, + 21.502234907005914, + 21.58304817500175, + 21.658339768997394, + 21.72167882701615, + 21.78620697199949, + 21.850938309013145, + 21.916423916001804, + 21.98277640499873, + 22.04765502800001, + 22.112199333001627, + 22.17715246800799, + 22.241093099990394, + 22.303566296992358, + 22.36843926901929, + 22.43375096199452, + 22.499570179003058, + 22.563620063010603, + 22.628296067006886, + 22.69296904699877, + 22.756025531009072, + 22.820233420992736, + 22.88388776901411, + 22.94780071900459, + 23.01276180599234, + 23.0771415749914, + 23.14203548899968, + 23.209047614014708, + 23.274545904016122, + 23.3393633190135, + 23.404253599990625, + 23.469367234996753, + 23.533675573009532, + 23.596790089010028, + 23.662284507008735, + 23.728190508001717, + 23.7916855509975, + 23.85940613600542, + 23.926176540000597, + 24.00138860399602, + 24.076137096999446, + 24.148642764019314, + 24.212965596001595, + 24.278297508019023, + 24.34234748201561, + 24.407963453006232, + 24.472430964000523, + 24.5367688210099, + 24.602081676013768, + 24.66710055200383, + 24.730157286016038, + 24.794861302012578, + 24.860466665006243, + 24.932297855993966, + 24.99472007001168, + 25.057987036998384, + 25.123015112010762, + 25.198696611012565, + 25.271244196017506, + 25.3443633709976, + 25.421298856002977, + 25.5036307600094, + 25.56882997599314, + 25.64414784600376, + 25.715038388996618, + 25.778714921005303, + 25.843003237008816, + 25.908662071014987, + 25.97351205800078, + 26.03383041001507 + ], + "chunk_gaps_ms": { + "count": 141, + "p50": 65.41153299622238, + "p95": 84.08490801230073, + "max": 154.9731710110791 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 56.1, + "per_stream_tps": 56.1, + "ttft_s_max": 0.15, + "timing_valid": true, + "end_to_end_tps": 54.495886321860844, + "makespan_s": 4.6976022830058355, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 56.05562719508342, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 4.6976022830058355, + "end_to_end_tps": 54.495886321860844, + "timing_valid": true, + "per_stream_tps": [ + 56.1 + ], + "ttft_s": [ + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 133137.030673762, + "first": 133137.179078275, + "last": 133141.728130935, + "end": 133141.728276045, + "elapsed_s": 4.6976022830058355, + "chunk_offsets_s": [ + 0.14840451299096458, + 0.2245780449884478, + 0.2968298589985352, + 0.3745069850119762, + 0.4583583800122142, + 0.5448023700155318, + 0.631601013999898, + 0.716005085996585, + 0.7979487270058598, + 0.8818578800128307, + 0.9631819580099545, + 1.044388656009687, + 1.1297750340017956, + 1.2124856779992115, + 1.294602667010622, + 1.3783685509988572, + 1.4610578650026582, + 1.5332554659980815, + 1.5983201210037805, + 1.662421249988256, + 1.729959919990506, + 1.8059703219914809, + 1.8814161549962591, + 1.9539043719996698, + 2.028559471014887, + 2.1131781520089135, + 2.19669786299346, + 2.277005626994651, + 2.3552490879956167, + 2.4599304730072618, + 2.5630346309917513, + 2.668979807989672, + 2.7779940640029963, + 2.8817450249916874, + 2.9918938470073044, + 3.0980423240107484, + 3.209432858013315, + 3.3195383670099545, + 3.429886780009838, + 3.5381297540152445, + 3.6480859449948184, + 3.7553520319925155, + 3.8640582840016577, + 3.9738810409908183, + 4.084352687001228, + 4.189623654005118, + 4.2986114190134685, + 4.405937072006054, + 4.51271056101541, + 4.623355093004648, + 4.6974571729952 + ], + "chunk_gaps_ms": { + "count": 50, + "p50": 84.51137649535667, + "p95": 110.47164601041004, + "max": 111.39053400256671 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 62.8, + "per_stream_tps": 32.0, + "ttft_s_max": 0.37, + "timing_valid": true, + "end_to_end_tps": 61.966239094653794, + "makespan_s": 8.262563736003358, + "steady_tps": 64.2, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 62.836244452087335, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 8.262563736003358, + "end_to_end_tps": 61.966239094653794, + "timing_valid": true, + "steady_tps": 64.19935448288678, + "per_stream_tps": [ + 31.7, + 32.3 + ], + "ttft_s": [ + 0.15, + 0.37 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 133141.729447545, + "first": 133141.875474435, + "last": 133149.918570983, + "end": 133149.91864985, + "elapsed_s": 8.189202305016806, + "chunk_offsets_s": [ + 0.14602688999730162, + 0.22010544600198045, + 0.4694109790143557, + 0.577668904006714, + 0.6842023780045565, + 0.7929407370102126, + 0.9025645220244769, + 1.0103659809974488, + 1.1156156290089712, + 1.2182001970068086, + 1.3162158870254643, + 1.4183266280160751, + 1.524505957000656, + 1.6404892430000473, + 1.7523877659987193, + 1.8653758990112692, + 1.980386859999271, + 2.0954778880113736, + 2.2106965650164057, + 2.3243120280094445, + 2.440399426006479, + 2.557811242004391, + 2.6708548270107713, + 2.7808510970207863, + 2.8935157240193803, + 3.007257432996994, + 3.1183895520225633, + 3.2239028780022636, + 3.3323860460077412, + 3.440965904999757, + 3.5502856000093743, + 3.6570529260206968, + 3.767777699016733, + 3.8811803170247003, + 4.0192316500179, + 4.165070523013128, + 4.310648615006357, + 4.446373084996594, + 4.557681906997459, + 4.672659141011536, + 4.78767562401481, + 4.898392903996864, + 5.011116676003439, + 5.126335709006526, + 5.239409025001805, + 5.3542541050119326, + 5.47091743000783, + 5.585680634016171, + 5.700383181014331, + 5.813583670009393, + 5.930008074006764, + 6.044186423998326, + 6.1581993410072755, + 6.269299586012494, + 6.383760886004893, + 6.4979189750156365, + 6.61057310301112, + 6.7252385550236795, + 6.8401258070080075, + 6.955148121021921, + 7.068996134010376, + 7.178277026017895, + 7.29269275200204, + 7.406706750014564, + 7.515401219017804, + 7.627062137005851, + 7.741465462022461, + 7.852743277006084, + 7.965848952997476, + 8.080008924996946, + 8.189123438001843 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 113.15308249322698, + "p95": 138.05133299320005, + "max": 249.30553301237524 + } + }, + { + "start": 133141.729317781, + "first": 133142.094545646, + "last": 133149.991809274, + "end": 133149.991881517, + "elapsed_s": 8.262563736003358, + "chunk_offsets_s": [ + 0.3652278649969958, + 0.4693233110010624, + 0.5776032520225272, + 0.6844509789953008, + 0.7932405559986364, + 0.9025683290092275, + 1.010667912021745, + 1.1155128090176731, + 1.2181608829996549, + 1.3161760920193046, + 1.4182848500204273, + 1.52446470601717, + 1.6405333400180098, + 1.752483641001163, + 1.8654630689998157, + 1.9807469850056805, + 2.095641493011499, + 2.210789991018828, + 2.324277866020566, + 2.4404923080001026, + 2.557903900014935, + 2.6709481260040775, + 2.7809468760096934, + 2.8937406440090854, + 3.007353883003816, + 3.118474562012125, + 3.224197464005556, + 3.332553315005498, + 3.4410203380102757, + 3.5503812030074187, + 3.6571455050143413, + 3.768067693017656, + 3.881273039994994, + 4.019322900014231, + 4.165144140017219, + 4.310743866022676, + 4.44646654400276, + 4.557774198008701, + 4.672752664017025, + 4.787899232003838, + 4.898609199008206, + 5.011212823010283, + 5.126433038996765, + 5.239492835011333, + 5.354349388013361, + 5.471012535999762, + 5.5857726210088, + 5.700477535021491, + 5.813663048000308, + 5.930102412006818, + 6.044484290003311, + 6.158291694999207, + 6.2696240130171645, + 6.384051344008185, + 6.498014786018757, + 6.6107651090133, + 6.72531506899395, + 6.840218994009774, + 6.95524151599966, + 7.069083337002667, + 7.178374020993942, + 7.2929855940165, + 7.40687016400625, + 7.5155653370020445, + 7.627226126001915, + 7.7415585209964775, + 7.8529071700177155, + 7.9659424440178555, + 8.080105088010896, + 8.189212560013402, + 8.262491493020207 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 113.05201100185513, + "p95": 135.72267798008397, + "max": 145.8212400029879 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.9, + "per_stream_tps": 26.1, + "ttft_s_max": 10.08, + "timing_valid": true, + "end_to_end_tps": 69.46815527665999, + "makespan_s": 14.7405670399894, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.89064068855477, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 14.7405670399894, + "end_to_end_tps": 69.46815527665999, + "timing_valid": true, + "per_stream_tps": [ + 26.0, + 54.8, + 26.2, + 25.9 + ], + "ttft_s": [ + 0.15, + 10.08, + 0.61, + 0.37 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 133149.993837831, + "first": 133150.139998193, + "last": 133159.933151184, + "end": 133159.933238915, + "elapsed_s": 9.939401084004203, + "chunk_offsets_s": [ + 0.14616036199731752, + 0.22142263301066123, + 0.46902412999770604, + 0.7489414250012487, + 0.8797537150094286, + 1.0129611020092852, + 1.1461763920087833, + 1.2788650870206766, + 1.408449009002652, + 1.5329258890124038, + 1.653332424000837, + 1.7775995999982115, + 1.905834027013043, + 2.0457923990034033, + 2.190081717999419, + 2.3297034690040164, + 2.4700047340011224, + 2.6133409129979555, + 2.752107688022079, + 2.89743593899766, + 3.040174952999223, + 3.182940097001847, + 3.328278714005137, + 3.465541858022334, + 3.6023415770032443, + 3.739442218997283, + 3.8765831000055186, + 4.009464346017921, + 4.141192412003875, + 4.274465256021358, + 4.404557485016994, + 4.539791907009203, + 4.675048458011588, + 4.809524227021029, + 4.943773343024077, + 5.079909841006156, + 5.219424451002851, + 5.353470215020934, + 5.488269446999766, + 5.627929645997938, + 5.767861551023088, + 5.905321584024932, + 6.041394186002435, + 6.181641617004061, + 6.318906872998923, + 6.460094195004785, + 6.602656322007533, + 6.744251868018182, + 6.8852736450207885, + 7.024523016996682, + 7.165447380015394, + 7.3042832810024265, + 7.443240434018662, + 7.582691621006234, + 7.7203720710240304, + 7.859367287019268, + 7.998259600019082, + 8.14055029701558, + 8.280651391018182, + 8.420250562019646, + 8.560193042998435, + 8.697726889018668, + 8.835853124008281, + 8.974272134015337, + 9.111270385998068, + 9.248925063002389, + 9.38572730802116, + 9.525371243013069, + 9.664359090005746, + 9.802983627014328, + 9.93931335301022 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 138.27262249833439, + "p95": 145.3282509755809, + "max": 279.9172950035427 + } + }, + { + "start": 133149.994589443, + "first": 133160.077126012, + "last": 133164.734226976, + "end": 133164.734293874, + "elapsed_s": 14.73970443100552, + "chunk_offsets_s": [ + 10.0825365690107, + 10.220877952990122, + 10.33226724198903, + 10.417119152989471, + 10.503978402994107, + 10.588780665013473, + 10.67490206300863, + 10.757979604008142, + 10.838781650993042, + 10.921380337997107, + 11.005619567004032, + 11.087737357011065, + 11.17167543899268, + 11.25452627899358, + 11.33676055300748, + 11.420671706000576, + 11.503230424015783, + 11.575624010001775, + 11.640228072996251, + 11.703453891997924, + 11.77250834601, + 11.848311978013953, + 11.923811084998306, + 11.996095864014933, + 12.071106088988017, + 12.155290080001578, + 12.238801182014868, + 12.319525844010059, + 12.397740189015167, + 12.501385898998706, + 12.604598200996406, + 12.710585421998985, + 12.819610932987416, + 12.922842414991464, + 13.033129249000922, + 13.139252230990678, + 13.250718772003893, + 13.36077988499892, + 13.471242228988558, + 13.579980109003372, + 13.689530883013504, + 13.796862087008776, + 13.905733710998902, + 14.015370180015452, + 14.126095009996789, + 14.231719264993444, + 14.340731486008735, + 14.44802404398797, + 14.554824471008033, + 14.665147109015379, + 14.73963753299904 + ], + "chunk_gaps_ms": { + "count": 50, + "p50": 85.48665449779946, + "p95": 111.38928899890743, + "max": 138.34138397942297 + } + }, + { + "start": 133149.994008941, + "first": 133150.608068671, + "last": 133160.326504655, + "end": 133160.326607267, + "elapsed_s": 10.332598326000152, + "chunk_offsets_s": [ + 0.6140597299963702, + 0.7486732710094657, + 0.8794257830013521, + 1.012617872998817, + 1.1458417880057823, + 1.278834718017606, + 1.4081597980111837, + 1.532558164006332, + 1.6529789070191327, + 1.7773757679970004, + 1.9055835220206063, + 2.045576199016068, + 2.1898155480157584, + 2.3294531560095493, + 2.4697534129954875, + 2.613129049015697, + 2.7520834790193476, + 2.897193386015715, + 3.040182682016166, + 3.1829203050001524, + 3.327919613017002, + 3.465299338015029, + 3.60233773701475, + 3.7394790050166193, + 3.8763738769921474, + 4.0091516459942795, + 4.141204029001528, + 4.274500538012944, + 4.40454916501767, + 4.539834373019403, + 4.675082027009921, + 4.809523475996684, + 4.943772799015278, + 5.079905138001777, + 5.219293406000361, + 5.353464262996567, + 5.488037806993816, + 5.627928205998614, + 5.767853950994322, + 5.9050895270193, + 6.04138781799702, + 6.18163206600002, + 6.318697041017003, + 6.460034657997312, + 6.602660625998396, + 6.7442608769924846, + 6.88526878101402, + 7.024518826015992, + 7.165328496019356, + 7.304284097015625, + 7.4432421140081715, + 7.582694646000164, + 7.720013114012545, + 7.859366696007783, + 7.998257585015381, + 8.140537977014901, + 8.28044079101528, + 8.420126893994166, + 8.560182707005879, + 8.69772274501156, + 8.835843395994743, + 8.974146913009463, + 9.111259810015326, + 9.248919734993251, + 9.38560191099532, + 9.525248518999433, + 9.664232269016793, + 9.802859671006445, + 9.939096017013071, + 10.221364116005134, + 10.33249571401393 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 137.8902879805537, + "p95": 144.23934899969026, + "max": 282.26809899206273 + } + }, + { + "start": 133149.993726834, + "first": 133150.359087586, + "last": 133160.214963794, + "end": 133160.21509071, + "elapsed_s": 10.221363875985844, + "chunk_offsets_s": [ + 0.36536075198091567, + 0.4690637159801554, + 0.7487670029804576, + 0.8798944409936666, + 1.0129454859998077, + 1.1463173899974208, + 1.2790148059721105, + 1.408607959980145, + 1.5330674149736296, + 1.6534749419952277, + 1.777745862986194, + 1.9059131029935088, + 2.0459311249724124, + 2.1901534969802015, + 2.3297825449844822, + 2.4700828489731066, + 2.6134937679744326, + 2.7522623499971814, + 2.897516534983879, + 3.040243228984764, + 3.183240524987923, + 3.3284234239836223, + 3.465621381998062, + 3.602415948989801, + 3.7395905459998176, + 3.876505577994976, + 4.0095333569915965, + 4.141339681984391, + 4.274611757980892, + 4.404632879974088, + 4.539940840972122, + 4.675197199976537, + 4.809463009994943, + 4.943742334988201, + 5.079907697974704, + 5.219422499998473, + 5.353389939991757, + 5.48800106998533, + 5.627931454975624, + 5.767713177978294, + 5.905164042982506, + 6.041344280994963, + 6.1814259459788445, + 6.318731282983208, + 6.459897596971132, + 6.602621456986526, + 6.744205834984314, + 6.885151960974326, + 7.024399876972893, + 7.165227548975963, + 7.3040547449782025, + 7.443194336985471, + 7.58258675399702, + 7.720117405988276, + 7.859326262987452, + 7.998128203995293, + 8.140382690995466, + 8.280426934972638, + 8.420116781984689, + 8.559981179976603, + 8.697659190976992, + 8.835684334975667, + 8.974140304984758, + 9.111188926995965, + 9.248835459991824, + 9.385476242983714, + 9.525337001978187, + 9.66418426798191, + 9.802885383978719, + 9.939161299989792, + 10.221236959972885 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 138.2405570038827, + "p95": 145.18289899569936, + "max": 282.0756599830929 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 30.6, + "per_stream_tps": 30.6, + "ttft_s_max": 0.22, + "timing_valid": true, + "end_to_end_tps": 29.907326707600312, + "makespan_s": 8.559775419009384, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 30.560446346477907, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 8.559775419009384, + "end_to_end_tps": 29.907326707600312, + "timing_valid": true, + "per_stream_tps": [ + 30.6 + ], + "ttft_s": [ + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 133188.201927927, + "first": 133188.417307426, + "last": 133196.761426393, + "end": 133196.761703346, + "elapsed_s": 8.559775419009384, + "chunk_offsets_s": [ + 0.2153794989862945, + 0.2925868449965492, + 0.3695540069893468, + 0.4486210379982367, + 0.5312211079872213, + 0.615820562990848, + 0.6969579370052088, + 0.7775067139882594, + 0.8574533729988616, + 0.9399415960069746, + 1.0166887629893608, + 1.0876429839991033, + 1.1512177539989352, + 1.2154197139898315, + 1.279223835997982, + 1.3430134929949418, + 1.4102690840081777, + 1.4745585189957637, + 1.5386304109997582, + 1.6032510260120034, + 1.6762873359839432, + 1.752462910983013, + 1.8265114969981369, + 1.8987646430032328, + 1.961728250986198, + 2.027974027005257, + 2.103802645986434, + 2.176786522002658, + 2.251124781992985, + 2.3254000489832833, + 2.4086925059964415, + 2.4868741059908643, + 2.551784890005365, + 2.623484493989963, + 2.6986392620019615, + 2.7640310999995563, + 2.8292579079861753, + 2.898444060003385, + 2.9743730629852507, + 3.0486368220008444, + 3.122930854995502, + 3.201842270005727, + 3.282366400002502, + 3.3482776650052983, + 3.420514791010646, + 3.485340240993537, + 3.5502658860059455, + 3.617010365007445, + 3.6931682220019866, + 3.7697896309837233, + 3.8440280749928206, + 3.90809778298717, + 3.970310670993058, + 4.033298931986792, + 4.096979889989598, + 4.1615585279942024, + 4.227297720994102, + 4.292795112007298, + 4.358005964983022, + 4.424445887008915, + 4.488518394995481, + 4.552190215996234, + 4.615890038985526, + 4.679616531997453, + 4.744677040987881, + 4.809397729986813, + 4.872979834006401, + 4.937563509010943, + 5.004563961003441, + 5.079571740992833, + 5.156856404006248, + 5.235692674003076, + 5.314796971011674, + 5.403502237983048, + 5.4843239879992325, + 5.567484014987713, + 5.649447292991681, + 5.739495485002408, + 5.822848570998758, + 5.892203664989211, + 5.956092689011712, + 6.019482784002321, + 6.082287464989349, + 6.146829696983332, + 6.211023484996986, + 6.2755713970109355, + 6.341782408999279, + 6.412857964984141, + 6.483287960989401, + 6.548148220987059, + 6.612658130994532, + 6.6784971749875695, + 6.7515927529893816, + 6.823136755003361, + 6.8980028920050245, + 6.9699059469858184, + 7.033222261001356, + 7.097748442000011, + 7.161813949991483, + 7.226581132010324, + 7.2927684919850435, + 7.367723640985787, + 7.443707050988451, + 7.516518393997103, + 7.589985569007695, + 7.665410476998659, + 7.748837608000031, + 7.832832710992079, + 7.922063489997527, + 8.003432819008594, + 8.085243028006516, + 8.167780397983734, + 8.251433377008652, + 8.337935866991756, + 8.422684165008832, + 8.498945517989341, + 8.559498465998331 + ], + "chunk_gaps_ms": { + "count": 116, + "p50": 71.80132948269602, + "p95": 84.59945500362664, + "max": 90.04819201072678 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 43.5, + "per_stream_tps": 22.1, + "ttft_s_max": 0.51, + "timing_valid": true, + "end_to_end_tps": 42.84285383649368, + "makespan_s": 11.95065113902092, + "steady_tps": 44.4, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 43.46046195412692, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 11.95065113902092, + "end_to_end_tps": 42.84285383649368, + "timing_valid": true, + "steady_tps": 44.40051750785169, + "per_stream_tps": [ + 22.3, + 22.0 + ], + "ttft_s": [ + 0.51, + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 133196.763187512, + "first": 133197.269052006, + "last": 133208.713319704, + "end": 133208.713647988, + "elapsed_s": 11.950460476014996, + "chunk_offsets_s": [ + 0.5058644940145314, + 0.6141218900156673, + 0.7262060639914125, + 0.8357434589997865, + 0.947165880992543, + 1.0646062309970148, + 1.1821429159899708, + 1.295252370997332, + 1.411543635011185, + 1.5281061260029674, + 1.641794144990854, + 1.7585376820061356, + 1.8734013120119926, + 1.994492444995558, + 2.115996022010222, + 2.2379513409978244, + 2.3593968029890675, + 2.480562254000688, + 2.5981230800098274, + 2.7179569310101215, + 2.8367070959939156, + 2.9597854480089154, + 3.0784579119936097, + 3.202800113009289, + 3.319628411991289, + 3.4365004250139464, + 3.5540157399955206, + 3.6700907170015853, + 3.7844899210031144, + 3.9102057540148962, + 4.034384541009786, + 4.1586034370120615, + 4.280273873009719, + 4.4016512900125235, + 4.518306923011551, + 4.638025449006818, + 4.758264361997135, + 4.87550664000446, + 4.9941726319957525, + 5.110916184989037, + 5.231795597006567, + 5.352901623991784, + 5.4700437810097355, + 5.591360120015452, + 5.705868552002357, + 5.825379785994301, + 5.950550389999989, + 6.068497439991916, + 6.189901825011475, + 6.309345899993787, + 6.429403646005085, + 6.544662611995591, + 6.662680907989852, + 6.7801480810157955, + 6.891346741002053, + 7.0020985059964005, + 7.119275009987177, + 7.241446441999869, + 7.364835406013299, + 7.491978423000546, + 7.61369035099051, + 7.734750511997845, + 7.845021454995731, + 7.968774126988137, + 8.089564564987086, + 8.210399080999196, + 8.331105143995956, + 8.444802220998099, + 8.564214479003567, + 8.680721448006807, + 8.799793548998423, + 8.91927014500834, + 9.03842664900003, + 9.15014518800308, + 9.260310151003068, + 9.377652618015418, + 9.495272086991463, + 9.609516396012623, + 9.726501218014164, + 9.837850751006044, + 9.956744569994044, + 10.074399862991413, + 10.193033427000046, + 10.311660687992116, + 10.430204008996952, + 10.54489266499877, + 10.662174486002186, + 10.772487706009997, + 10.89468939800281, + 11.01429776501027, + 11.132973967003636, + 11.246355612995103, + 11.365756079991115, + 11.481478992995108, + 11.59772218001308, + 11.712611430994002, + 11.813122043007752, + 11.89032910700189, + 11.95013219199609 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 117.80117149464786, + "p95": 124.21889600227587, + "max": 127.14301698724739 + } + }, + { + "start": 133196.762996849, + "first": 133196.978515768, + "last": 133208.576142557, + "end": 133208.576334548, + "elapsed_s": 11.813337699015392, + "chunk_offsets_s": [ + 0.2155189189943485, + 0.29196405099355616, + 0.6139392759941984, + 0.7265321400191169, + 0.8361412810045294, + 0.9475059730175417, + 1.064474834012799, + 1.1822130790096708, + 1.2950877730036154, + 1.411371965019498, + 1.5279120710038114, + 1.641515191004146, + 1.7583843010070268, + 1.8732046649965923, + 1.9941936660034116, + 2.115813120006351, + 2.237664771004347, + 2.3591797230183147, + 2.4804244410188403, + 2.5979822110093664, + 2.7176958180207293, + 2.836772809998365, + 2.959643363021314, + 3.0781367490126286, + 3.2024633010150865, + 3.3195456890098285, + 3.4363327710016165, + 3.5538557190157007, + 3.6699041500105523, + 3.7843825250165537, + 3.909883854998043, + 4.034050960995955, + 4.1586678070016205, + 4.280096603004495, + 4.401522374013439, + 4.517971983004827, + 4.637898340995889, + 4.758125973021379, + 4.875371835019905, + 4.993848493002588, + 5.110752723994665, + 5.23160770200775, + 5.352510490018176, + 5.469693944993196, + 5.590986811002949, + 5.705674689001171, + 5.825276054005371, + 5.950363536016084, + 6.068292537005618, + 6.18972050701268, + 6.309236905013677, + 6.429248344007647, + 6.5444771490001585, + 6.662512437993428, + 6.779976203019032, + 6.891229201020906, + 7.0018259689968545, + 7.1189541820203885, + 7.241124063002644, + 7.3646807440090925, + 7.491654636018211, + 7.613516825018451, + 7.7346051140048075, + 7.844858665019274, + 7.968448996020015, + 8.08932121199905, + 8.210115071007749, + 8.33096605099854, + 8.44446299399715, + 8.564031351997983, + 8.680563906993484, + 8.799785789015004, + 8.919119996018708, + 9.038274691993138, + 9.149819320009556, + 9.259932874003425, + 9.37738238502061, + 9.495098737010267, + 9.609367991011823, + 9.726279321999755, + 9.83820817200467, + 9.956495841004653, + 10.074217520013917, + 10.192731609015027, + 10.311475305003114, + 10.430064852000214, + 10.544649232993834, + 10.661974974995246, + 10.772284467006102, + 10.894511374994181, + 11.014208274020348, + 11.132720887020696, + 11.246195047016954, + 11.365571386006195, + 11.481836270017084, + 11.597371800016845, + 11.712499746994581, + 11.813145708001684 + ], + "chunk_gaps_ms": { + "count": 97, + "p50": 118.0352889932692, + "p95": 124.61684600566514, + "max": 321.97522500064224 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 44.0, + "per_stream_tps": 17.6, + "ttft_s_max": 14.91, + "timing_valid": true, + "end_to_end_tps": 43.80047501346692, + "makespan_s": 23.37874189001741, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 44.035257250589964, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 23.37874189001741, + "end_to_end_tps": 43.80047501346692, + "timing_valid": true, + "per_stream_tps": [ + 17.6, + 30.1, + 17.4, + 17.6 + ], + "ttft_s": [ + 0.83, + 14.91, + 0.5, + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 133208.71538117, + "first": 133209.54049765, + "last": 133223.995133447, + "end": 133223.995220379, + "elapsed_s": 15.27983920898987, + "chunk_offsets_s": [ + 0.8251164800021797, + 0.9630215840006713, + 1.1043418799818028, + 1.239768170984462, + 1.3787243469851092, + 1.5201447859872133, + 1.6659154439985286, + 1.8062026749830693, + 1.9434312699886505, + 2.0843401069869287, + 2.2240576969925314, + 2.363702819973696, + 2.531990429997677, + 2.7143830679997336, + 2.9045686539902817, + 3.068922284001019, + 3.2208207009825855, + 3.3675788219843525, + 3.513858350983355, + 3.6608063189778477, + 3.807414212002186, + 3.9538657949888147, + 4.104630329995416, + 4.258315106999362, + 4.401064953999594, + 4.542651702999137, + 4.686778029979905, + 4.8281265949772205, + 4.975317670992808, + 5.126439411978936, + 5.28015464899363, + 5.434017474995926, + 5.585943304002285, + 5.7283608299912885, + 5.877166036982089, + 6.024433797982056, + 6.170277904981049, + 6.311042646993883, + 6.452954245993169, + 6.6009709399950225, + 6.7474560429982375, + 6.894094672985375, + 7.041505728993798, + 7.18636771599995, + 7.326925773988478, + 7.478180655976757, + 7.625376355979824, + 7.7699054009863175, + 7.916825730993878, + 8.065947001974564, + 8.202920144976815, + 8.34682129099383, + 8.486402764974628, + 8.626513905997854, + 8.763174044986954, + 8.899389864993282, + 9.041379886999493, + 9.187853872979758, + 9.337038734985981, + 9.491788272978738, + 9.647311615000945, + 9.787232039991068, + 9.937213281984441, + 10.088473417999921, + 10.238846812979318, + 10.395491707982728, + 10.53795786498813, + 10.682231893995777, + 10.826934960990911, + 10.9700869829976, + 11.11850685198442, + 11.26296939398162, + 11.40516043198295, + 11.542106677981792, + 11.683371203980641, + 11.829270883987192, + 11.972301264002454, + 12.114284053997835, + 12.25028755798121, + 12.385985851986334, + 12.528560417995322, + 12.675114371988457, + 12.82157550699776, + 12.967349721002392, + 13.112340483989101, + 13.251874410983874, + 13.383746052975766, + 13.535333729989361, + 13.684445975988638, + 13.834456413984299, + 13.984139815991512, + 14.130632448999677, + 14.27801213398925, + 14.420734662999166, + 14.564807238988578, + 14.6950267549837, + 15.047589297988452, + 15.177357979002409, + 15.279752276983345 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 145.3807104990119, + "p95": 164.3536300107371, + "max": 352.5625430047512 + } + }, + { + "start": 133208.715864131, + "first": 133223.6231498, + "last": 133232.093630914, + "end": 133232.093783479, + "elapsed_s": 23.377919348014984, + "chunk_offsets_s": [ + 14.907285669003613, + 15.047383739001816, + 15.177601683011744, + 15.27922435500659, + 15.358638042001985, + 15.423704043991165, + 15.497848780010827, + 15.571506778011099, + 15.643639011017513, + 15.714037663012277, + 15.784812664991478, + 15.848499387007905, + 15.912447381997481, + 15.9768584970152, + 16.04154767, + 16.105876061017625, + 16.169175343005918, + 16.23562663901248, + 16.299840288993437, + 16.36353262499324, + 16.427135278005153, + 16.49250317600672, + 16.569133003009483, + 16.64292698801728, + 16.71450203101267, + 16.77710618401761, + 16.842484738997882, + 16.918674198997905, + 16.991158249991713, + 17.066765977011528, + 17.140796848019818, + 17.223544598004082, + 17.301570456009358, + 17.36552406201372, + 17.437608767999336, + 17.513109946012264, + 17.578726122010266, + 17.643697763996897, + 17.712298365018796, + 17.78811272099847, + 17.861751768999966, + 17.936652845994104, + 18.015868743998, + 18.09643883400713, + 18.162113571015652, + 18.2332451690163, + 18.298001585004386, + 18.361999703018228, + 18.429206603992498, + 18.505828569992445, + 18.582175710005686, + 18.65633922201232, + 18.719474990008166, + 18.781679333013017, + 18.84397939100745, + 18.908191037015058, + 18.97235036001075, + 19.038866726012202, + 19.1040102429979, + 19.16887216400937, + 19.234386491996702, + 19.298734508018242, + 19.36170494000544, + 19.42563070100732, + 19.489091694005765, + 19.55403296201257, + 19.618673596007284, + 19.682169517996954, + 19.746180642017862, + 19.81290992401773, + 19.887252112006536, + 19.96524660201976, + 20.04447843201342, + 20.123309079004684, + 20.2105056779983, + 20.291284295002697, + 20.373629286012147, + 20.456933781999396, + 20.548613257997204, + 20.632451389014022, + 20.701682787999744, + 20.76613536500372, + 20.828614480997203, + 20.892020686005708, + 20.957092468015617, + 21.021784334006952, + 21.085769054014236, + 21.15168640000047, + 21.221679167007096, + 21.29195199199603, + 21.35561156301992, + 21.42039525299333, + 21.486101456015604, + 21.559488005004823, + 21.630840355006512, + 21.705083414010005, + 21.776546600012807, + 21.83924463301082, + 21.904473186994437, + 21.968289548007306, + 22.033596535999095, + 22.09931491399766, + 22.173923455004115, + 22.249101471999893, + 22.32098631799454, + 22.394123336009216, + 22.469835210009478, + 22.55558141801157, + 22.6503010239976, + 22.74181850400055, + 22.82318211099482, + 22.90520439701504, + 22.987619192019338, + 23.071789617009927, + 23.15823269399698, + 23.24198671701015, + 23.31807421799749, + 23.37776678299997 + ], + "chunk_gaps_ms": { + "count": 117, + "p50": 70.27282498893328, + "p95": 91.51748000294901, + "max": 140.09806999820285 + } + }, + { + "start": 133208.716021576, + "first": 133209.220933969, + "last": 133223.892931716, + "end": 133223.89306108, + "elapsed_s": 15.177039503993, + "chunk_offsets_s": [ + 0.504912392992992, + 0.6120158270059619, + 0.9625632000097539, + 1.104009805014357, + 1.2394737930153497, + 1.3783508299966343, + 1.5197928550187498, + 1.6655857849982567, + 1.8056858259951696, + 1.9430606329988223, + 2.0839563340123277, + 2.2235832330188714, + 2.3632484849949833, + 2.5312850460177287, + 2.713672547019087, + 2.9037053440115415, + 3.0683331440086477, + 3.220335004007211, + 3.3670220829953905, + 3.5133250530052464, + 3.6603293429943733, + 3.8068788179953117, + 3.95340774700162, + 4.104116280010203, + 4.257808610011125, + 4.400470869004494, + 4.542045378009789, + 4.686087606998626, + 4.8274204110202845, + 4.974496603012085, + 5.125622999999905, + 5.279343005007831, + 5.433157541003311, + 5.585267888993258, + 5.7280451389960945, + 5.876723638008116, + 6.02400242301519, + 6.169908804993611, + 6.3101017830194905, + 6.452347553014988, + 6.600611455010949, + 6.7467710280034225, + 6.893401865003398, + 7.040563663991634, + 7.18585656301002, + 7.326126241998281, + 7.477697024005465, + 7.624686668015784, + 7.769225841999287, + 7.916011174995219, + 8.065004330012016, + 8.202039971016347, + 8.346141668007476, + 8.48572472599335, + 8.625835818995256, + 8.762496117997216, + 8.898785380006302, + 9.04070183201111, + 9.187258316000225, + 9.336444682994625, + 9.491110378003214, + 9.646628439018968, + 9.786440460011363, + 9.936524251010269, + 10.087789954006439, + 10.238353035994805, + 10.394800420006504, + 10.537371876998805, + 10.681436298007611, + 10.826334365003277, + 10.969302715995582, + 11.117824123997707, + 11.26241969500552, + 11.40456518001156, + 11.541418687003898, + 11.682883747009328, + 11.828583980997792, + 11.97161279199645, + 12.113684178009862, + 12.24960191900027, + 12.38546151400078, + 12.528073808993213, + 12.674301352002658, + 12.820879242994124, + 12.96667195399641, + 13.111629243008792, + 13.25115220999578, + 13.383347991009941, + 13.53459642501548, + 13.683604282996384, + 13.834052991995122, + 13.983462864998728, + 14.130226642999332, + 14.277616856008535, + 14.420344985992415, + 14.56440353000653, + 14.694956929015461, + 15.047133778018178, + 15.176910139998654 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 145.7464724953752, + "p95": 168.03656102274545, + "max": 352.17684900271706 + } + }, + { + "start": 133208.715041589, + "first": 133208.930373477, + "last": 133223.41033301, + "end": 133223.410683871, + "elapsed_s": 14.695642282022163, + "chunk_offsets_s": [ + 0.2153318880009465, + 0.29225191500154324, + 0.6127913430100307, + 0.9631342609936837, + 1.104767784010619, + 1.2402177080221009, + 1.379409540008055, + 1.5205881310102995, + 1.6663228830148, + 1.806754360004561, + 1.9438611740188207, + 2.084769706998486, + 2.2245012820058037, + 2.364287514006719, + 2.5324967369961087, + 2.7144550710218027, + 2.9047826460155193, + 3.0693921580095775, + 3.221255053009372, + 3.36806244001491, + 3.514365954004461, + 3.661373620998347, + 3.807923062995542, + 3.9543207400129177, + 4.105037785018794, + 4.258725938998396, + 4.401537451019976, + 4.542829006008105, + 4.6872358079999685, + 4.828584293019958, + 4.9757590320077725, + 5.126849652006058, + 5.280561784020392, + 5.43442241800949, + 5.586340998997912, + 5.72876980502042, + 5.877778587018838, + 6.025054525001906, + 6.17069292900851, + 6.311153660004493, + 6.4533859259972814, + 6.601517520000925, + 6.747983278997708, + 6.894532081001671, + 7.041785212000832, + 7.18691565701738, + 7.327169238997158, + 7.478584591008257, + 7.625799572007963, + 7.770333305001259, + 7.91706097201677, + 8.066213876998518, + 8.203419124009088, + 8.347238746995572, + 8.486805149004795, + 8.626916320994496, + 8.76359554901137, + 8.899831881019054, + 9.041780335013755, + 9.188297521002823, + 9.33749576000264, + 9.492191761004506, + 9.647719054017216, + 9.787490450020414, + 9.937614401016617, + 10.08887898499961, + 10.239265948999673, + 10.39589012300712, + 10.538226403994486, + 10.682487040001433, + 10.827197356004035, + 10.970356017001905, + 11.118915202998323, + 11.263236188009614, + 11.405602481012465, + 11.5425114610116, + 11.683777651021956, + 11.82967390702106, + 11.97270057501737, + 12.114730439003324, + 12.250695877999533, + 12.38617158701527, + 12.52866669499781, + 12.675352813996142, + 12.822054085001582, + 12.967407165007899, + 13.112789028993575, + 13.252286410017405, + 13.384388013015268, + 13.53577901099925, + 13.684676528995624, + 13.83509400501498, + 13.98421063600108, + 14.131267337012105, + 14.278659517003689, + 14.421389151015319, + 14.565448303008452, + 14.695291420997819 + ], + "chunk_gaps_ms": { + "count": 97, + "p50": 145.6384040066041, + "p95": 168.20922298938967, + "max": 350.342917983653 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + } + ] +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/rows-without-bodies.jsonl b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/rows-without-bodies.jsonl new file mode 100644 index 0000000000..9e0b766995 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/rows-without-bodies.jsonl @@ -0,0 +1,44 @@ +{"i": 0, "conv": "S1", "turn": 1, "arm": "drafted", "group": "", "history": 7757, "allowed": [0], "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 6.157516856, "prefill": 6.156042772, "wall": 8.238416171021527, "finish": "tool_calls"}, "ok": true, "body_sha256": "c89863023849e1c523580f91d6e3e557e187ee1345ec141afe8843f12f6a48bb", "body_bytes": 29036} +{"i": 1, "conv": "S1", "turn": 1, "arm": "plain", "group": "", "history": 7757, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 6144, "prompt": 7759, "tokens": 61, "ttft": 1.290912251, "prefill": 1.288934247, "wall": 4.820883380976738, "finish": "tool_calls"}, "ok": true, "body_sha256": "990b29e03db5234f33dad6e1fa5929871234fa91c7f57ae49f27a1167d79423a", "body_bytes": 29050} +{"i": 2, "conv": "S1", "turn": 2, "arm": "drafted", "group": "", "history": 15548, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 6144, "prompt": 15550, "tokens": 44, "ttft": 7.282022014, "prefill": 7.279372804, "wall": 8.362898021994624, "finish": "tool_calls"}, "ok": true, "body_sha256": "7a3e4d8b9637b5ca4ea6afdc01f758a9bc5a01d94ea0ebd7c71d3dda60b0ff97", "body_bytes": 52453} +{"i": 3, "conv": "S1", "turn": 2, "arm": "plain", "group": "", "history": 15548, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 14336, "prompt": 15550, "tokens": 44, "ttft": 1.036756408, "prefill": 1.033653247, "wall": 3.5129178420174867, "finish": "tool_calls"}, "ok": true, "body_sha256": "5c31ca4b3301ec96d376c3f491676dc749bfbca47bf592d4b3893ef6824a59c5", "body_bytes": 52467} +{"i": 4, "conv": "S1", "turn": 3, "arm": "drafted", "group": "", "history": 19458, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 14336, "prompt": 19460, "tokens": 80, "ttft": 4.186702153, "prefill": 4.183549566, "wall": 6.867797106999205, "finish": "tool_calls"}, "ok": true, "body_sha256": "a756e173f60df71acd293ef6869ea8d453d67f9ccdfda1142ac895a157a924dc", "body_bytes": 64230} +{"i": 5, "conv": "S1", "turn": 3, "arm": "plain", "group": "", "history": 19458, "allowed": {"min": 11266, "max": 19458}, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 18432, "prompt": 19460, "tokens": 80, "ttft": 0.945698814, "prefill": 0.941980865, "wall": 5.434095955017256, "finish": "tool_calls"}, "ok": true, "body_sha256": "01d3a4cb3a1570ccb4fb0e8c7be26a07dd91aeb4ac298a877c23709f8cb2b601", "body_bytes": 64244} +{"i": 6, "conv": "S1", "turn": 4, "arm": "drafted", "group": "", "history": 21872, "allowed": {"min": 11266, "max": 19458}, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 18432, "prompt": 21874, "tokens": 148, "ttft": 2.780080465, "prefill": 2.776131949, "wall": 8.128511048009386, "finish": "tool_calls"}, "ok": true, "body_sha256": "5d411177eba06b9539ec523d78deb9f5ef7428da4fb46f460764e7318c8b2d1e", "body_bytes": 76137} +{"i": 7, "conv": "S1", "turn": 4, "arm": "plain", "group": "", "history": 21872, "allowed": {"min": 13680, "max": 21872}, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 20480, "prompt": 21874, "tokens": 148, "ttft": 1.13503555, "prefill": 1.130873795, "wall": 9.581383315991843, "finish": "tool_calls"}, "ok": true, "body_sha256": "81aefe61161163c7ba87b382e361855545dfebcd1954cabe3663a473ee939c70", "body_bytes": 76151} +{"i": 8, "conv": "S1", "turn": 5, "arm": "drafted", "group": "", "history": 26784, "allowed": {"min": 13680, "max": 21872}, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 20480, "prompt": 26786, "tokens": 160, "ttft": 5.169807572, "prefill": 5.164900211, "wall": 10.528310474008322, "finish": "length"}, "ok": true, "body_sha256": "6d2060ef5aa31bde5037a6fcd61fd000391b349dcf74f9d3cc174c096634eec4", "body_bytes": 99864} +{"i": 9, "conv": "S1", "turn": 5, "arm": "plain", "group": "", "history": 26784, "allowed": {"min": 18592, "max": 26784}, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 26624, "prompt": 26786, "tokens": 160, "ttft": 0.439872947, "prefill": 0.434545895, "wall": 9.5886274749937, "finish": "length"}, "ok": true, "body_sha256": "013b97c243613ff9a41bad43029e25cea8d2e49869e96f4461171ef5b93c00e5", "body_bytes": 99878} +{"i": 10, "conv": "S1", "turn": 6, "arm": "drafted", "group": "", "history": 29291, "allowed": {"min": 18592, "max": 26784}, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 26624, "prompt": 29293, "tokens": 160, "ttft": 2.454746111, "prefill": 2.448383537, "wall": 7.8996382580080535, "finish": "length"}, "ok": true, "body_sha256": "47b51d8ec5fbe4444442414990953c0b2287468db316e3300939474a0c99c4a7", "body_bytes": 112228} +{"i": 11, "conv": "S1", "turn": 6, "arm": "plain", "group": "", "history": 29291, "allowed": {"min": 21099, "max": 29291}, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 28672, "prompt": 29293, "tokens": 160, "ttft": 0.712718221, "prefill": 0.706534567, "wall": 9.608176273992285, "finish": "length"}, "ok": true, "body_sha256": "dfc78892219317a4b355880f7a13a57fc20f4a2c37b5b8e07a6b72bf2ca70793", "body_bytes": 112242} +{"i": 12, "conv": "S1", "turn": 7, "arm": "drafted", "group": "", "history": 32214, "allowed": {"min": 21099, "max": 29291}, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 28672, "prompt": 32216, "tokens": 160, "ttft": 2.819824569, "prefill": 2.81277281, "wall": 8.423287524987245, "finish": "length"}, "ok": true, "body_sha256": "0495d1577642e8b8e54a19be26f9869af4af0cf94b87203ed90d60c035c934d6", "body_bytes": 124515} +{"i": 13, "conv": "S1", "turn": 7, "arm": "plain", "group": "", "history": 32214, "allowed": {"min": 24022, "max": 32214}, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 30720, "prompt": 32216, "tokens": 160, "ttft": 1.193667897, "prefill": 1.186245632, "wall": 10.22392868399038, "finish": "length"}, "ok": true, "body_sha256": "1eef29ee599bde1dbac60e535cd57067f589c09993e2dfbff63f98aca6067b19", "body_bytes": 124529} +{"i": 14, "conv": "S1", "turn": 8, "arm": "drafted", "group": "", "history": 35140, "allowed": {"min": 24022, "max": 32214}, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 30720, "prompt": 35142, "tokens": 160, "ttft": 3.725857932, "prefill": 3.71787642, "wall": 9.284945380000863, "finish": "length"}, "ok": true, "body_sha256": "6d9724f59da3d9dee5c451e92c76ef875b1327481b792d9e3304f2044e9537d3", "body_bytes": 136847} +{"i": 15, "conv": "S1", "turn": 8, "arm": "plain", "group": "", "history": 35140, "allowed": {"min": 26948, "max": 35140}, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 34816, "prompt": 35142, "tokens": 160, "ttft": 0.559165536, "prefill": 0.551012084, "wall": 9.479312902025413, "finish": "length"}, "ok": true, "body_sha256": "8d22bc32d95c5ae33cfe3ba99e80810ac1fb1deeca3fa017c3519fc0e53a8901", "body_bytes": 136861} +{"i": 16, "conv": "S1", "turn": 9, "arm": "drafted", "group": "", "history": 37858, "allowed": {"min": 26948, "max": 35140}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 34816, "prompt": 37860, "tokens": 160, "ttft": 2.556917896, "prefill": 2.547223152, "wall": 5.273598886997206, "finish": "length"}, "ok": true, "body_sha256": "3002fe8fa32e141eef9da7426519b3b6b99548ba68efdaaa2138e48d2a04b4c5", "body_bytes": 149173} +{"i": 17, "conv": "S1", "turn": 9, "arm": "plain", "group": "", "history": 37858, "allowed": {"min": 29666, "max": 37858}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 36864, "prompt": 37860, "tokens": 160, "ttft": 0.937733743, "prefill": 0.928300144, "wall": 9.89457444701111, "finish": "length"}, "ok": true, "body_sha256": "631d332bc4019c84c08c45b5a6f79f66b15b3ce24ab620c76c7e8446dfc6269d", "body_bytes": 149187} +{"i": 18, "conv": "X", "turn": 1, "arm": "drafted", "group": "", "history": 2113, "allowed": [0], "got": {"sha": "9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.766129987, "prefill": 1.765159798, "wall": 7.000440812000306, "finish": "length"}, "ok": true, "body_sha256": "ea7f1fb5d610d52f549d66c8d5bac3876b0c32a1f369f7230266491a9a674737", "body_bytes": 9524} +{"i": 19, "conv": "A", "turn": 1, "arm": "drafted", "group": "", "history": 7410, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 6144, "prompt": 7412, "tokens": 95, "ttft": 1.090777578, "prefill": 1.089291293, "wall": 3.7861892000073567, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a153933bfbacdc64dd54d832df32d21139497927a2527fd60217851614cf603", "body_bytes": 28006} +{"i": 20, "conv": "A", "turn": 1, "arm": "plain", "group": "", "history": 7410, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 6144, "prompt": 7412, "tokens": 95, "ttft": 1.047073092, "prefill": 1.04509441, "wall": 6.318344567989698, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a6c1cda2af13b7752b30514bbb91dc76381167e03b207540e31a0f810a462b3", "body_bytes": 28020} +{"i": 21, "conv": "B", "turn": 1, "arm": "drafted", "group": "", "history": 7472, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 6144, "prompt": 7474, "tokens": 72, "ttft": 1.12476692, "prefill": 1.122966355, "wall": 3.0317041810194496, "finish": "tool_calls"}, "ok": true, "body_sha256": "6685678d8326dff3c264f306810b45e76303da9dbc2ff27f9b4eec69f8a6384c", "body_bytes": 27976} +{"i": 22, "conv": "B", "turn": 1, "arm": "plain", "group": "", "history": 7472, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 6144, "prompt": 7474, "tokens": 72, "ttft": 1.082601777, "prefill": 1.080666952, "wall": 5.078056944999844, "finish": "tool_calls"}, "ok": true, "body_sha256": "ac1e48a68c55011216bc0008268ca26a6a788c311372fed6bc6dc9ed62769bf4", "body_bytes": 27990} +{"i": 23, "conv": "A", "turn": 2, "arm": "drafted", "group": "", "history": 12621, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 6144, "prompt": 12623, "tokens": 120, "ttft": 5.300232163, "prefill": 5.297334573, "wall": 9.26117299098405, "finish": "tool_calls"}, "ok": true, "body_sha256": "cf48f0ec4fce5224b66a822f28982358320db7d6c39edb0335add78421509b56", "body_bytes": 51797} +{"i": 24, "conv": "A", "turn": 2, "arm": "plain", "group": "", "history": 12621, "allowed": {"min": 4429, "max": 12621}, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 12288, "prompt": 12623, "tokens": 120, "ttft": 0.549162557, "prefill": 0.546118804, "wall": 7.171731492009712, "finish": "tool_calls"}, "ok": true, "body_sha256": "31e0672224ef3feed1e23485b13d642ad936dae1dd8795ea68e4ce69011155af", "body_bytes": 51811} +{"i": 25, "conv": "B", "turn": 2, "arm": "drafted", "group": "", "history": 12718, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 6144, "prompt": 12720, "tokens": 79, "ttft": 5.233805756, "prefill": 5.230623935, "wall": 8.009808183007408, "finish": "tool_calls"}, "ok": true, "body_sha256": "3fce97c1e3f24e1d31d0840d11744c9e9139e23e27b5e0a0a39a130c6ff309f0", "body_bytes": 51398} +{"i": 26, "conv": "B", "turn": 2, "arm": "plain", "group": "", "history": 12718, "allowed": {"min": 4526, "max": 12718}, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 12288, "prompt": 12720, "tokens": 79, "ttft": 0.598439053, "prefill": 0.595515529, "wall": 4.966791229991941, "finish": "tool_calls"}, "ok": true, "body_sha256": "6bbff3581e361c0961bd03b55693879ab31f24fd0f80145da85baf2f59908f3d", "body_bytes": 51412} +{"i": 27, "conv": "A", "turn": 3, "arm": "drafted", "group": "", "history": 18061, "allowed": {"min": 4429, "max": 12621}, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 12288, "prompt": 18063, "tokens": 160, "ttft": 4.45241867, "prefill": 4.4489002, "wall": 10.078571847989224, "finish": "length"}, "ok": true, "body_sha256": "0d61221b675d580be9a5f5df34118e27a0916746c70b9d72627d5e93b96f688b", "body_bytes": 75423} +{"i": 28, "conv": "A", "turn": 3, "arm": "plain", "group": "", "history": 18061, "allowed": {"min": 9869, "max": 18061}, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 16384, "prompt": 18063, "tokens": 160, "ttft": 1.29301301, "prefill": 1.288565859, "wall": 10.211812923982507, "finish": "length"}, "ok": true, "body_sha256": "9fa827e5a0cd9e17ef37eaea6e8a96090dc222cecccc5e9fda783cd35b8a4761", "body_bytes": 75437} +{"i": 29, "conv": "B", "turn": 3, "arm": "drafted", "group": "", "history": 15030, "allowed": {"min": 4526, "max": 12718}, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 12288, "prompt": 15032, "tokens": 85, "ttft": 2.323877066, "prefill": 2.32005619, "wall": 5.259558468009345, "finish": "tool_calls"}, "ok": true, "body_sha256": "ac4150ca1f9aeff7a560a39c005e23dd9538d1438ff803a39a4f2beb08d29ef2", "body_bytes": 63313} +{"i": 30, "conv": "B", "turn": 3, "arm": "plain", "group": "", "history": 15030, "allowed": {"min": 6838, "max": 15030}, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 14336, "prompt": 15032, "tokens": 85, "ttft": 0.740508407, "prefill": 0.736717117, "wall": 5.45007304099272, "finish": "tool_calls"}, "ok": true, "body_sha256": "dbc6fc2aa0f94173808cc0fb4aa37314f7f2b934557689c208758f9807a4714c", "body_bytes": 63327} +{"i": 31, "conv": "E", "turn": 1, "arm": "drafted", "group": "", "history": 40607, "allowed": {"min": 0, "max": 6611}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 6144, "prompt": 40609, "tokens": 160, "ttft": 26.393823947, "prefill": 26.383778866, "wall": 28.756016230996465, "finish": "length"}, "ok": true, "body_sha256": "07f4b8828c734cd3271a28b20a6eef3bfa6cbb4c9112dfed4fbe1f521bab8aad", "body_bytes": 161550} +{"i": 32, "conv": "E", "turn": 1, "arm": "plain", "group": "", "history": 40607, "allowed": {"min": 32415, "max": 40607}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 38912, "prompt": 40609, "tokens": 160, "ttft": 1.342860812, "prefill": 1.332853689, "wall": 10.311403473024257, "finish": "length"}, "ok": true, "body_sha256": "21d21e3744b095d4d7237d112d9942de384ae5a96432e6009bc4cf24ef0a834c", "body_bytes": 161564} +{"i": 33, "conv": "T", "turn": 1, "arm": "drafted", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.137407277, "prefill": 1.13651738, "wall": 6.18965090601705, "finish": "stop"}, "ok": true, "body_sha256": "45cd3d7f3f990fbf5badf7e4179006aece0dac6ff89e43c1a049c9c47e65c0af", "body_bytes": 6672} +{"i": 34, "conv": "T", "turn": 1, "arm": "plain", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.097601446, "prefill": 1.096545978, "wall": 9.602748875011457, "finish": "stop"}, "ok": true, "body_sha256": "17133648c09b8f37c9e3f3f57cf38d2952a766d4633876fb61d87b50e01f6e13", "body_bytes": 6686} +{"i": 35, "conv": "T", "turn": 2, "arm": "drafted", "group": "", "history": 4205, "allowed": [0], "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.590276714, "prefill": 3.588037263, "wall": 3.899093426996842, "finish": "stop"}, "ok": true, "body_sha256": "697cbd599a80dd68e03e110992a6355933628f9f0b27307c22cb85d2a1282482", "body_bytes": 19059} +{"i": 36, "conv": "T", "turn": 2, "arm": "plain", "group": "", "history": 4205, "allowed": {"min": 1, "max": 4205}, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 4096, "prompt": 4208, "tokens": 12, "ttft": 0.372334877, "prefill": 0.370168498, "wall": 0.9990105210163165, "finish": "stop"}, "ok": true, "body_sha256": "a477a66d50c218bb9084b3eaa0fb91a039270992aee4478d3ab1de5260bdb370", "body_bytes": 19073} +{"i": 37, "conv": "C", "turn": 1, "arm": "cancel", "group": "", "history": 0, "allowed": null, "got": {"cancelled_after": 4.0}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 38, "conv": "C", "turn": 1, "arm": "drafted", "group": "", "history": 9700, "allowed": {"min": 0, "max": 9688}, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 8.203552022, "prefill": 8.113103978, "wall": 11.052864652010612, "finish": "tool_calls"}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 39, "conv": "C", "turn": 1, "arm": "plain", "group": "", "history": 9700, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 8192, "prompt": 9702, "tokens": 82, "ttft": 1.192650934, "prefill": 1.190129379, "wall": 5.753331376006827, "finish": "tool_calls"}, "ok": true, "body_sha256": "a3d8ec39bf405406cdbde28193f74e4e2ac61b2857b32a308870da9fadd5869f", "body_bytes": 43551} +{"i": 40, "conv": "C", "turn": 2, "arm": "drafted", "group": "", "history": 15150, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 8192, "prompt": 15152, "tokens": 160, "ttft": 5.484096965, "prefill": 5.480935361, "wall": 10.588825090992032, "finish": "length"}, "ok": true, "body_sha256": "5ea6a600bb47aae478cf97dfb331773a5cafbc7f0759bc248a1a4f5477875c20", "body_bytes": 66946} +{"i": 41, "conv": "C", "turn": 2, "arm": "plain", "group": "", "history": 15150, "allowed": {"min": 6958, "max": 15150}, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 14336, "prompt": 15152, "tokens": 160, "ttft": 0.821292604, "prefill": 0.818014134, "wall": 9.708633952977834, "finish": "length"}, "ok": true, "body_sha256": "dedfe74cd1dc47519be1b744f373ecfc9b510897d3707ac4fc6c7dbdef62c0ef", "body_bytes": 66960} +{"i": 42, "conv": "A", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 20870, "allowed": {"min": 9869, "max": 18061}, "got": {"sha": "eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c", "cached": 16384, "prompt": 20872, "tokens": 160, "ttft": 6.663550463, "prefill": 6.65914579, "wall": 13.317091307981173, "finish": "length"}, "ok": true, "body_sha256": "a710ebc28ab481bcea7479789f1d363f7c32760712a8c93d2eda75f1513abfe3", "body_bytes": 87155} +{"i": 43, "conv": "B", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 17548, "allowed": {"min": 6838, "max": 15030}, "got": {"sha": "fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8", "cached": 14336, "prompt": 17550, "tokens": 160, "ttft": 2.611913098, "prefill": 2.608250364, "wall": 13.38299127400387, "finish": "length"}, "ok": true, "body_sha256": "1c720033c617f84ef3bf70b2ad558f1fa564f96150f8417e04cd645c35b26ffe", "body_bytes": 75226} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/server-rank0.log new file mode 100644 index 0000000000..24b3ff0b78 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/server-rank0.log @@ -0,0 +1,557 @@ +[kvp-serve] 2026-10-22T14:55:19+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native d82043de5d8b1f04 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 4 +host-d:17192:17192 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:17192:17192 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:17192:17192 [0] NCCL INFO cudaDriverVersion 13030 +host-d:17192:17192 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:17192:17192 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:17192:17192 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:17192:17192 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:17192:17192 [0] NCCL INFO Count not open libibverbs.so +host-d:17192:17192 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:17192:17192 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:17192:17192 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:17192:17192 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:17192:17192 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x3e1717e0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:17192:17192 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:17192:17192 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:17192:17192 [0] NCCL INFO Initialized NET plugin IB +host-d:17192:17192 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:17192:17192 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:17192:17192 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:17192:17192 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:17192:17192 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:17192:17192 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:17192:17192 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:17192:17192 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:17192:17192 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:17192:17192 [0] NCCL INFO Using network IB +host-d:17192:17192 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:17192:17192 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x3b53f7a0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:17192:17192 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:17192:17192 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:17192:17192 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:17192:17192 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:17192:17192 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:17192:17192 [0] NCCL INFO Rank 0: 1 Net devices +host-d:17192:17192 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:17192:17192 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:17192:17192 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:17192:17192 [0] NCCL INFO comm 0x3b53f7a0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:17192:17192 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:17192:17192 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:17192:17192 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:17192:17192 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:17192:17192 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:17192:17192 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:17192:17192 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:17192:17192 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:17192:17192 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:17192:17192 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:17192:17211 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-d:17192:17211 [0] NCCL INFO proxy listening socket at <52053> +host-d:17192:17212 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 4 +host-d:17192:17192 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:17192:17192 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:17192:17192 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:17192:17213 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-d:17192:17192 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:17192:17192 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:17192:17192 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:17192:17192 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:17192:17192 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe349a4010000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe349a4054000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe349a409c000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1627 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1627 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bbdba ctsFifoLkey=0x1bbdba +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1627 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1628 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1628 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1abdbb ctsFifoLkey=0x1abdbb +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1628 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe349a414e000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe349a4196000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe349a41da000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe349a4222000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1629 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1630 pkey=0 pd=0xe349a4053590 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe349a42b6000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1627 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1628 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1631 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1632 pkey=0 pd=0xe349a4053590 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1633 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1633 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5a59 ctsFifoLkey=0x1b5a59 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1633 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1634 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1635 pkey=0 pd=0xe349a4053590 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1636 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1636 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b6764 ctsFifoLkey=0x1b6764 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1636 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1637 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1638 pkey=0 pd=0xe349a4053590 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1633 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1636 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17192 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:17192:17192 [0] NCCL INFO Connected all trees +host-d:17192:17192 [0] NCCL INFO Connected binomial trees +host-d:17192:17192 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:17192:17192 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:17192:17192 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:17192:17192 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, the engine's own at a 131072-token window (1.77 GiB, in the model's), 3 more taking 504 pages of 512 rows (3.14 GiB) as they need them, 180 MiB fixed each, of 7.7 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 4.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 306.5s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-2972ae19e1cb4ceb0bf015b11ec66cde prompt=14 cached=0 thinking=False effort=none tokens=16 sha=cedaf3b614aa finish=length rounds=6 accepted=9/13 ttft=0.161 decode=0.452 elapsed=0.613 queued=0.000 tok/s=33.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-7c9125405dc801cba5cdad9bab097770 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.488 elapsed=6.633 queued=0.000 tok/s=39.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-0200548d11a5d5f7feaa996fdb1d9a80 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.145 decode=8.531 elapsed=8.677 queued=0.000 tok/s=29.9 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 4, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-7a248df880ce1aa40a6e0be2c992ee38 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.145 decode=7.276 elapsed=7.421 queued=0.000 tok/s=35.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 5, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-e2fa47a49370d50624efa9c05d409898 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=77 accepted=178/221 ttft=0.144 decode=6.119 elapsed=6.263 queued=0.000 tok/s=41.7 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-964d039be5333294841f501a3825088c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=255 accepted=0/0 ttft=0.138 decode=14.243 elapsed=14.381 queued=0.000 tok/s=17.9 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-f204e3e8f1353bd488e2a64b7cb9f57b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=255 accepted=0/0 ttft=0.137 decode=13.874 elapsed=14.011 queued=0.000 tok/s=18.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 8, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-2b4891319058583cfb659bd9a5853e61 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=255 accepted=0/0 ttft=0.137 decode=13.990 elapsed=14.127 queued=0.000 tok/s=18.2 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 9, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-e717e3a86a7d420f8334e6e5d7ff2f1d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=255 accepted=0/0 ttft=0.137 decode=13.865 elapsed=14.002 queued=0.000 tok/s=18.4 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 10, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-2b77ebd88adb230de00cb7d5436aca97 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.521 elapsed=6.667 queued=0.000 tok/s=39.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 11, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-856148be23f1c8593770e5c2fe3945cd prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=78 accepted=178/229 ttft=0.364 decode=9.460 elapsed=9.823 queued=0.218 tok/s=27.0 seed=1234 +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-4439768dd804d0af57479dda3d12b007 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=103 accepted=152/276 ttft=0.146 decode=11.396 elapsed=11.543 queued=0.000 tok/s=22.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 13, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-d53960eac234643bbae98ac3a597f23f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=78 accepted=178/229 ttft=0.363 decode=11.956 elapsed=12.319 queued=0.218 tok/s=21.3 seed=1234 +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 17, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-a4ef0ce4bd256e576860ed089af1bab8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=80 accepted=175/237 ttft=0.145 decode=12.458 elapsed=12.603 queued=0.000 tok/s=20.5 seed=1237 +[tensorfold] done cmpl-ac1e82f561f8e1a788dbba812fff3909 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=98 accepted=157/292 ttft=0.610 decode=14.389 elapsed=15.000 queued=0.218 tok/s=17.7 seed=1235 +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-0adf785c20130c1f12557c27b74aaa35 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=90 accepted=165/244 ttft=12.462 decode=8.007 elapsed=20.469 queued=0.217 tok/s=31.8 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 18, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-bfc2479081712bb1ee7f6a8adab276b2 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=3f46b826a241 finish=length rounds=10 accepted=5/13 ttft=0.225 decode=0.676 elapsed=0.901 queued=0.000 tok/s=22.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 19, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-8ba0fe389ec47a7f02406127f149af55 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.216 decode=8.844 elapsed=9.060 queued=0.000 tok/s=28.8 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-11cf9d516682a0494333869b30db725d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.214 decode=8.998 elapsed=9.212 queued=0.000 tok/s=28.3 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 21, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-aebf55808f8e2bd8c8706f48bd60ee28 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.216 decode=8.967 elapsed=9.183 queued=0.000 tok/s=28.4 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 22, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-18fcf71d76154f57fe282e452b253722 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=143 accepted=113/184 ttft=0.215 decode=9.651 elapsed=9.866 queued=0.000 tok/s=26.4 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 23, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-aac95e80924a1381ae76515252da2bc4 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=255 accepted=0/0 ttft=0.205 decode=14.052 elapsed=14.257 queued=0.000 tok/s=18.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-3178ac5d60ab8c5f8a327937a1cf18b6 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=255 accepted=0/0 ttft=0.204 decode=13.943 elapsed=14.147 queued=0.000 tok/s=18.3 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 25, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-06035b0644447d9808b99c991e9bcb31 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=255 accepted=0/0 ttft=0.205 decode=14.043 elapsed=14.248 queued=0.000 tok/s=18.2 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 26, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-469474bb0277fa2cbe595453c336d5bd prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=255 accepted=0/0 ttft=0.204 decode=13.918 elapsed=14.122 queued=0.000 tok/s=18.3 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 27, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-a9f3f2ddd90b4cfb039a0b7e8d363c71 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.214 decode=8.729 elapsed=8.943 queued=0.000 tok/s=29.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 29, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 29, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-3ecf94b720b3159c843bf6af6dd249d5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=102 accepted=154/300 ttft=0.214 decode=12.503 elapsed=12.717 queued=0.000 tok/s=20.4 seed=1234 +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-f183aec49d10aeb02de52de32c77ff60 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=105 accepted=150/308 ttft=0.505 decode=12.481 elapsed=12.986 queued=0.000 tok/s=20.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 30, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 33, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 33, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-72a369fc8315a7757d30a6f725c1a312 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=102 accepted=154/300 ttft=0.215 decode=15.692 elapsed=15.906 queued=0.000 tok/s=16.3 seed=1234 +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 34, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0e29a5ece3ed2f315168444bbc262430 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=103 accepted=152/306 ttft=0.822 decode=15.739 elapsed=16.562 queued=0.291 tok/s=16.2 seed=1236 +[tensorfold] done chatcmpl-8eb92818d2850c711d6a5e10017fce8b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=105 accepted=150/312 ttft=0.506 decode=16.161 elapsed=16.667 queued=0.292 tok/s=15.8 seed=1235 +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-c9fb38593f2171cab672b61214d1c5a7 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=141 accepted=115/188 ttft=16.118 decode=9.914 elapsed=26.032 queued=0.291 tok/s=25.7 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 35, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-2f3b7a992a72016737759dfdd831d3df prompt=14 cached=0 thinking=False effort=none tokens=16 sha=3000ccd48624 finish=length rounds=5 accepted=10/11 ttft=0.145 decode=0.379 elapsed=0.523 queued=0.000 tok/s=39.6 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 36, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-49d94100dc2965f9c383dfcbbe15181a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.146 decode=4.547 elapsed=4.694 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 37, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-1a270940c02c653c8497bc76876a3b5c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=255 accepted=0/0 ttft=0.139 decode=13.918 elapsed=14.057 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 38, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-6dba869efe4eb5667b1dd46916f9c043 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.147 decode=4.549 elapsed=4.697 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 40, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 40, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-221f497868a6a003b25891d20e97d500 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.145 decode=8.043 elapsed=8.188 queued=0.000 tok/s=31.7 +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-5fb6969a9faa746dfaaa0d069e61ba15 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.364 decode=7.898 elapsed=8.261 queued=0.000 tok/s=32.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 41, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 44, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 44, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-972f47430ea6dd34b3fcde5466aed01d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.145 decode=9.793 elapsed=9.939 queued=0.000 tok/s=26.0 +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 45, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-a5c84acbd694bd43752f3d85d2d27c3a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.364 decode=9.856 elapsed=10.220 queued=0.219 tok/s=25.9 +[tensorfold] done cmpl-6b910ae4a947a48700c5e7d14b4894c9 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.612 decode=9.719 elapsed=10.331 queued=0.219 tok/s=26.2 +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done cmpl-7183eea17060f6a880e5d37d87663f69 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/221 ttft=10.081 decode=4.657 elapsed=14.739 queued=0.219 tok/s=54.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 46, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-9331f64f7bc802538563f832b4a0b4bd prompt=32 cached=0 thinking=False effort=none tokens=16 sha=daface6f9795 finish=length rounds=6 accepted=10/12 ttft=0.214 decode=0.446 elapsed=0.660 queued=0.000 tok/s=33.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 47, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-5bb40ccf5d5f2f152283e0d25309c185 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.215 decode=8.347 elapsed=8.562 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-cec13f1493a1d1d9cf1b170f38b40d48 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=255 accepted=0/0 ttft=0.205 decode=14.035 elapsed=14.240 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 49, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-8f1f2aaad47f624bc9046c3a2a094917 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.344 elapsed=8.559 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 51, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 51, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-bd375d15cf5835af00a49d827644410a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=97 accepted=158/287 ttft=0.214 decode=11.598 elapsed=11.812 queued=0.000 tok/s=22.0 +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-810df5d2d38dd5a96749b19547fd2cb9 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.505 decode=11.445 elapsed=11.949 queued=0.000 tok/s=22.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 55, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 55, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-950753d906865136c04e209f173c4e00 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=97 accepted=158/287 ttft=0.214 decode=14.480 elapsed=14.694 queued=0.000 tok/s=17.6 +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-81f5ef4c6edaba63787f2e82448149fc prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.504 decode=14.672 elapsed=15.176 queued=0.290 tok/s=17.4 +[tensorfold] done chatcmpl-1ab84a1e327ce24720c0b264e6c02358 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.823 decode=14.455 elapsed=15.278 queued=0.290 tok/s=17.6 +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-9535d1be94d191370497592c11aefe6d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=117 accepted=138/200 ttft=14.906 decode=8.471 elapsed=23.377 queued=0.290 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 0, kept 2; 2 states, 490 MiB and 3606 MiB spare of 4096 MiB (hits 0, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-bc471344a13d7ea1e9154d0b07e41d23 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=28 accepted=33/46 ttft=6.158 decode=2.080 elapsed=8.237 queued=0.000 tok/s=28.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 6144, kept 0; 2 states, 490 MiB and 3606 MiB spare of 4096 MiB (hits 1, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-914baa66e0e9ffefd009d76f8afc3c7e prompt=7759 cached=6144 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=1.291 decode=3.529 elapsed=4.819 queued=0.000 tok/s=17.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 6144, kept 1; 3 states, 850 MiB and 3246 MiB spare of 4096 MiB (hits 2, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-bee3d40782992fb152cab03804ea62cf prompt=15550 cached=6144 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=13 accepted=30/36 ttft=7.282 decode=1.079 elapsed=8.361 queued=0.000 tok/s=39.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 14336, kept 0; 3 states, 850 MiB and 3246 MiB spare of 4096 MiB (hits 3, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-d81c4822dcc81c54de2c8d3f61cbd0d9 prompt=15550 cached=14336 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=43 accepted=0/0 ttft=1.037 decode=2.475 elapsed=3.512 queued=0.000 tok/s=17.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 19460 tokens, resumed at 14336, kept 1; 4 states, 1260 MiB and 2836 MiB spare of 4096 MiB (hits 4, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-04a7a13f49bf6af7813a4aa63cfd1990 prompt=19460 cached=14336 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=35 accepted=45/68 ttft=4.187 decode=2.679 elapsed=6.866 queued=0.000 tok/s=29.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 19460 tokens, resumed at 18432, kept 0; 4 states, 1260 MiB and 2836 MiB spare of 4096 MiB (hits 5, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-6e0b35b19d4f068f3a995231fdd88bc8 prompt=19460 cached=18432 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=79 accepted=0/0 ttft=0.946 decode=4.486 elapsed=5.432 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 21874 tokens, resumed at 18432, kept 1; 5 states, 1696 MiB and 2400 MiB spare of 4096 MiB (hits 6, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-6a3e872ebfb80706b50da9815c451e0e prompt=21874 cached=18432 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=70 accepted=78/124 ttft=2.780 decode=5.347 elapsed=8.127 queued=0.000 tok/s=27.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 21874 tokens, resumed at 20480, kept 0; 5 states, 1696 MiB and 2400 MiB spare of 4096 MiB (hits 7, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-97370dd8a981d0dcd573a8f79a9194c4 prompt=21874 cached=20480 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=147 accepted=0/0 ttft=1.135 decode=8.445 elapsed=9.580 queued=0.000 tok/s=17.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 26786 tokens, resumed at 20480, kept 1; 6 states, 2208 MiB and 1888 MiB spare of 4096 MiB (hits 8, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-2c7c9832198b169555185cc967211a2e prompt=26786 cached=20480 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=72 accepted=88/126 ttft=5.170 decode=5.357 elapsed=10.527 queued=0.000 tok/s=29.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 26786 tokens, resumed at 26624, kept 0; 6 states, 2208 MiB and 1888 MiB spare of 4096 MiB (hits 9, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-b09eb96e61b0456accf1c8cf6031ab5a prompt=26786 cached=26624 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=159 accepted=0/0 ttft=0.440 decode=9.147 elapsed=9.587 queued=0.000 tok/s=17.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 29293 tokens, resumed at 26624, kept 1; 7 states, 2746 MiB and 1350 MiB spare of 4096 MiB (hits 10, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-31127cfdae423d032c4e34fac37bbb31 prompt=29293 cached=26624 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=73 accepted=86/129 ttft=2.455 decode=5.443 elapsed=7.898 queued=0.000 tok/s=29.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 29293 tokens, resumed at 28672, kept 0; 7 states, 2746 MiB and 1350 MiB spare of 4096 MiB (hits 11, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-8680c8a24722b6a1f6a64629066cb87c prompt=29293 cached=28672 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=159 accepted=0/0 ttft=0.713 decode=8.894 elapsed=9.606 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 32216 tokens, resumed at 28672, kept 1; 8 states, 3310 MiB and 786 MiB spare of 4096 MiB (hits 12, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-9cf97f2a526457f5f15c1f931de9fa94 prompt=32216 cached=28672 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=78 accepted=82/126 ttft=2.820 decode=5.601 elapsed=8.421 queued=0.000 tok/s=28.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32216 tokens, resumed at 30720, kept 0; 8 states, 3310 MiB and 786 MiB spare of 4096 MiB (hits 13, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-757d0ba8d9e05d038bf88074401addc1 prompt=32216 cached=30720 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=159 accepted=0/0 ttft=1.194 decode=9.028 elapsed=10.222 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 35142 tokens, resumed at 30720, kept 1; 9 states, 3924 MiB and 172 MiB spare of 4096 MiB (hits 14, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-5a9db50b2fdf0ba47cb8c4753ea3508a prompt=35142 cached=30720 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=78 accepted=82/128 ttft=3.726 decode=5.557 elapsed=9.283 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 35142 tokens, resumed at 34816, kept 0; 9 states, 3924 MiB and 172 MiB spare of 4096 MiB (hits 15, misses 57, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bbc7f52934bc8abd9fcd418076c5979b prompt=35142 cached=34816 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=159 accepted=0/0 ttft=0.559 decode=8.918 elapsed=9.477 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 37860 tokens, resumed at 34816, kept 1; 8 states, 3794 MiB and 302 MiB spare of 4096 MiB (hits 16, misses 57, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-a4c36720c170bd053718f8c0135817d3 prompt=37860 cached=34816 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=29 accepted=130/135 ttft=2.557 decode=2.715 elapsed=5.272 queued=0.000 tok/s=58.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 37860 tokens, resumed at 36864, kept 0; 8 states, 3794 MiB and 302 MiB spare of 4096 MiB (hits 17, misses 57, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bfce1234ee40428b49bbe21d62669cf2 prompt=37860 cached=36864 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=0.938 decode=8.954 elapsed=9.892 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 2115 tokens, resumed at 0, kept 1; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 17, misses 58, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-d206f5c858b39a56e0890893a3eb994e prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=9761cff1a208 finish=length rounds=70 accepted=89/135 ttft=1.766 decode=5.233 elapsed=6.999 queued=0.000 tok/s=30.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 18, misses 58, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-0930f10aee431e431fedf3bd3d53c5c2 prompt=7412 cached=6144 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=34 accepted=61/80 ttft=1.091 decode=2.694 elapsed=3.785 queued=0.000 tok/s=34.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 19, misses 58, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-9ace4a6abd3040a10487287c41171986 prompt=7412 cached=6144 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=94 accepted=0/0 ttft=1.047 decode=5.270 elapsed=6.317 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 20, misses 58, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-913f6e55d4590758d1abbd0a7f845870 prompt=7474 cached=6144 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=23 accepted=49/62 ttft=1.125 decode=1.905 elapsed=3.030 queued=0.000 tok/s=37.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 21, misses 58, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-3bad4095c86ae2c7976dc1e55560aab5 prompt=7474 cached=6144 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=71 accepted=0/0 ttft=1.083 decode=3.994 elapsed=5.076 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12623 tokens, resumed at 6144, kept 1; 9 states, 3886 MiB and 210 MiB spare of 4096 MiB (hits 22, misses 58, evicted 3, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-71b7c6f468263c0261481d545313aea2 prompt=12623 cached=6144 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=56 accepted=64/89 ttft=5.300 decode=3.959 elapsed=9.260 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12623 tokens, resumed at 12288, kept 0; 9 states, 3886 MiB and 210 MiB spare of 4096 MiB (hits 23, misses 58, evicted 3, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-cdb03e5a473f6271d8836471aeda7b88 prompt=12623 cached=12288 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=119 accepted=0/0 ttft=0.549 decode=6.621 elapsed=7.170 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12720 tokens, resumed at 6144, kept 1; 9 states, 3708 MiB and 388 MiB spare of 4096 MiB (hits 24, misses 58, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-cef1b6459d50d22b1a640b0e77b09063 prompt=12720 cached=6144 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=40 accepted=39/56 ttft=5.234 decode=2.775 elapsed=8.008 queued=0.000 tok/s=28.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12720 tokens, resumed at 12288, kept 0; 9 states, 3708 MiB and 388 MiB spare of 4096 MiB (hits 25, misses 58, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-01b648627a6d04cc389e3cf6b179fc50 prompt=12720 cached=12288 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=78 accepted=0/0 ttft=0.598 decode=4.366 elapsed=4.965 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 18063 tokens, resumed at 12288, kept 1; 10 states, 4092 MiB and 4 MiB spare of 4096 MiB (hits 26, misses 58, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-69792594d89fec1bcdb45fe50edc63c1 prompt=18063 cached=12288 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=78 accepted=82/123 ttft=4.452 decode=5.625 elapsed=10.077 queued=0.000 tok/s=28.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 18063 tokens, resumed at 16384, kept 0; 10 states, 4092 MiB and 4 MiB spare of 4096 MiB (hits 27, misses 58, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-bddc5860e3a84a9c06ad829ac3e9b943 prompt=18063 cached=16384 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=159 accepted=0/0 ttft=1.293 decode=8.917 elapsed=10.210 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15032 tokens, resumed at 12288, kept 1; 10 states, 3914 MiB and 182 MiB spare of 4096 MiB (hits 28, misses 58, evicted 5, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-91774ed5a0e1c47176911e5f5d56c2d1 prompt=15032 cached=12288 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=42 accepted=43/62 ttft=2.324 decode=2.934 elapsed=5.258 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15032 tokens, resumed at 14336, kept 0; 10 states, 3914 MiB and 182 MiB spare of 4096 MiB (hits 29, misses 58, evicted 5, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-b5ea0ea35e9f695cd265e3d83cbc9b69 prompt=15032 cached=14336 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=84 accepted=0/0 ttft=0.741 decode=4.708 elapsed=5.448 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 40609 tokens, resumed at 6144, kept 1; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 30, misses 58, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-6d3e188561736d71d23977f44eeeb821 prompt=40609 cached=6144 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=23 accepted=136/138 ttft=26.394 decode=2.360 elapsed=28.754 queued=0.000 tok/s=67.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 40609 tokens, resumed at 38912, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 58, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-181304f9fbc4ff4677a3cbf71a32be64 prompt=40609 cached=38912 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=1.343 decode=8.966 elapsed=10.309 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 59, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-f5b7a2448b70a5865243429d0ec50c32 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=68 accepted=88/131 ttft=1.137 decode=5.051 elapsed=6.189 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 60, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-5345e3f43205b4b07bad63061ac0c951 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=155 accepted=0/0 ttft=1.098 decode=8.504 elapsed=9.601 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 4208 tokens, resumed at 0, kept 1; 10 states, 3634 MiB and 462 MiB spare of 4096 MiB (hits 31, misses 61, evicted 7, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-374778b45ef1234251ae4f32974e458e prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.590 decode=0.308 elapsed=3.898 queued=0.000 tok/s=35.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 4208 tokens, resumed at 4096, kept 0; 10 states, 3634 MiB and 462 MiB spare of 4096 MiB (hits 32, misses 61, evicted 7, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-2571d7e8dd5b8e9389a964c3600c5938 prompt=4208 cached=4096 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=0.372 decode=0.625 elapsed=0.998 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] ended chatcmpl-b764e01480b9b9eae05d9190365c8d11 reason=client-left prompt=9702 tokens=0 after=4.11s +info: prompt cache: 9702 tokens, resumed at 0, kept 2; 11 states, 3840 MiB and 256 MiB spare of 4096 MiB (hits 32, misses 63, evicted 8, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-85c8915dea87fc0a1e54c3c7c8c9d6f8 prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=39 accepted=43/59 ttft=8.204 decode=2.848 elapsed=11.051 queued=0.088 tok/s=28.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 9702 tokens, resumed at 8192, kept 0; 11 states, 3840 MiB and 256 MiB spare of 4096 MiB (hits 33, misses 63, evicted 8, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] done chatcmpl-838a5f2cdcea29c3c30edbd9425bd9b6 prompt=9702 cached=8192 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=81 accepted=0/0 ttft=1.193 decode=4.559 elapsed=5.752 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15152 tokens, resumed at 8192, kept 1; 11 states, 3866 MiB and 230 MiB spare of 4096 MiB (hits 34, misses 63, evicted 9, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-d15935eb992c5c914001c198569ba312 prompt=15152 cached=8192 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=71 accepted=88/117 ttft=5.484 decode=5.103 elapsed=10.587 queued=0.000 tok/s=31.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15152 tokens, resumed at 14336, kept 0; 11 states, 3866 MiB and 230 MiB spare of 4096 MiB (hits 35, misses 63, evicted 9, refused 0, failed 0) +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-d1239aa87e11e46405e250c9ca9ec7e4 prompt=15152 cached=14336 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=159 accepted=0/0 ttft=0.821 decode=8.885 elapsed=9.706 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 17550 tokens, resumed at 14336, kept 1; 11 states, 4018 MiB and 78 MiB spare of 4096 MiB (hits 36, misses 63, evicted 10, refused 0, failed 0) +info: prompt cache: 20872 tokens, resumed at 16384, kept 2; 11 states, 4094 MiB and 2 MiB spare of 4096 MiB (hits 37, misses 63, evicted 11, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bf38002935faf52a336dabda361333dc prompt=20872 cached=16384 thinking=True effort=high tokens=160 sha=eab19e6ff8fd finish=length rounds=54 accepted=105/161 ttft=6.664 decode=6.652 elapsed=13.315 queued=0.000 tok/s=23.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 504 + store 0 == total 504 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-db44e37ea82f6d282bd40644492aedf3 prompt=17550 cached=14336 thinking=True effort=high tokens=160 sha=fe9efe1d4302 finish=length rounds=58 accepted=102/167 ttft=2.612 decode=10.769 elapsed=13.381 queued=0.000 tok/s=14.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:17192:17192 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:17192:17192 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:17192:17192 [0] NCCL INFO ncclCommInitRank comm 0x3b53f7a0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:17192:17192 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.42 (kernels 0.26, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-d:17192:17192 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:17192:17192 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe349a444c000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe349a448d000) +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1639 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1639 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a1211 ctsFifoLkey=0x1a1211 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1639 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1640 pkey=0 pd=0xe349a4053590 oooRq=0 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1641 pkey=0 pd=0xe349a4053590 +host-d:17192:17211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1639 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17192:17192 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/server-rank1.log new file mode 100644 index 0000000000..f4b9f839cc --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2all/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T14:55:18+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native d82043de5d8b1f04 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 4 +host-e:1698:1698 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:1698:1698 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:1698:1698 [0] NCCL INFO cudaDriverVersion 13030 +host-e:1698:1698 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:1698:1698 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:1698:1698 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:1698:1698 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:1698:1698 [0] NCCL INFO Count not open libibverbs.so +host-e:1698:1698 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:1698:1698 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:1698:1698 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:1698:1698 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:1698:1698 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x3dfff690 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:1698:1698 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:1698:1698 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:1698:1698 [0] NCCL INFO Initialized NET plugin IB +host-e:1698:1698 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:1698:1698 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:1698:1698 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:1698:1698 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:1698:1698 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:1698:1698 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:1698:1698 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:1698:1698 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:1698:1698 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:1698:1698 [0] NCCL INFO Using network IB +host-e:1698:1698 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:1698:1698 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x3b3cd860 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:1698:1698 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:1698:1698 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1698:1698 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1698:1698 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1698:1698 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:1698:1698 [0] NCCL INFO Rank 1: 1 Net devices +host-e:1698:1698 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:1698:1698 [0] NCCL INFO comm 0x3b3cd860 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:1698:1698 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:1698:1698 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:1698:1698 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:1698:1698 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:1698:1698 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:1698:1698 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:1698:1710 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-e:1698:1710 [0] NCCL INFO proxy listening socket at <46333> +host-e:1698:1711 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 10 +host-e:1698:1698 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1698:1712 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-e:1698:1698 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1698:1698 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1698:1698 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1698:1698 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1698:1698 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1698:1698 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1698:1698 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe8ae0c010000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe8ae0c054000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe8ae0c09c000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe8ae0c0e0000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe8ae0c128000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe8ae0c16c000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe8ae0c1b4000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe8ae0c1f8000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1371 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1371 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1371 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1372 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1373 pkey=0 pd=0xe8ae0c053eb0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1374 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1374 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1703b8 ctsFifoLkey=0x1703b8 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1374 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1375 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1376 pkey=0 pd=0xe8ae0c053eb0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1377 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1377 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x13b26d ctsFifoLkey=0x13b26d +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1377 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1378 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1378 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1706c5 ctsFifoLkey=0x1706c5 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1378 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1371 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1374 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1379 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1380 pkey=0 pd=0xe8ae0c053eb0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1381 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1382 pkey=0 pd=0xe8ae0c053eb0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1377 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1378 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1698 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:1698:1698 [0] NCCL INFO Connected all trees +host-e:1698:1698 [0] NCCL INFO Connected binomial trees +host-e:1698:1698 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:1698:1698 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:1698:1698 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:1698:1698 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:1698:1698 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:1698:1698 [0] NCCL INFO ncclCommInitRank comm 0x3b3cd860 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:1698:1698 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.42 (kernels 0.24, alloc 0.03, bootstrap 0.03, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-e:1698:1698 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Sharinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +ed +host-e:1698:1698 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe8ae0c447000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe8ae0c488000) +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1383 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1383 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16ce8b ctsFifoLkey=0x16ce8b +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1383 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1384 pkey=0 pd=0xe8ae0c053eb0 oooRq=0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1385 pkey=0 pd=0xe8ae0c053eb0 +host-e:1698:1710 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1383 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1698:1698 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/client.log new file mode 100644 index 0000000000..22ff48d723 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/client.log @@ -0,0 +1,45 @@ +server up 2026-10-23 00:41:01 + 0 S1 t1 drafted prompt 7759 cached 0 ttft 5.99s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 1 S1 t1 plain prompt 7759 cached 0 ttft 5.94s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 2 S1 t2 drafted prompt 15550 cached 0 ttft 11.91s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 + 3 S1 t2 plain prompt 15550 cached 0 ttft 11.87s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 + 4 S1 t3 drafted prompt 19460 cached 0 ttft 14.90s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 + 5 S1 t3 plain prompt 19460 cached 0 ttft 14.87s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 + 6 S1 t4 drafted prompt 21874 cached 0 ttft 16.67s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 + 7 S1 t4 plain prompt 21874 cached 0 ttft 16.63s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 + 8 S1 t5 drafted prompt 26786 cached 0 ttft 20.59s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c + 9 S1 t5 plain prompt 26786 cached 0 ttft 20.57s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c + 10 S1 t6 drafted prompt 29293 cached 0 ttft 22.44s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 + 11 S1 t6 plain prompt 29293 cached 0 ttft 22.39s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 + 12 S1 t7 drafted prompt 32216 cached 0 ttft 24.50s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e + 13 S1 t7 plain prompt 32216 cached 0 ttft 24.44s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e + 14 S1 t8 drafted prompt 35142 cached 0 ttft 26.95s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 + 15 S1 t8 plain prompt 35142 cached 0 ttft 26.93s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 + 16 S1 t9 drafted prompt 37860 cached 0 ttft 28.90s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 17 S1 t9 plain prompt 37860 cached 0 ttft 28.86s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 18 X t1 drafted prompt 2115 cached 0 ttft 1.77s sha 9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd + 19 A t1 drafted prompt 7412 cached 0 ttft 5.81s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 + 20 A t1 plain prompt 7412 cached 0 ttft 5.76s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 + 21 B t1 drafted prompt 7474 cached 0 ttft 5.85s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a + 22 B t1 plain prompt 7474 cached 0 ttft 5.80s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a + 23 A t2 drafted prompt 12623 cached 0 ttft 9.88s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 + 24 A t2 plain prompt 12623 cached 0 ttft 9.86s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 + 25 B t2 drafted prompt 12720 cached 0 ttft 9.94s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c + 26 B t2 plain prompt 12720 cached 0 ttft 9.91s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c + 27 A t3 drafted prompt 18063 cached 0 ttft 13.74s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc + 28 A t3 plain prompt 18063 cached 0 ttft 13.68s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc + 29 B t3 drafted prompt 15032 cached 0 ttft 11.63s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa + 30 B t3 plain prompt 15032 cached 0 ttft 11.59s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa + 31 E t1 drafted prompt 40609 cached 0 ttft 30.89s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 32 E t1 plain prompt 40609 cached 0 ttft 30.82s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 33 T t1 drafted prompt 1520 cached 0 ttft 1.14s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 + 34 T t1 plain prompt 1520 cached 0 ttft 1.10s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 + 35 T t2 drafted prompt 4208 cached 0 ttft 3.59s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 36 T t2 plain prompt 4208 cached 0 ttft 3.57s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 38 C t1 drafted prompt 9702 cached 0 ttft 7.48s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 + 39 C t1 plain prompt 9702 cached 0 ttft 7.42s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 + 40 C t2 drafted prompt 15152 cached 0 ttft 11.72s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 + 41 C t2 plain prompt 15152 cached 0 ttft 11.69s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 + 42 A t4 drafted prompt 20872 cached 0 ttft 30.56s sha eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c + 43 B t4 drafted prompt 17550 cached 0 ttft 13.46s sha fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8 +=== kvp-client r2auto rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/fresh.jsonl b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/fresh.jsonl new file mode 100644 index 0000000000..0502ff42f2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/fresh.jsonl @@ -0,0 +1,43 @@ +{"i": 0, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.987745432, "prefill": 5.982330831, "wall": 8.013181532995077, "finish": "tool_calls"}} +{"i": 1, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.943437402, "prefill": 5.942040348, "wall": 9.31006534700282, "finish": "tool_calls"}} +{"i": 2, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 0, "prompt": 15550, "tokens": 44, "ttft": 11.911739592, "prefill": 11.907781449, "wall": 12.96706995301065, "finish": "tool_calls"}} +{"i": 3, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 0, "prompt": 15550, "tokens": 44, "ttft": 11.87404734, "prefill": 11.871634409, "wall": 14.305380492005497, "finish": "tool_calls"}} +{"i": 4, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 0, "prompt": 19460, "tokens": 80, "ttft": 14.903972698, "prefill": 14.901167051, "wall": 17.487209471990354, "finish": "tool_calls"}} +{"i": 5, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 0, "prompt": 19460, "tokens": 80, "ttft": 14.866049747, "prefill": 14.863678638, "wall": 19.285275830014143, "finish": "tool_calls"}} +{"i": 6, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 0, "prompt": 21874, "tokens": 148, "ttft": 16.665612076, "prefill": 16.661217739, "wall": 21.787589572981233, "finish": "tool_calls"}} +{"i": 7, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 0, "prompt": 21874, "tokens": 148, "ttft": 16.626698189, "prefill": 16.623841674, "wall": 24.829707569995662, "finish": "tool_calls"}} +{"i": 8, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 0, "prompt": 26786, "tokens": 160, "ttft": 20.590949381, "prefill": 20.585682999, "wall": 25.86036802100716, "finish": "length"}} +{"i": 9, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 0, "prompt": 26786, "tokens": 160, "ttft": 20.569985621, "prefill": 20.566141143, "wall": 29.418625235994114, "finish": "length"}} +{"i": 10, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 0, "prompt": 29293, "tokens": 160, "ttft": 22.435963077, "prefill": 22.430026325, "wall": 27.792044868023368, "finish": "length"}} +{"i": 11, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 0, "prompt": 29293, "tokens": 160, "ttft": 22.389467849, "prefill": 22.385143945, "wall": 31.265108930005226, "finish": "length"}} +{"i": 12, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 0, "prompt": 32216, "tokens": 160, "ttft": 24.497292122, "prefill": 24.490725286, "wall": 30.08347808101098, "finish": "length"}} +{"i": 13, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 0, "prompt": 32216, "tokens": 160, "ttft": 24.4375639, "prefill": 24.432613482, "wall": 33.359787550987676, "finish": "length"}} +{"i": 14, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 0, "prompt": 35142, "tokens": 160, "ttft": 26.948137322, "prefill": 26.941539343, "wall": 32.50843022399931, "finish": "length"}} +{"i": 15, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 0, "prompt": 35142, "tokens": 160, "ttft": 26.925850063, "prefill": 26.920637818, "wall": 35.7970487929997, "finish": "length"}} +{"i": 16, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 37860, "tokens": 160, "ttft": 28.899802535, "prefill": 28.891910363, "wall": 31.611358674999792, "finish": "length"}} +{"i": 17, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 37860, "tokens": 160, "ttft": 28.861389057, "prefill": 28.855337641, "wall": 37.7917621259985, "finish": "length"}} +{"i": 18, "got": {"sha": "9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.765977197, "prefill": 1.76397314, "wall": 6.9804639810172375, "finish": "length"}} +{"i": 19, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 0, "prompt": 7412, "tokens": 95, "ttft": 5.813749246, "prefill": 5.811753189, "wall": 8.49858529100311, "finish": "tool_calls"}} +{"i": 20, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 0, "prompt": 7412, "tokens": 95, "ttft": 5.762894782, "prefill": 5.761373394, "wall": 11.02341419699951, "finish": "tool_calls"}} +{"i": 21, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 0, "prompt": 7474, "tokens": 72, "ttft": 5.849727331, "prefill": 5.84792292, "wall": 7.74150723600178, "finish": "tool_calls"}} +{"i": 22, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 0, "prompt": 7474, "tokens": 72, "ttft": 5.801493412, "prefill": 5.799975016, "wall": 9.790296332008438, "finish": "tool_calls"}} +{"i": 23, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 0, "prompt": 12623, "tokens": 120, "ttft": 9.884570054, "prefill": 9.881123393, "wall": 13.834311173006427, "finish": "tool_calls"}} +{"i": 24, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 0, "prompt": 12623, "tokens": 120, "ttft": 9.858600711, "prefill": 9.856502653, "wall": 16.470275612024125, "finish": "tool_calls"}} +{"i": 25, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 0, "prompt": 12720, "tokens": 79, "ttft": 9.943720667, "prefill": 9.9404449, "wall": 12.702250520989764, "finish": "tool_calls"}} +{"i": 26, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 0, "prompt": 12720, "tokens": 79, "ttft": 9.906175371, "prefill": 9.904135072, "wall": 14.265218640008243, "finish": "tool_calls"}} +{"i": 27, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 0, "prompt": 18063, "tokens": 160, "ttft": 13.740778672, "prefill": 13.73662657, "wall": 19.288884251989657, "finish": "length"}} +{"i": 28, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 0, "prompt": 18063, "tokens": 160, "ttft": 13.681700197, "prefill": 13.678712316, "wall": 22.59434704700834, "finish": "length"}} +{"i": 29, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 0, "prompt": 15032, "tokens": 85, "ttft": 11.628619391, "prefill": 11.624856645, "wall": 14.558157688006759, "finish": "tool_calls"}} +{"i": 30, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 0, "prompt": 15032, "tokens": 85, "ttft": 11.586130442, "prefill": 11.58403275, "wall": 16.273647376016015, "finish": "tool_calls"}} +{"i": 31, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 40609, "tokens": 160, "ttft": 30.88511667, "prefill": 30.876075628, "wall": 33.246787948010024, "finish": "length"}} +{"i": 32, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 40609, "tokens": 160, "ttft": 30.824142487, "prefill": 30.817731167, "wall": 39.75185300598969, "finish": "length"}} +{"i": 33, "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.138866303, "prefill": 1.137217601, "wall": 6.110236741020344, "finish": "stop"}} +{"i": 34, "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.099017372, "prefill": 1.098167884, "wall": 9.581343415018637, "finish": "stop"}} +{"i": 35, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.588797557, "prefill": 3.586334967, "wall": 3.895463794004172, "finish": "stop"}} +{"i": 36, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.573676689, "prefill": 3.571854642, "wall": 4.2009513300145045, "finish": "stop"}} +{"i": 38, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 7.476737504, "prefill": 7.470625636, "wall": 10.230087773001287, "finish": "tool_calls"}} +{"i": 39, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 7.421199624, "prefill": 7.419515625, "wall": 11.971817195997573, "finish": "tool_calls"}} +{"i": 40, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 0, "prompt": 15152, "tokens": 160, "ttft": 11.724216259, "prefill": 11.720359668, "wall": 16.827148882002803, "finish": "length"}} +{"i": 41, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 0, "prompt": 15152, "tokens": 160, "ttft": 11.687963334, "prefill": 11.685876999, "wall": 20.536979697993957, "finish": "length"}} +{"i": 42, "got": {"sha": "eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c", "cached": 0, "prompt": 20872, "tokens": 160, "ttft": 30.557695247, "prefill": 29.716243017, "wall": 37.01368465399719, "finish": "length"}} +{"i": 43, "got": {"sha": "fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8", "cached": 0, "prompt": 17550, "tokens": 160, "ttft": 13.460088441, "prefill": 13.455697418, "wall": 36.637677328981226, "finish": "length"}} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/server-rank0.log new file mode 100644 index 0000000000..f734227461 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/server-rank0.log @@ -0,0 +1,231 @@ +[kvp-serve] 2026-10-22T15:35:52+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native d82043de5d8b1f04 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:18299:18299 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:18299:18299 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:18299:18299 [0] NCCL INFO cudaDriverVersion 13030 +host-d:18299:18299 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:18299:18299 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:18299:18299 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:18299:18299 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:18299:18299 [0] NCCL INFO Count not open libibverbs.so +host-d:18299:18299 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:18299:18299 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:18299:18299 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:18299:18299 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:18299:18299 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x30ef5980 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:18299:18299 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:18299:18299 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:18299:18299 [0] NCCL INFO Initialized NET plugin IB +host-d:18299:18299 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:18299:18299 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:18299:18299 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:18299:18299 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:18299:18299 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:18299:18299 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:18299:18299 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:18299:18299 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:18299:18299 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:18299:18299 [0] NCCL INFO Using network IB +host-d:18299:18299 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:18299:18299 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x2e2c3900 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:18299:18299 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:18299:18299 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:18299:18299 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:18299:18299 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:18299:18299 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:18299:18299 [0] NCCL INFO Rank 0: 1 Net devices +host-d:18299:18299 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:18299:18299 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:18299:18299 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:18299:18299 [0] NCCL INFO comm 0x2e2c3900 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:18299:18299 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:18299:18299 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:18299:18299 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:18299:18299 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:18299:18299 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:18299:18299 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:18299:18299 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:18299:18299 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:18299:18299 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:18299:18299 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:18299:18319 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-d:18299:18318 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-d:18299:18318 [0] NCCL INFO proxy listening socket at <33287> +host-d:18299:18299 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:18299:18299 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:18299:18299 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:18299:18320 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-d:18299:18299 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:18299:18299 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:18299:18299 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:18299:18299 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:18299:18299 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39d8c010000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39d8c054000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39d8c09c000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1659 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1659 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bbdba ctsFifoLkey=0x1bbdba +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1659 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1660 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1660 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a4f4c ctsFifoLkey=0x1a4f4c +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1660 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39d8c14e000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39d8c196000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39d8c1da000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39d8c222000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39d8c266000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1661 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1662 pkey=0 pd=0xf39d8c053590 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1659 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1660 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1663 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1664 pkey=0 pd=0xf39d8c053590 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1665 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1665 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5a59 ctsFifoLkey=0x1b5a59 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1665 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1666 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1667 pkey=0 pd=0xf39d8c053590 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1668 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1668 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b6764 ctsFifoLkey=0x1b6764 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1668 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1669 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1670 pkey=0 pd=0xf39d8c053590 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1665 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1668 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18299 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:18299:18299 [0] NCCL INFO Connected all trees +host-d:18299:18299 [0] NCCL INFO Connected binomial trees +host-d:18299:18299 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:18299:18299 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:18299:18299 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:18299:18299 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.6 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 307.6s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0266042dadf75f7e2cffde0e2336d84e prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=28 accepted=33/46 ttft=5.988 decode=1.999 elapsed=7.987 queued=0.000 tok/s=30.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-918d2ae14fed52e62dbe5eed94bc6354 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=5.943 decode=3.365 elapsed=9.308 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5277de7f1d21c759eb42b11c54d20e9e prompt=15550 cached=0 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=13 accepted=30/36 ttft=11.912 decode=1.054 elapsed=12.965 queued=0.000 tok/s=40.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5814ed127104c479b4b9908c8b781c35 prompt=15550 cached=0 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=43 accepted=0/0 ttft=11.874 decode=2.429 elapsed=14.303 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-01a0045969a8d8f6d27989447b8f454a prompt=19460 cached=0 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=35 accepted=45/68 ttft=14.904 decode=2.582 elapsed=17.485 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6980f5a3fbeaa6e10d14229c6ee5d81b prompt=19460 cached=0 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=79 accepted=0/0 ttft=14.866 decode=4.417 elapsed=19.283 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0bfed12a659f4120f86e725399012650 prompt=21874 cached=0 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=70 accepted=78/124 ttft=16.666 decode=5.120 elapsed=21.785 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-548cc1607f0923641b7ff3d13846e391 prompt=21874 cached=0 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=147 accepted=0/0 ttft=16.627 decode=8.201 elapsed=24.828 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-c5a864c3fbf8bd66224fdcef3f40130e prompt=26786 cached=0 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=72 accepted=88/126 ttft=20.591 decode=5.268 elapsed=25.859 queued=0.000 tok/s=30.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-02679d7f6c8dae01aff14e131866ac47 prompt=26786 cached=0 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=159 accepted=0/0 ttft=20.570 decode=8.846 elapsed=29.416 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-e0ad10862b99b89f5a4530726f953e4e prompt=29293 cached=0 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=73 accepted=86/129 ttft=22.436 decode=5.354 elapsed=27.790 queued=0.000 tok/s=29.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-01cf1e127006e66aead884a95cf97386 prompt=29293 cached=0 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=159 accepted=0/0 ttft=22.389 decode=8.873 elapsed=31.263 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-f754b85c941fb11b6014364c415da92d prompt=32216 cached=0 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=78 accepted=82/126 ttft=24.497 decode=5.584 elapsed=30.081 queued=0.000 tok/s=28.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bc87eed01e708ff91cea5b9930673e69 prompt=32216 cached=0 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=159 accepted=0/0 ttft=24.438 decode=8.919 elapsed=33.357 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-2e506ff23d777196db0084191d3fb119 prompt=35142 cached=0 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=78 accepted=82/128 ttft=26.948 decode=5.558 elapsed=32.506 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-28565871140c0acfd52339fd211f32b0 prompt=35142 cached=0 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=159 accepted=0/0 ttft=26.926 decode=8.869 elapsed=35.795 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-6714fe9f9743892ca9f182dbf9e65070 prompt=37860 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=29 accepted=130/135 ttft=28.900 decode=2.710 elapsed=31.609 queued=0.000 tok/s=58.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-6cb640e365d80f56d6b1688caa51799c prompt=37860 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=28.861 decode=8.928 elapsed=37.789 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6bc7e4895d371f3d4fe7d33fa81af932 prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=9761cff1a208 finish=length rounds=70 accepted=89/135 ttft=1.766 decode=5.213 elapsed=6.979 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a3c38450578dfad50d438d076c00f70b prompt=7412 cached=0 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=34 accepted=61/80 ttft=5.814 decode=2.683 elapsed=8.497 queued=0.000 tok/s=35.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0dc068c03b9cbd39e10ff4ec8f03bd57 prompt=7412 cached=0 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=94 accepted=0/0 ttft=5.763 decode=5.259 elapsed=11.022 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7a16ed1294f4cc85fe49b54252514fcd prompt=7474 cached=0 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=23 accepted=49/62 ttft=5.850 decode=1.890 elapsed=7.740 queued=0.000 tok/s=37.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1b47b0e1f8e605e9f190dcc18a65d150 prompt=7474 cached=0 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=71 accepted=0/0 ttft=5.801 decode=3.987 elapsed=9.789 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-361e105b9ebcf7400e65f9e13d7c4618 prompt=12623 cached=0 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=56 accepted=64/89 ttft=9.885 decode=3.948 elapsed=13.833 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f49299f9882d212a1a7dc275fd17868f prompt=12623 cached=0 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=119 accepted=0/0 ttft=9.859 decode=6.610 elapsed=16.469 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e29aea1c6849fa0ccc1302930c4bb381 prompt=12720 cached=0 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=40 accepted=39/56 ttft=9.944 decode=2.756 elapsed=12.700 queued=0.000 tok/s=28.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2cb1d407f057582e34faee620adea34e prompt=12720 cached=0 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=78 accepted=0/0 ttft=9.906 decode=4.357 elapsed=14.263 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2beffcefdab0fdcd8ccabad2d1080083 prompt=18063 cached=0 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=78 accepted=82/123 ttft=13.741 decode=5.547 elapsed=19.287 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0fec0e468d4a93e1c6023c039e5771ba prompt=18063 cached=0 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=159 accepted=0/0 ttft=13.682 decode=8.911 elapsed=22.592 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-08e70215c10566903272d5d698f992c1 prompt=15032 cached=0 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=42 accepted=43/62 ttft=11.629 decode=2.928 elapsed=14.556 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7d04a72b308800597158718f085f700e prompt=15032 cached=0 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=84 accepted=0/0 ttft=11.586 decode=4.686 elapsed=16.272 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-85a223f66a71236ea6ed5d244d620b8b prompt=40609 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=23 accepted=136/138 ttft=30.885 decode=2.359 elapsed=33.244 queued=0.000 tok/s=67.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-f57a4b0aacb13ec53e19e8e265c0f671 prompt=40609 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=30.824 decode=8.925 elapsed=39.749 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-92dce662a94a10f9f5ac084d7d15d511 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=68 accepted=88/131 ttft=1.139 decode=4.970 elapsed=6.109 queued=0.000 tok/s=31.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5999a10a55f0d8975cc53ccb726ba7e5 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=155 accepted=0/0 ttft=1.099 decode=8.481 elapsed=9.580 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d29525678ff3f423042b90b8b7ac71ed prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.589 decode=0.305 elapsed=3.894 queued=0.000 tok/s=36.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-afe7ab4d01929cc547022419dedcde86 prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=3.574 decode=0.626 elapsed=4.199 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7360d0fde0e0f96b03a980bca3193210 prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=39 accepted=43/59 ttft=7.477 decode=2.752 elapsed=10.228 queued=0.000 tok/s=29.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-db077369e84f37e56e803b258d899212 prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=81 accepted=0/0 ttft=7.421 decode=4.549 elapsed=11.970 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-0c5924cb87f183ea8e2d9693621fb412 prompt=15152 cached=0 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=71 accepted=88/117 ttft=11.724 decode=5.101 elapsed=16.825 queued=0.000 tok/s=31.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-13bce540902f9edad83d855cb4f0b2c2 prompt=15152 cached=0 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=159 accepted=0/0 ttft=11.688 decode=8.847 elapsed=20.535 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-f03cfe4c07dca63a2f16a5b2da6c30db prompt=17550 cached=0 thinking=True effort=high tokens=160 sha=fe9efe1d4302 finish=length rounds=61 accepted=99/167 ttft=13.460 decode=23.176 elapsed=36.636 queued=0.000 tok/s=6.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-2528095516cf579c9c13055183eec06d prompt=20872 cached=0 thinking=True effort=high tokens=160 sha=eab19e6ff8fd finish=length rounds=54 accepted=105/161 ttft=30.558 decode=6.454 elapsed=37.011 queued=0.837 tok/s=24.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:18299:18299 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:18299:18299 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:18299:18299 [0] NCCL INFO ncclCommInitRank comm 0x2e2c3900 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:18299:18299 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.41 (kernels 0.25, alloc 0.03, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-d:18299:18299 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:18299:18299 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39d8c44c000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39d8c48d000) +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1671 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1671 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ab2af ctsFifoLkey=0x1ab2af +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1671 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1672 pkey=0 pd=0xf39d8c053590 oooRq=0 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1673 pkey=0 pd=0xf39d8c053590 +host-d:18299:18318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1671 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18299:18299 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/server-rank1.log new file mode 100644 index 0000000000..072ba10415 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2auto/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T15:35:51+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native d82043de5d8b1f04 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:1780:1780 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:1780:1780 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:1780:1780 [0] NCCL INFO cudaDriverVersion 13030 +host-e:1780:1780 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:1780:1780 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:1780:1780 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:1780:1780 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:1780:1780 [0] NCCL INFO Count not open libibverbs.so +host-e:1780:1780 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:1780:1780 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:1780:1780 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:1780:1780 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:1780:1780 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0xefdc7a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:1780:1780 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:1780:1780 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:1780:1780 [0] NCCL INFO Initialized NET plugin IB +host-e:1780:1780 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:1780:1780 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:1780:1780 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:1780:1780 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:1780:1780 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:1780:1780 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:1780:1780 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:1780:1780 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:1780:1780 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:1780:1780 [0] NCCL INFO Using network IB +host-e:1780:1780 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:1780:1780 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0xc3aa970 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:1780:1780 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:1780:1780 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1780:1780 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1780:1780 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1780:1780 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:1780:1780 [0] NCCL INFO Rank 1: 1 Net devices +host-e:1780:1780 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:1780:1780 [0] NCCL INFO comm 0xc3aa970 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:1780:1780 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:1780:1780 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:1780:1780 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:1780:1780 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:1780:1780 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:1780:1780 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:1780:1793 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 4 +host-e:1780:1792 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-e:1780:1792 [0] NCCL INFO proxy listening socket at <56681> +host-e:1780:1780 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1780:1794 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-e:1780:1780 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1780:1780 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1780:1780 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1780:1780 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1780:1780 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1780:1780 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1780:1780 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc80e0010000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc80e0054000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc80e009c000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc80e00e0000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc80e0128000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc80e016c000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc80e01b4000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc80e01f8000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1403 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1403 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1403 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1404 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1405 pkey=0 pd=0xfc80e0053eb0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1406 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1406 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17eba8 ctsFifoLkey=0x17eba8 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1406 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1407 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1408 pkey=0 pd=0xfc80e0053eb0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1409 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1409 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x13b26d ctsFifoLkey=0x13b26d +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1409 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1410 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1410 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1703b8 ctsFifoLkey=0x1703b8 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1410 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1403 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1406 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1409 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1411 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1412 pkey=0 pd=0xfc80e0053eb0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1410 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1413 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1414 pkey=0 pd=0xfc80e0053eb0 +host-e:1780:1780 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:1780:1780 [0] NCCL INFO Connected all trees +host-e:1780:1780 [0] NCCL INFO Connected binomial trees +host-e:1780:1780 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:1780:1780 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:1780:1780 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:1780:1780 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:1780:1780 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:1780:1780 [0] NCCL INFO ncclCommInitRank comm 0xc3aa970 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:1780:1780 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.41 (kernels 0.24, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-e:1780:1780 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shared +edinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +host-e:1780:1780 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc80e0443000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc80e0484000) +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1415 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1415 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16ce8b ctsFifoLkey=0x16ce8b +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1415 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1416 pkey=0 pd=0xfc80e0053eb0 oooRq=0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1417 pkey=0 pd=0xfc80e0053eb0 +host-e:1780:1792 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1415 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1780:1780 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/client.log new file mode 100644 index 0000000000..7b6208c017 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/client.log @@ -0,0 +1,45 @@ +server up 2026-10-23 01:02:52 + 0 S1 t1 drafted prompt 7759 cached 0 ttft 6.01s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 1 S1 t1 plain prompt 7759 cached 0 ttft 5.95s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 2 S1 t2 drafted prompt 15550 cached 0 ttft 11.92s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 + 3 S1 t2 plain prompt 15550 cached 0 ttft 11.90s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 + 4 S1 t3 drafted prompt 19460 cached 0 ttft 14.92s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 + 5 S1 t3 plain prompt 19460 cached 0 ttft 14.92s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 + 6 S1 t4 drafted prompt 21874 cached 0 ttft 16.72s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 + 7 S1 t4 plain prompt 21874 cached 0 ttft 16.67s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 + 8 S1 t5 drafted prompt 26786 cached 0 ttft 20.65s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c + 9 S1 t5 plain prompt 26786 cached 0 ttft 20.62s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c + 10 S1 t6 drafted prompt 29293 cached 0 ttft 22.50s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 + 11 S1 t6 plain prompt 29293 cached 0 ttft 22.48s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 + 12 S1 t7 drafted prompt 32216 cached 0 ttft 24.98s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e + 13 S1 t7 plain prompt 32216 cached 0 ttft 24.86s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e + 14 S1 t8 drafted prompt 35142 cached 0 ttft 27.22s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 + 15 S1 t8 plain prompt 35142 cached 0 ttft 26.99s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 + 16 S1 t9 drafted prompt 37860 cached 0 ttft 29.00s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 17 S1 t9 plain prompt 37860 cached 0 ttft 28.97s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 18 X t1 drafted prompt 2115 cached 0 ttft 1.77s sha 9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd + 19 A t1 drafted prompt 7412 cached 0 ttft 5.83s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 + 20 A t1 plain prompt 7412 cached 0 ttft 5.79s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 + 21 B t1 drafted prompt 7474 cached 0 ttft 5.86s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a + 22 B t1 plain prompt 7474 cached 0 ttft 5.81s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a + 23 A t2 drafted prompt 12623 cached 0 ttft 9.93s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 + 24 A t2 plain prompt 12623 cached 0 ttft 9.90s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 + 25 B t2 drafted prompt 12720 cached 0 ttft 9.97s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c + 26 B t2 plain prompt 12720 cached 0 ttft 9.93s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c + 27 A t3 drafted prompt 18063 cached 0 ttft 13.80s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc + 28 A t3 plain prompt 18063 cached 0 ttft 13.74s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc + 29 B t3 drafted prompt 15032 cached 0 ttft 11.66s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa + 30 B t3 plain prompt 15032 cached 0 ttft 11.62s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa + 31 E t1 drafted prompt 40609 cached 0 ttft 31.01s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 32 E t1 plain prompt 40609 cached 0 ttft 30.94s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 33 T t1 drafted prompt 1520 cached 0 ttft 1.14s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 + 34 T t1 plain prompt 1520 cached 0 ttft 1.10s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 + 35 T t2 drafted prompt 4208 cached 0 ttft 3.60s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 36 T t2 plain prompt 4208 cached 0 ttft 3.58s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 38 C t1 drafted prompt 9702 cached 0 ttft 7.48s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 + 39 C t1 plain prompt 9702 cached 0 ttft 7.43s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 + 40 C t2 drafted prompt 15152 cached 0 ttft 11.76s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 + 41 C t2 plain prompt 15152 cached 0 ttft 11.72s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 + 42 A t4 drafted prompt 20872 cached 0 ttft 30.67s sha eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c + 43 B t4 drafted prompt 17550 cached 0 ttft 13.50s sha fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8 +=== kvp-client r2main rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/fresh.jsonl b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/fresh.jsonl new file mode 100644 index 0000000000..f78eb32539 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/fresh.jsonl @@ -0,0 +1,43 @@ +{"i": 0, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 6.005392306, "prefill": 6.000055984, "wall": 8.021633669995936, "finish": "tool_calls"}} +{"i": 1, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.952800402, "prefill": 5.951410329, "wall": 9.318812579993391, "finish": "tool_calls"}} +{"i": 2, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 0, "prompt": 15550, "tokens": 44, "ttft": 11.915183349, "prefill": 11.91105761, "wall": 12.97617468298995, "finish": "tool_calls"}} +{"i": 3, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 0, "prompt": 15550, "tokens": 44, "ttft": 11.901532077, "prefill": 11.899384, "wall": 14.331427381024696, "finish": "tool_calls"}} +{"i": 4, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 0, "prompt": 19460, "tokens": 80, "ttft": 14.922795821, "prefill": 14.920137101, "wall": 17.50658780199592, "finish": "tool_calls"}} +{"i": 5, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 0, "prompt": 19460, "tokens": 80, "ttft": 14.915911372, "prefill": 14.913661774, "wall": 19.331952039996395, "finish": "tool_calls"}} +{"i": 6, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 0, "prompt": 21874, "tokens": 148, "ttft": 16.724322414, "prefill": 16.720056596, "wall": 21.855412714008708, "finish": "tool_calls"}} +{"i": 7, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 0, "prompt": 21874, "tokens": 148, "ttft": 16.665424005, "prefill": 16.662544932, "wall": 24.86955321498681, "finish": "tool_calls"}} +{"i": 8, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 0, "prompt": 26786, "tokens": 160, "ttft": 20.654189407, "prefill": 20.648727071, "wall": 25.923374161997344, "finish": "length"}} +{"i": 9, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 0, "prompt": 26786, "tokens": 160, "ttft": 20.616813678, "prefill": 20.613031673, "wall": 29.495399768988136, "finish": "length"}} +{"i": 10, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 0, "prompt": 29293, "tokens": 160, "ttft": 22.503152339, "prefill": 22.497444866, "wall": 27.861310431995662, "finish": "length"}} +{"i": 11, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 0, "prompt": 29293, "tokens": 160, "ttft": 22.481919275, "prefill": 22.477584083, "wall": 31.561195243004477, "finish": "length"}} +{"i": 12, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 0, "prompt": 32216, "tokens": 160, "ttft": 24.980390767, "prefill": 24.974338958, "wall": 30.702973934996407, "finish": "length"}} +{"i": 13, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 0, "prompt": 32216, "tokens": 160, "ttft": 24.86118151, "prefill": 24.856229291, "wall": 33.968799164984375, "finish": "length"}} +{"i": 14, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 0, "prompt": 35142, "tokens": 160, "ttft": 27.218896514, "prefill": 27.212124974, "wall": 32.760028827993665, "finish": "length"}} +{"i": 15, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 0, "prompt": 35142, "tokens": 160, "ttft": 26.986886645, "prefill": 26.981549481, "wall": 35.84385872699204, "finish": "length"}} +{"i": 16, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 37860, "tokens": 160, "ttft": 29.002025566, "prefill": 28.994446327, "wall": 31.707601928996155, "finish": "length"}} +{"i": 17, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 37860, "tokens": 160, "ttft": 28.974700521, "prefill": 28.968674938, "wall": 37.85622533300193, "finish": "length"}} +{"i": 18, "got": {"sha": "9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.765264539, "prefill": 1.763436561, "wall": 6.999669137003366, "finish": "length"}} +{"i": 19, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 0, "prompt": 7412, "tokens": 95, "ttft": 5.82895664, "prefill": 5.82694655, "wall": 8.512692087999312, "finish": "tool_calls"}} +{"i": 20, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 0, "prompt": 7412, "tokens": 95, "ttft": 5.785080229, "prefill": 5.783515598, "wall": 11.043854401010321, "finish": "tool_calls"}} +{"i": 21, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 0, "prompt": 7474, "tokens": 72, "ttft": 5.862744928, "prefill": 5.860903064, "wall": 7.756049646006431, "finish": "tool_calls"}} +{"i": 22, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 0, "prompt": 7474, "tokens": 72, "ttft": 5.809824917, "prefill": 5.808535886, "wall": 9.784674938011449, "finish": "tool_calls"}} +{"i": 23, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 0, "prompt": 12623, "tokens": 120, "ttft": 9.925508103, "prefill": 9.922131829, "wall": 13.874759544996778, "finish": "tool_calls"}} +{"i": 24, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 0, "prompt": 12623, "tokens": 120, "ttft": 9.901075449, "prefill": 9.899194369, "wall": 16.51369705799152, "finish": "tool_calls"}} +{"i": 25, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 0, "prompt": 12720, "tokens": 79, "ttft": 9.967764296, "prefill": 9.964587145, "wall": 12.725597810000181, "finish": "tool_calls"}} +{"i": 26, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 0, "prompt": 12720, "tokens": 79, "ttft": 9.931615924, "prefill": 9.929818635, "wall": 14.274520740989828, "finish": "tool_calls"}} +{"i": 27, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 0, "prompt": 18063, "tokens": 160, "ttft": 13.798787863, "prefill": 13.794594258, "wall": 19.3460255569953, "finish": "length"}} +{"i": 28, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 0, "prompt": 18063, "tokens": 160, "ttft": 13.735078065, "prefill": 13.732181938, "wall": 22.635835474997293, "finish": "length"}} +{"i": 29, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 0, "prompt": 15032, "tokens": 85, "ttft": 11.664898079, "prefill": 11.661158573, "wall": 14.592577686009463, "finish": "tool_calls"}} +{"i": 30, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 0, "prompt": 15032, "tokens": 85, "ttft": 11.6222499, "prefill": 11.61983496, "wall": 16.32443855900783, "finish": "tool_calls"}} +{"i": 31, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 40609, "tokens": 160, "ttft": 31.010276431, "prefill": 31.001180196, "wall": 33.37452480799402, "finish": "length"}} +{"i": 32, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 40609, "tokens": 160, "ttft": 30.937814677, "prefill": 30.931022005, "wall": 39.8927154619887, "finish": "length"}} +{"i": 33, "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.142929399, "prefill": 1.141160814, "wall": 6.130138580978382, "finish": "stop"}} +{"i": 34, "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.099356275, "prefill": 1.098412255, "wall": 9.587529818003532, "finish": "stop"}} +{"i": 35, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.600909408, "prefill": 3.598035555, "wall": 3.9084176199976355, "finish": "stop"}} +{"i": 36, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.575188711, "prefill": 3.573850786, "wall": 4.201100703998236, "finish": "stop"}} +{"i": 38, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 7.479690643, "prefill": 7.474008281, "wall": 10.221459418011364, "finish": "tool_calls"}} +{"i": 39, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 7.429225139, "prefill": 7.427438987, "wall": 11.978234272013651, "finish": "tool_calls"}} +{"i": 40, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 0, "prompt": 15152, "tokens": 160, "ttft": 11.757077255, "prefill": 11.753547991, "wall": 16.846801528008655, "finish": "length"}} +{"i": 41, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 0, "prompt": 15152, "tokens": 160, "ttft": 11.719868534, "prefill": 11.717764525, "wall": 20.55021308499272, "finish": "length"}} +{"i": 42, "got": {"sha": "eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c", "cached": 0, "prompt": 20872, "tokens": 160, "ttft": 30.67466344, "prefill": 29.83339564, "wall": 37.141429798008176, "finish": "length"}} +{"i": 43, "got": {"sha": "fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8", "cached": 0, "prompt": 17550, "tokens": 160, "ttft": 13.50312277, "prefill": 13.498639885, "wall": 36.76508772201487, "finish": "length"}} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/server-rank0.log new file mode 100644 index 0000000000..4b306e0167 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/server-rank0.log @@ -0,0 +1,231 @@ +[kvp-serve] 2026-10-22T15:57:44+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:18754:18754 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:18754:18754 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:18754:18754 [0] NCCL INFO cudaDriverVersion 13030 +host-d:18754:18754 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:18754:18754 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:18754:18754 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:18754:18754 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:18754:18754 [0] NCCL INFO Count not open libibverbs.so +host-d:18754:18754 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:18754:18754 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:18754:18754 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:18754:18754 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:18754:18754 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x18a9e810 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:18754:18754 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:18754:18754 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:18754:18754 [0] NCCL INFO Initialized NET plugin IB +host-d:18754:18754 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:18754:18754 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:18754:18754 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:18754:18754 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:18754:18754 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:18754:18754 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:18754:18754 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:18754:18754 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:18754:18754 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:18754:18754 [0] NCCL INFO Using network IB +host-d:18754:18754 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:18754:18754 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x15e6c7c0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:18754:18754 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:18754:18754 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:18754:18754 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:18754:18754 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:18754:18754 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:18754:18754 [0] NCCL INFO Rank 0: 1 Net devices +host-d:18754:18754 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:18754:18754 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:18754:18754 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:18754:18754 [0] NCCL INFO comm 0x15e6c7c0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:18754:18754 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:18754:18754 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:18754:18754 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:18754:18754 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:18754:18754 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:18754:18754 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:18754:18754 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:18754:18754 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:18754:18754 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:18754:18754 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:18754:18774 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-d:18754:18773 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-d:18754:18773 [0] NCCL INFO proxy listening socket at <46923> +host-d:18754:18754 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:18754:18754 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:18754:18775 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-d:18754:18754 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:18754:18754 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:18754:18754 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:18754:18754 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:18754:18754 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:18754:18754 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfa36c4010000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfa36c4054000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfa36c409c000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1675 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1675 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9c99 ctsFifoLkey=0x1b9c99 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1675 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1676 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1676 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ab2af ctsFifoLkey=0x1ab2af +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1676 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfa36c414e000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfa36c4196000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfa36c41da000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfa36c4222000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfa36c4266000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1675 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1677 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1678 pkey=0 pd=0xfa36c4053590 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1679 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1679 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b8986 ctsFifoLkey=0x1b8986 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1679 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1680 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1681 pkey=0 pd=0xfa36c4053590 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1682 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1682 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b6764 ctsFifoLkey=0x1b6764 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1682 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1683 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1684 pkey=0 pd=0xfa36c4053590 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1685 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1686 pkey=0 pd=0xfa36c4053590 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1676 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1679 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1682 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18754 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:18754:18754 [0] NCCL INFO Connected all trees +host-d:18754:18754 [0] NCCL INFO Connected binomial trees +host-d:18754:18754 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:18754:18754 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:18754:18754 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:18754:18754 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.5 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 304.7s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] done chatcmpl-323acff5145cc161ac1fc763e6583104 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=28 accepted=33/46 ttft=6.005 decode=2.001 elapsed=8.006 queued=0.000 tok/s=30.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1c176e7808ba68c59e182faff3a0b85d prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=5.953 decode=3.364 elapsed=9.317 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0ec03ee4292e03e35ed62b1f350813bb prompt=15550 cached=0 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=13 accepted=30/36 ttft=11.915 decode=1.060 elapsed=12.975 queued=0.000 tok/s=40.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-bc953dbd72eb17ebb1aa870563d94e25 prompt=15550 cached=0 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=43 accepted=0/0 ttft=11.902 decode=2.428 elapsed=14.330 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b9cc592a1affad8b04b7f7bfa7dc3be2 prompt=19460 cached=0 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=35 accepted=45/68 ttft=14.923 decode=2.582 elapsed=17.505 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7ff9862238e480a88a1f39427b1ade71 prompt=19460 cached=0 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=79 accepted=0/0 ttft=14.916 decode=4.414 elapsed=19.330 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4fe2022e4d38090d99e02517fe34ab05 prompt=21874 cached=0 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=70 accepted=78/124 ttft=16.724 decode=5.129 elapsed=21.854 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fd19189f89f12bb8f4dfb597405a92a8 prompt=21874 cached=0 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=147 accepted=0/0 ttft=16.665 decode=8.202 elapsed=24.867 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-0100bdc17765b607691b3272318ed239 prompt=26786 cached=0 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=72 accepted=88/126 ttft=20.654 decode=5.267 elapsed=25.921 queued=0.000 tok/s=30.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bd1a1e972e40e929886c428be3c1192f prompt=26786 cached=0 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=159 accepted=0/0 ttft=20.617 decode=8.877 elapsed=29.493 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-83e2d984f426ca414b4870eb956de769 prompt=29293 cached=0 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=73 accepted=86/129 ttft=22.503 decode=5.356 elapsed=27.859 queued=0.000 tok/s=29.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-acd5057767cc21531f65f9c8b1546d52 prompt=29293 cached=0 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=159 accepted=0/0 ttft=22.482 decode=9.077 elapsed=31.559 queued=0.000 tok/s=17.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-215252b3817d2b5aad9be30c2ae6ed0a prompt=32216 cached=0 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=78 accepted=82/126 ttft=24.980 decode=5.721 elapsed=30.701 queued=0.000 tok/s=27.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-6c86d70117b8980752df91204668dfed prompt=32216 cached=0 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=159 accepted=0/0 ttft=24.861 decode=9.105 elapsed=33.967 queued=0.000 tok/s=17.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-cd0cd812e6b6b819a5ed238db452ab1e prompt=35142 cached=0 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=78 accepted=82/128 ttft=27.219 decode=5.539 elapsed=32.758 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-d5f1570f22cf16cec209a878e39989e4 prompt=35142 cached=0 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=159 accepted=0/0 ttft=26.987 decode=8.855 elapsed=35.842 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-3e5029a6a948e8c1588087d607b4e762 prompt=37860 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=29 accepted=130/135 ttft=29.002 decode=2.703 elapsed=31.705 queued=0.000 tok/s=58.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-53d57d3021ffde72f639e977bdec8d0e prompt=37860 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=28.975 decode=8.879 elapsed=37.853 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-82c4fc4edf541cf5c8f2b36cef63d59e prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=9761cff1a208 finish=length rounds=70 accepted=89/135 ttft=1.765 decode=5.233 elapsed=6.998 queued=0.000 tok/s=30.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4f87401e0230e88a10ce83777fcca2af prompt=7412 cached=0 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=34 accepted=61/80 ttft=5.829 decode=2.682 elapsed=8.511 queued=0.000 tok/s=35.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e7f7fe42bc66039314e99a98e20406f3 prompt=7412 cached=0 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=94 accepted=0/0 ttft=5.785 decode=5.257 elapsed=11.042 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-170dfac8667ba4608f9fdbc6c2e8864f prompt=7474 cached=0 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=23 accepted=49/62 ttft=5.863 decode=1.892 elapsed=7.754 queued=0.000 tok/s=37.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-54635d05f1b1d674cfd232cd76669783 prompt=7474 cached=0 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=71 accepted=0/0 ttft=5.810 decode=3.973 elapsed=9.783 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fd57199245e6829e8f1e17227a3eea29 prompt=12623 cached=0 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=56 accepted=64/89 ttft=9.926 decode=3.948 elapsed=13.873 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-20aa3b95ea86f615581d858d27ad7d17 prompt=12623 cached=0 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=119 accepted=0/0 ttft=9.901 decode=6.611 elapsed=16.512 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-de093e8b6ac602e62bc2d86ae9931089 prompt=12720 cached=0 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=40 accepted=39/56 ttft=9.968 decode=2.756 elapsed=12.724 queued=0.000 tok/s=28.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fc98ac7c5cf2a1940584a18e0e8f12d1 prompt=12720 cached=0 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=78 accepted=0/0 ttft=9.932 decode=4.341 elapsed=14.273 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6446ee0b1335e6abf1f3722559c708b7 prompt=18063 cached=0 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=78 accepted=82/123 ttft=13.799 decode=5.545 elapsed=19.344 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e756cb3dbbdc5ae7a023b77a1e2969a7 prompt=18063 cached=0 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=159 accepted=0/0 ttft=13.735 decode=8.899 elapsed=22.634 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-873f135a2bd0ebc942c9d3eba7967a4d prompt=15032 cached=0 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=42 accepted=43/62 ttft=11.665 decode=2.926 elapsed=14.591 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fa18b94b08783e7d7e30f290575d43c8 prompt=15032 cached=0 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=84 accepted=0/0 ttft=11.622 decode=4.700 elapsed=16.323 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-d814619ebf063eab9bb642ca5cbf5ca6 prompt=40609 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=23 accepted=136/138 ttft=31.010 decode=2.362 elapsed=33.373 queued=0.000 tok/s=67.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-01c34b048cb21e15a4bf23fd02c4a8aa prompt=40609 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=30.938 decode=8.952 elapsed=39.890 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-97bf9b62ba9369e45c2eb0f25b0c6c7c prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=68 accepted=88/131 ttft=1.143 decode=4.986 elapsed=6.129 queued=0.000 tok/s=31.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-8cf5bf35ae884ff99e799fd1ca875547 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=155 accepted=0/0 ttft=1.099 decode=8.487 elapsed=9.586 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-05a0b4999b9fc14afe9ceca85b5f908e prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.601 decode=0.306 elapsed=3.907 queued=0.000 tok/s=36.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-029d92cf3999883e988324585d8635dd prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=3.575 decode=0.624 elapsed=4.199 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-125918e6482e34ec7d621b888f923ff1 prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=39 accepted=43/59 ttft=7.480 decode=2.740 elapsed=10.219 queued=0.000 tok/s=29.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-feb17f80cd828bf59e7bbc78ffcaee1b prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=81 accepted=0/0 ttft=7.429 decode=4.547 elapsed=11.976 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-5e2a91bafe0052ebb7ecbb3884d5a7e7 prompt=15152 cached=0 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=71 accepted=88/117 ttft=11.757 decode=5.088 elapsed=16.845 queued=0.000 tok/s=31.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-a1ba41caa03cb879a7fb19c082efef7d prompt=15152 cached=0 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=159 accepted=0/0 ttft=11.720 decode=8.828 elapsed=20.548 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-1c958c6c38dec5a3aeeb3bc5cb8ddd2d prompt=17550 cached=0 thinking=True effort=high tokens=160 sha=fe9efe1d4302 finish=length rounds=61 accepted=99/167 ttft=13.503 decode=23.261 elapsed=36.764 queued=0.000 tok/s=6.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-5d84d3037baebc9cba1fc34631aa5b66 prompt=20872 cached=0 thinking=True effort=high tokens=160 sha=eab19e6ff8fd finish=length rounds=54 accepted=105/161 ttft=30.675 decode=6.465 elapsed=37.140 queued=0.836 tok/s=24.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:18754:18754 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:18754:18754 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:18754:18754 [0] NCCL INFO ncclCommInitRank comm 0x15e6c7c0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:18754:18754 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.38 (kernels 0.24, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-d:18754:18754 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:18754:18754 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfa36c4450000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfa36c4491000) +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1687 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1687 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bbdba ctsFifoLkey=0x1bbdba +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1687 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1688 pkey=0 pd=0xfa36c4053590 oooRq=0 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1689 pkey=0 pd=0xfa36c4053590 +host-d:18754:18773 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1687 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:18754:18754 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/server-rank1.log new file mode 100644 index 0000000000..1cf24f698a --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2main/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T15:57:43+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:1822:1822 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:1822:1822 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:1822:1822 [0] NCCL INFO cudaDriverVersion 13030 +host-e:1822:1822 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:1822:1822 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:1822:1822 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:1822:1822 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:1822:1822 [0] NCCL INFO Count not open libibverbs.so +host-e:1822:1822 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:1822:1822 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:1822:1822 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:1822:1822 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:1822:1822 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x329367a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:1822:1822 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:1822:1822 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:1822:1822 [0] NCCL INFO Initialized NET plugin IB +host-e:1822:1822 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:1822:1822 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:1822:1822 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:1822:1822 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:1822:1822 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:1822:1822 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:1822:1822 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:1822:1822 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:1822:1822 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:1822:1822 [0] NCCL INFO Using network IB +host-e:1822:1822 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:1822:1822 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x2fd04970 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:1822:1822 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:1822:1822 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1822:1822 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1822:1822 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1822:1822 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:1822:1822 [0] NCCL INFO Rank 1: 1 Net devices +host-e:1822:1822 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:1822:1822 [0] NCCL INFO comm 0x2fd04970 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:1822:1822 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:1822:1822 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:1822:1822 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:1822:1822 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:1822:1822 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:1822:1822 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:1822:1835 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-e:1822:1834 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-e:1822:1834 [0] NCCL INFO proxy listening socket at <40957> +host-e:1822:1822 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1822:1836 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-e:1822:1822 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1822:1822 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1822:1822 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1822:1822 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1822:1822 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1822:1822 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1822:1822 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6206c010000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6206c054000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6206c09c000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6206c0e0000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6206c128000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6206c16c000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6206c1b4000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6206c1f8000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1419 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1420 pkey=0 pd=0xf6206c053eb0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1421 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1421 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16dd9e ctsFifoLkey=0x16dd9e +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1421 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1422 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1423 pkey=0 pd=0xf6206c053eb0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1424 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1424 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x14f1b0 ctsFifoLkey=0x14f1b0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1424 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1425 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1425 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1715d0 ctsFifoLkey=0x1715d0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1425 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1426 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1426 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1426 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1421 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1424 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1427 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1428 pkey=0 pd=0xf6206c053eb0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1425 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1429 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1430 pkey=0 pd=0xf6206c053eb0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1426 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1822 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:1822:1822 [0] NCCL INFO Connected all trees +host-e:1822:1822 [0] NCCL INFO Connected binomial trees +host-e:1822:1822 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:1822:1822 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:1822:1822 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:1822:1822 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:1822:1822 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:1822:1822 [0] NCCL INFO ncclCommInitRank comm 0x2fd04970 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:1822:1822 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.38 (kernels 0.24, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-e:1822:1822 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:1822:1822 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6206c448000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6206c489000) +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1431 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1431 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x14a461 ctsFifoLkey=0x14a461 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1431 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1432 pkey=0 pd=0xf6206c053eb0 oooRq=0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1433 pkey=0 pd=0xf6206c053eb0 +host-e:1822:1834 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1431 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1822:1822 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/client.log new file mode 100644 index 0000000000..79d55368b3 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/client.log @@ -0,0 +1,61 @@ +server up 2026-10-23 00:20:55 +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 39.2, "per_stream_tps": 39.2, "ttft_s_max": 0.15, "timing_valid": true, "end_to_end_tps": 38.45567859253743, "makespan_s": 6.657014240015997, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 45.2, "per_stream_tps": 25.0, "ttft_s_max": 0.36, "timing_valid": true, "end_to_end_tps": 44.787010803879134, "makespan_s": 11.431885960017098, "steady_tps": 48.5, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 60.8, "per_stream_tps": 17.0, "ttft_s_max": 0.9, "timing_valid": true, "end_to_end_tps": 60.48392132767589, "makespan_s": 16.930119237018516, "steady_tps": 65.5, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 29.1, "per_stream_tps": 29.1, "ttft_s_max": 0.22, "timing_valid": true, "end_to_end_tps": 28.52712743571779, "makespan_s": 8.973914411006263, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 40.1, "per_stream_tps": 20.5, "ttft_s_max": 0.5, "timing_valid": true, "end_to_end_tps": 39.621976647960935, "makespan_s": 12.922121593001066, "steady_tps": 40.9, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 50.8, "per_stream_tps": 13.4, "ttft_s_max": 1.18, "timing_valid": true, "end_to_end_tps": 50.44091556949408, "makespan_s": 20.300979640014702, "steady_tps": 53.6, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 56.1, "per_stream_tps": 56.1, "ttft_s_max": 0.14, "timing_valid": true, "end_to_end_tps": 54.53536226153051, "makespan_s": 4.694201878999593, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 63.9, "per_stream_tps": 32.5, "ttft_s_max": 0.37, "timing_valid": true, "end_to_end_tps": 62.95505745299879, "makespan_s": 8.132785843015881, "steady_tps": 64.8, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 87.0, "per_stream_tps": 22.6, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 86.21559456155363, "makespan_s": 11.877201627008617, "steady_tps": 90.6, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 30.6, "per_stream_tps": 30.6, "ttft_s_max": 0.22, "timing_valid": true, "end_to_end_tps": 29.93718500316345, "makespan_s": 8.551238200016087, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 43.8, "per_stream_tps": 22.4, "ttft_s_max": 0.51, "timing_valid": true, "end_to_end_tps": 43.17382772853212, "makespan_s": 11.859036526002456, "steady_tps": 44.8, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 58.5, "per_stream_tps": 15.1, "ttft_s_max": 1.17, "timing_valid": true, "end_to_end_tps": 58.05884809337646, "makespan_s": 17.637277239002287, "steady_tps": 61.2, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +logs/r2on-conc.json: 12 cells; alone 28/28 equal (0 unequal, 0 failed); serial 30/30 equal (0 unequal, 0 failed) + 0 S1 t1 drafted prompt 7759 history 7757 cached 0 (allowed [0]) ttft 6.02s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok + 1 S1 t1 plain prompt 7759 history 7757 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 1.24s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok +system cuts [4563, 6099, 6611] + 2 S1 t2 drafted prompt 15550 history 15548 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 7.19s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 tool_calls ok + 3 S1 t2 plain prompt 15550 history 15548 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 1.02s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 tool_calls ok + 4 S1 t3 drafted prompt 19460 history 19458 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 4.04s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 tool_calls ok + 5 S1 t3 plain prompt 19460 history 19458 cached 18432 (allowed {'min': 11266, 'max': 19458}) ttft 0.91s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 tool_calls ok + 6 S1 t4 drafted prompt 21874 history 21872 cached 18432 (allowed {'min': 11266, 'max': 19458}) ttft 2.72s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 tool_calls ok + 7 S1 t4 plain prompt 21874 history 21872 cached 20480 (allowed {'min': 13680, 'max': 21872}) ttft 1.12s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 tool_calls ok + 8 S1 t5 drafted prompt 26786 history 26784 cached 20480 (allowed {'min': 13680, 'max': 21872}) ttft 5.09s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c length ok + 9 S1 t5 plain prompt 26786 history 26784 cached 26624 (allowed {'min': 18592, 'max': 26784}) ttft 0.42s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c length ok + 10 S1 t6 drafted prompt 29293 history 29291 cached 26624 (allowed {'min': 18592, 'max': 26784}) ttft 2.30s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 length ok + 11 S1 t6 plain prompt 29293 history 29291 cached 28672 (allowed {'min': 21099, 'max': 29291}) ttft 0.70s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 length ok + 12 S1 t7 drafted prompt 32216 history 32214 cached 28672 (allowed {'min': 21099, 'max': 29291}) ttft 2.81s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e length ok + 13 S1 t7 plain prompt 32216 history 32214 cached 30720 (allowed {'min': 24022, 'max': 32214}) ttft 1.19s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e length ok + 14 S1 t8 drafted prompt 35142 history 35140 cached 30720 (allowed {'min': 24022, 'max': 32214}) ttft 3.70s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 length ok + 15 S1 t8 plain prompt 35142 history 35140 cached 34816 (allowed {'min': 26948, 'max': 35140}) ttft 0.55s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 length ok + 16 S1 t9 drafted prompt 37860 history 37858 cached 34816 (allowed {'min': 26948, 'max': 35140}) ttft 2.54s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 17 S1 t9 plain prompt 37860 history 37858 cached 36864 (allowed {'min': 29666, 'max': 37858}) ttft 0.93s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 18 X t1 drafted prompt 2115 history 2113 cached 0 (allowed [0]) ttft 1.76s sha 9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd length ok + 19 A t1 drafted prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 1.09s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 tool_calls ok + 20 A t1 plain prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 1.05s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 tool_calls ok + 21 B t1 drafted prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 1.13s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a tool_calls ok + 22 B t1 plain prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 1.08s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a tool_calls ok + 23 A t2 drafted prompt 12623 history 12621 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 5.16s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 tool_calls ok + 24 A t2 plain prompt 12623 history 12621 cached 12288 (allowed {'min': 4429, 'max': 12621}) ttft 0.55s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 tool_calls ok + 25 B t2 drafted prompt 12720 history 12718 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 5.22s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c tool_calls ok + 26 B t2 plain prompt 12720 history 12718 cached 12288 (allowed {'min': 4526, 'max': 12718}) ttft 0.59s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c tool_calls ok + 27 A t3 drafted prompt 18063 history 18061 cached 12288 (allowed {'min': 4429, 'max': 12621}) ttft 4.43s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc length ok + 28 A t3 plain prompt 18063 history 18061 cached 16384 (allowed {'min': 9869, 'max': 18061}) ttft 1.28s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc length ok + 29 B t3 drafted prompt 15032 history 15030 cached 12288 (allowed {'min': 4526, 'max': 12718}) ttft 2.31s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa tool_calls ok + 30 B t3 plain prompt 15032 history 15030 cached 14336 (allowed {'min': 6838, 'max': 15030}) ttft 0.74s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa tool_calls ok + 31 E t1 drafted prompt 40609 history 40607 cached 6144 (allowed {'min': 0, 'max': 6611}) ttft 26.11s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 32 E t1 plain prompt 40609 history 40607 cached 38912 (allowed {'min': 32415, 'max': 40607}) ttft 1.32s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 33 T t1 drafted prompt 1520 history 1517 cached 0 (allowed [0]) ttft 1.14s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 stop ok + 34 T t1 plain prompt 1520 history 1517 cached 0 (allowed [0]) ttft 1.09s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 stop ok + 35 T t2 drafted prompt 4208 history 4205 cached 0 (allowed [0]) ttft 3.57s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 36 T t2 plain prompt 4208 history 4205 cached 4096 (allowed {'min': 1, 'max': 4205}) ttft 0.37s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 37 C t1 cancel prompt None history 0 cached None (allowed None) ttft 0.00s sha None None ok + 38 C t1 drafted prompt 9702 history 9700 cached 0 (allowed {'min': 0, 'max': 9688}) ttft 8.17s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 tool_calls ok + 39 C t1 plain prompt 9702 history 9700 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 1.18s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 tool_calls ok + 40 C t2 drafted prompt 15152 history 15150 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 5.46s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 length ok + 41 C t2 plain prompt 15152 history 15150 cached 14336 (allowed {'min': 6958, 'max': 15150}) ttft 0.82s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 length ok + 42 A t4 drafted prompt 20872 history 20870 cached 16384 (allowed {'min': 9869, 'max': 18061}) ttft 6.54s sha eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c length ok + 43 B t4 drafted prompt 17550 history 17548 cached 14336 (allowed {'min': 6838, 'max': 15030}) ttft 2.60s sha fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8 length ok +record: 44 requests, 0 not cached as planned +=== kvp-client r2on rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/conc.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/conc.json new file mode 100644 index 0000000000..c10e053160 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/conc.json @@ -0,0 +1,4589 @@ +{ + "label": "", + "tokens": 256, + "mixed": false, + "cells": [ + { + "prompt": "code", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.2, + "per_stream_tps": 39.2, + "ttft_s_max": 0.15, + "timing_valid": true, + "end_to_end_tps": 38.45567859253743, + "makespan_s": 6.657014240015997, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.16525906255894, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 6.657014240015997, + "end_to_end_tps": 38.45567859253743, + "timing_valid": true, + "per_stream_tps": [ + 39.2 + ], + "ttft_s": [ + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + } + ], + "client_intervals": [ + { + "start": 134163.21008768, + "first": 134163.356125291, + "last": 134169.866997582, + "end": 134169.86710192, + "elapsed_s": 6.657014240015997, + "chunk_offsets_s": [ + 0.1460376110044308, + 0.21943263299181126, + 0.29132671299157664, + 0.36726663101580925, + 0.44958038500044495, + 0.53556585800834, + 0.6142033170035575, + 0.6993891099991743, + 0.7821810170134995, + 0.8647278950084001, + 0.946101854002336, + 1.0306877630064264, + 1.110156709997682, + 1.192567409016192, + 1.2666823799954727, + 1.3417683290026616, + 1.4176292380143423, + 1.4961582779942546, + 1.5821080380119383, + 1.6667604919930454, + 1.7530044180166442, + 1.835436829016544, + 1.9086782940139528, + 1.9799971780157648, + 2.051780312001938, + 2.12886305100983, + 2.2098130020021927, + 2.2987308189913165, + 2.383833361993311, + 2.4676626640139148, + 2.5499989450036082, + 2.634415298991371, + 2.718615999998292, + 2.8037206059962045, + 2.88621602699277, + 2.9698653730156366, + 3.0547078140079975, + 3.142226660012966, + 3.2266524209990166, + 3.3132182720000856, + 3.402304635004839, + 3.491792318993248, + 3.577534681011457, + 3.662030172010418, + 3.744975023990264, + 3.8280646789935417, + 3.911670839996077, + 3.993791707995115, + 4.075352998013841, + 4.1574751299922355, + 4.239222295000218, + 4.3215894880122505, + 4.407756083004642, + 4.48705987300491, + 4.558564759994624, + 4.632142262998968, + 4.708912355010398, + 4.795242424006574, + 4.884711962018628, + 4.967828385008033, + 5.050202314014314, + 5.15108408901142, + 5.260778866009787, + 5.347146760002943, + 5.429955928993877, + 5.513450775004458, + 5.600956675014459, + 5.690427316993009, + 5.7782739590038545, + 5.866117272991687, + 5.945069039007649, + 6.026944092998747, + 6.114137202996062, + 6.1972787459963, + 6.2831968709942885, + 6.364000800997019, + 6.449096221011132, + 6.52449446299579, + 6.595338456012541, + 6.656909901998006 + ], + "chunk_gaps_ms": { + "count": 79, + "p50": 82.94485197984613, + "p95": 89.4706419785507, + "max": 109.69477699836716 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 45.2, + "per_stream_tps": 25.0, + "ttft_s_max": 0.36, + "timing_valid": true, + "end_to_end_tps": 44.787010803879134, + "makespan_s": 11.431885960017098, + "steady_tps": 48.5, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 45.19701631584876, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 11.431885960017098, + "end_to_end_tps": 44.787010803879134, + "timing_valid": true, + "steady_tps": 48.49784402241219, + "per_stream_tps": [ + 27.3, + 22.6 + ], + "ttft_s": [ + 0.36, + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + } + ], + "client_intervals": [ + { + "start": 134169.869887382, + "first": 134170.23345596, + "last": 134179.569893158, + "end": 134179.570089578, + "elapsed_s": 9.70020219599246, + "chunk_offsets_s": [ + 0.36356857800274156, + 0.4650407299923245, + 0.5806290440086741, + 0.6992105990066193, + 0.8183001010038424, + 0.933628744998714, + 1.045779392006807, + 1.1669507090118714, + 1.2826686399930622, + 1.3990224080043845, + 1.519433134002611, + 1.639957573992433, + 1.7586262510158122, + 1.8774477940169163, + 1.9958390569954645, + 2.1156725400069263, + 2.236171202996047, + 2.355644005991053, + 2.47627537601511, + 2.5947068149980623, + 2.7170706180040725, + 2.836207062995527, + 2.9566210040065926, + 3.075840937992325, + 3.1998387560015544, + 3.3266262440010905, + 3.4496453070023563, + 3.5693115050089546, + 3.6911758260102943, + 3.8124739280028734, + 3.934895892016357, + 4.056149112002458, + 4.172671041014837, + 4.2893387009971775, + 4.40881996700773, + 4.531427693989826, + 4.648988699016627, + 4.769012806995306, + 4.8952475800178945, + 5.019859184016241, + 5.139536822010996, + 5.261221266002394, + 5.3820587169902865, + 5.505273607006529, + 5.629012730001705, + 5.7486796640150715, + 5.871764918992994, + 5.9963150169933215, + 6.112876738014165, + 6.233795453998027, + 6.356164743017871, + 6.470256847998826, + 6.589147029008018, + 6.712724101002095, + 6.83308567100903, + 6.956078715011245, + 7.082159370998852, + 7.2013726939912885, + 7.319976133003365, + 7.434821611008374, + 7.5479792339901906, + 7.667292191006709, + 7.789402672991855, + 7.91260602499824, + 8.034689738997258, + 8.159173387015471, + 8.284745328011923, + 8.403908409003634, + 8.53141995499027, + 8.650014913990162, + 8.771799150010338, + 8.893738220009254, + 9.014315681008156, + 9.133522969990736, + 9.251522444013972, + 9.37490968601196, + 9.491338730003918, + 9.595480048999889, + 9.700005776016042 + ], + "chunk_gaps_ms": { + "count": 78, + "p50": 120.19283899280708, + "p95": 126.08065598760732, + "max": 127.51154598663561 + } + }, + { + "start": 134169.869478638, + "first": 134170.017008545, + "last": 134181.300939246, + "end": 134181.301364598, + "elapsed_s": 11.431885960017098, + "chunk_offsets_s": [ + 0.14752990700071678, + 0.22010785300517455, + 0.4651921089971438, + 0.5808787129935808, + 0.6997842590208165, + 0.8185884740087204, + 0.9342022610071581, + 1.0461515430069994, + 1.1675323680101428, + 1.282969766005408, + 1.399398334993748, + 1.5200092410086654, + 1.6405335220042616, + 1.7592009979998693, + 1.8778928749961779, + 1.9961000379989855, + 2.1161479570146184, + 2.2367658860166557, + 2.3562004810082726, + 2.4767784569994546, + 2.5952126329939347, + 2.717740231018979, + 2.8368283070158213, + 2.957244968012674, + 3.076464757992653, + 3.2003488460031804, + 3.3271504460135475, + 3.450245543004712, + 3.569885164994048, + 3.691757566004526, + 3.813015843014, + 3.9353980779997073, + 4.05664950600476, + 4.173367342998972, + 4.289887240011012, + 4.409325768996496, + 4.531930791010382, + 4.649531557020964, + 4.769682836020365, + 4.895923289004713, + 5.0203798029979225, + 5.1401668500038795, + 5.261766732000979, + 5.382712762017036, + 5.505895811016671, + 5.629538325010799, + 5.749182969011599, + 5.872267616010504, + 5.997061432019109, + 6.113379340007668, + 6.234455963014625, + 6.356740626011742, + 6.470890268014045, + 6.589648830995429, + 6.713360849011224, + 6.833674562018132, + 6.9566951589949895, + 7.08266826000181, + 7.201875279017258, + 7.320582033018582, + 7.435336326016113, + 7.548483947000932, + 7.667795864021173, + 7.789921627001604, + 7.91311733101611, + 8.035235590010416, + 8.159874056000262, + 8.285409533011261, + 8.404560246010078, + 8.532133841014002, + 8.650517322996166, + 8.772367337020114, + 8.89435700001195, + 9.014890540012857, + 9.134049123997102, + 9.252191481005866, + 9.37548083299771, + 9.491869621007936, + 9.596006412000861, + 9.700586827006191, + 9.778642940014834, + 9.844937576999655, + 9.907583168998826, + 9.972396066004876, + 10.037657834996935, + 10.103465566993691, + 10.176437693007756, + 10.251149629009888, + 10.32280822499888, + 10.388614789000712, + 10.455942917993525, + 10.530681014002766, + 10.606185493001249, + 10.680263737012865, + 10.75216526602162, + 10.81662514799973, + 10.883890555996913, + 10.958775503007928, + 11.03665892299614, + 11.115265589003684, + 11.196032761014067, + 11.280163709016051, + 11.36022243500338, + 11.431460608000634 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 119.15071299881674, + "p95": 125.53547701099887, + "max": 245.08425599196926 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 60.8, + "per_stream_tps": 17.0, + "ttft_s_max": 0.9, + "timing_valid": true, + "end_to_end_tps": 60.48392132767589, + "makespan_s": 16.930119237018516, + "steady_tps": 65.5, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 60.775331104981774, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 16.930119237018516, + "end_to_end_tps": 60.48392132767589, + "timing_valid": true, + "steady_tps": 65.48386419091553, + "per_stream_tps": [ + 17.7, + 15.6, + 16.4, + 17.7 + ], + "ttft_s": [ + 0.15, + 0.61, + 0.37, + 0.9 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6c97574b6d3c", + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692" + } + ], + "client_intervals": [ + { + "start": 134181.303882598, + "first": 134181.45067141, + "last": 134195.87603334, + "end": 134195.876172687, + "elapsed_s": 14.572290089010494, + "chunk_offsets_s": [ + 0.1467888120096177, + 0.22252681400277652, + 0.4680099590041209, + 0.7544403869833332, + 1.0685617959825322, + 1.2455399399914313, + 1.4076274929975625, + 1.5799282180087175, + 1.7475785499846097, + 1.9189310960064176, + 2.0902441540092696, + 2.261520219006343, + 2.436252766987309, + 2.6217956339824013, + 2.805370607005898, + 2.9866402559855487, + 3.1648652699950617, + 3.3446966189949308, + 3.526545829983661, + 3.7110917170066386, + 3.8976774030015804, + 4.07874432700919, + 4.2626197839854285, + 4.44457019699621, + 4.623270705982577, + 4.809607793984469, + 4.993645285983803, + 5.182803932984825, + 5.370567256002687, + 5.554126433999045, + 5.74022959699505, + 5.932646016997751, + 6.115648031991441, + 6.298394154000562, + 6.486798559984891, + 6.67130681098206, + 6.854542974004289, + 7.035922635986935, + 7.224324370006798, + 7.409480809001252, + 7.59755309598404, + 7.787989428004948, + 7.978723781008739, + 8.161399515985977, + 8.344323670986341, + 8.527744908002205, + 8.710397232993273, + 8.898280906985747, + 9.081599692988675, + 9.264836700982414, + 9.454162146983435, + 9.632336384995142, + 9.816736679000314, + 10.000234139006352, + 10.18030305398861, + 10.367125329998089, + 10.554965657007415, + 10.738757377985166, + 10.914253099996131, + 11.094000887998845, + 11.274936409987276, + 11.462264198984485, + 11.650893276004354, + 11.836669737997, + 12.01650758998585, + 12.201406290987507, + 12.39852716299356, + 12.589203206996899, + 12.778065022983355, + 12.963019707996864, + 13.140237829007674, + 13.327182120992802, + 13.504168844985543, + 13.686775228008628, + 13.86498983099591, + 14.048508567997487, + 14.22829529698356, + 14.40124528700835, + 14.572150741994847 + ], + "chunk_gaps_ms": { + "count": 78, + "p50": 183.4593485109508, + "p95": 197.1208720060531, + "max": 314.12140899919905 + } + }, + { + "start": 134181.304877945, + "first": 134181.917650308, + "last": 134198.233796758, + "end": 134198.233907529, + "elapsed_s": 16.92902958401828, + "chunk_offsets_s": [ + 0.612772363005206, + 0.7533999030129053, + 1.067905703006545, + 1.244644418999087, + 1.4065205439983401, + 1.579092906002188, + 1.7467199730162974, + 1.9179025490011554, + 2.089211303013144, + 2.260697452002205, + 2.435226012021303, + 2.6207201900251675, + 2.8046177930082195, + 2.9859651710139588, + 3.1643510200083256, + 3.3439409240090754, + 3.525441441015573, + 3.7100616650131997, + 3.8968880920147058, + 4.077715891005937, + 4.261520387022756, + 4.4435398890054785, + 4.622208542015869, + 4.808542654005578, + 4.992684420023579, + 5.181923948024632, + 5.369535268022446, + 5.55324873799691, + 5.739129720022902, + 5.931885282014264, + 6.114806527999463, + 6.297582377999788, + 6.4856440580042545, + 6.670043810998322, + 6.8533547109982464, + 7.03476771802525, + 7.223605908016907, + 7.408385028014891, + 7.596675655018771, + 7.7870992830139585, + 7.9776221600186545, + 8.160353656014195, + 8.343292692006798, + 8.526713800005382, + 8.709364334004931, + 8.89722133500618, + 9.080545065022307, + 9.263806842005579, + 9.453091789997416, + 9.631240012007765, + 9.815892871003598, + 9.999205655010883, + 10.179545710998354, + 10.366048525000224, + 10.554164282017155, + 10.737665582011687, + 10.913221527996939, + 11.092971220001346, + 11.274109322024742, + 11.46123363499646, + 11.650042363005923, + 11.835470051009906, + 12.015809593023732, + 12.200374703999842, + 12.397499432001496, + 12.588142913999036, + 12.776998075016309, + 12.961917943001026, + 13.139203985017957, + 13.326136245013913, + 13.503565970022464, + 13.686162753001554, + 13.864112215000205, + 14.047851644019829, + 14.227238988998579, + 14.400115474010818, + 14.571025040000677, + 14.726238641014788, + 14.876181389001431, + 15.030712273000972, + 15.18125992800924, + 15.32203336400562, + 15.45024047899642, + 15.571553878020495, + 15.68484056199668, + 15.804127473005792, + 15.91097992201685, + 15.991745406005066, + 16.057222278002882, + 16.1253128790122, + 16.19984938102425, + 16.275318268017145, + 16.346543249004753, + 16.424409110011766, + 16.510992705996614, + 16.601564427022822, + 16.691891150025185, + 16.777815869019832, + 16.857711288001155, + 16.92891881300602 + ], + "chunk_gaps_ms": { + "count": 99, + "p50": 181.4130070270039, + "p95": 190.52287700469606, + "max": 314.5057999936398 + } + }, + { + "start": 134181.304515666, + "first": 134181.670714144, + "last": 134197.216022446, + "end": 134197.216101631, + "elapsed_s": 15.911585964990081, + "chunk_offsets_s": [ + 0.3661984780046623, + 0.46733906600275077, + 0.7539341049850918, + 1.068100570992101, + 1.2446605149889365, + 1.4073748319933657, + 1.5796167879889254, + 1.7472415349911898, + 1.9187847419816535, + 2.0898064659850206, + 2.260851550992811, + 2.4354245429858565, + 2.620920897985343, + 2.8047745479852892, + 2.98597027498181, + 3.1643165549903642, + 3.3441059839969967, + 3.5259986839955673, + 3.710511865996523, + 3.897319299983792, + 4.078150202985853, + 4.261919882992515, + 4.443968121981015, + 4.622606293996796, + 4.80894096600241, + 4.992975497996667, + 5.18221376099973, + 5.369966668979032, + 5.553444997989573, + 5.73956027199165, + 5.932047924987273, + 6.115201574983075, + 6.2979786259820685, + 6.486229157977505, + 6.670708607998677, + 6.853405511996243, + 7.035355009982595, + 7.22364382899832, + 7.408782363985665, + 7.596986508986447, + 7.787389368982986, + 7.978053608996561, + 8.160797377000563, + 8.343723643978592, + 8.527145280997502, + 8.70984701497946, + 8.897617821989115, + 9.080716828000732, + 9.264237553987186, + 9.4535633350024, + 9.631667412002571, + 9.816069177992176, + 9.999688224983402, + 10.179705393995391, + 10.366250560997287, + 10.554118152998853, + 10.738062357006129, + 10.913654736999888, + 11.09340761200292, + 11.274238779995358, + 11.461697580001783, + 11.65017580598942, + 11.836148639995372, + 12.016209071996855, + 12.20080354399397, + 12.397649082005955, + 12.588348469987977, + 12.777479540003696, + 12.96242024100502, + 13.13964832999045, + 13.326685742998961, + 13.503962344984757, + 13.686194993002573, + 13.864523102995008, + 14.047918236989062, + 14.22769762200187, + 14.400577242980944, + 14.571472312993137, + 14.726658760977443, + 14.876351440005237, + 15.031320045003667, + 15.181683553004405, + 15.322549661999801, + 15.450691623991588, + 15.572041039005853, + 15.685334042995237, + 15.804587497987086, + 15.911506779986667 + ], + "chunk_gaps_ms": { + "count": 87, + "p50": 182.77705099899322, + "p95": 190.69938798202202, + "max": 314.16646600700915 + } + }, + { + "start": 134181.303788292, + "first": 134182.203660663, + "last": 134196.627217107, + "end": 134196.627417671, + "elapsed_s": 15.323629379010526, + "chunk_offsets_s": [ + 0.899872371024685, + 1.0687041350174695, + 1.2456653980189003, + 1.4078779300034512, + 1.580245665012626, + 1.7477202489972115, + 1.9192490710120182, + 2.090497615019558, + 2.261657886003377, + 2.4361138210224453, + 2.62185770002543, + 2.805536195024615, + 2.986892261018511, + 3.1652184289996512, + 3.3448689260112587, + 3.526687336998293, + 3.7112686170148663, + 3.8980120810156222, + 4.0789090980251785, + 4.262680842017289, + 4.444763081002748, + 4.623399557021912, + 4.809737493022112, + 4.993812906002859, + 5.183066833997145, + 5.37073013902409, + 5.554304310004227, + 5.740254142001504, + 5.932807988021523, + 6.115994407009566, + 6.298522252007388, + 6.4869252830103505, + 6.671169289009413, + 6.854307001020061, + 7.0360503350093495, + 7.224452357011614, + 7.409541355009424, + 7.597682475025067, + 7.787896434019785, + 7.978748534020269, + 8.161594032018911, + 8.344485195004381, + 8.527907151001273, + 8.71052070800215, + 8.898406622000039, + 9.081728912016843, + 9.265002305008238, + 9.454222868022043, + 9.632362738018855, + 9.817072141013341, + 10.000371518020984, + 10.180750502011506, + 10.36717845100793, + 10.55513314899872, + 10.738818164012628, + 10.914418111002306, + 11.094177595019573, + 11.274999355024192, + 11.462389369000448, + 11.651051455002744, + 11.836809052998433, + 12.01702337601455, + 12.201566197996726, + 12.398442330013495, + 12.589265448012156, + 12.77831920402241, + 12.963061437010765, + 13.14042146501015, + 13.327378797024721, + 13.50449598801788, + 13.687075121997623, + 13.865137370012235, + 14.049097923998488, + 14.228286305005895, + 14.401173734018812, + 14.571949522010982, + 14.727172451006481, + 14.87696583601064, + 15.031955256999936, + 15.182091128022876, + 15.323428814997897 + ], + "chunk_gaps_ms": { + "count": 80, + "p50": 183.16206549934577, + "p95": 190.21395899471827, + "max": 196.87613201676868 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 29.1, + "per_stream_tps": 29.1, + "ttft_s_max": 0.22, + "timing_valid": true, + "end_to_end_tps": 28.52712743571779, + "makespan_s": 8.973914411006263, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 29.11765839178817, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 8.973914411006263, + "end_to_end_tps": 28.52712743571779, + "timing_valid": true, + "per_stream_tps": [ + 29.1 + ], + "ttft_s": [ + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + } + ], + "client_intervals": [ + { + "start": 134293.166810498, + "first": 134293.382773481, + "last": 134302.14034583, + "end": 134302.140724909, + "elapsed_s": 8.973914411006263, + "chunk_offsets_s": [ + 0.2159629829984624, + 0.29515692801214755, + 0.36961470401729457, + 0.4345667349989526, + 0.5025475900038145, + 0.5749285430065356, + 0.6398131330206525, + 0.7050427530193701, + 0.7703110610018484, + 0.8365693870000541, + 0.9001128480012994, + 0.9643269600055646, + 1.0289408400130924, + 1.0938735420058947, + 1.157880531012779, + 1.2217709090036806, + 1.2889854460081551, + 1.3654082030116115, + 1.4414131760131568, + 1.5147357780078892, + 1.590496395016089, + 1.6800375090097077, + 1.7639851900166832, + 1.842449393006973, + 1.9060437490115874, + 1.9700741620035842, + 2.0325269770110026, + 2.0973383969976567, + 2.163448592007626, + 2.240663172007771, + 2.3138068430125713, + 2.387363850022666, + 2.4612428469990846, + 2.5457786030019633, + 2.6316964169964194, + 2.7125558800180443, + 2.799120554002002, + 2.8809346579946578, + 2.965568304003682, + 3.0414534660230856, + 3.115404832002241, + 3.188896236999426, + 3.2639762480102945, + 3.3479522340057883, + 3.4302182670217007, + 3.5136731220118236, + 3.581709882011637, + 3.658324273012113, + 3.732663661998231, + 3.8067157820041757, + 3.88536942700739, + 3.9661451920110267, + 4.033064779010601, + 4.104797591018723, + 4.169156457006466, + 4.232756909012096, + 4.29870503701386, + 4.375254531012615, + 4.4473777990206145, + 4.522963739000261, + 4.595699113997398, + 4.66043165102019, + 4.728524395002751, + 4.802090377022978, + 4.875004795001587, + 4.949867873016046, + 5.02520995301893, + 5.114239904010901, + 5.191226862021722, + 5.257959772017784, + 5.331060849013738, + 5.39574396199896, + 5.458862357016187, + 5.523889586998848, + 5.589599854021799, + 5.654536875023041, + 5.720265655021649, + 5.784796171996277, + 5.851409896014957, + 5.915719481010456, + 5.980345503019635, + 6.043836433003889, + 6.106149661005475, + 6.170167848002166, + 6.23580099400715, + 6.3069363060058095, + 6.382554342009826, + 6.457809763000114, + 6.535979078995297, + 6.615326865023235, + 6.6802273410139605, + 6.745296875014901, + 6.811720035009785, + 6.8763585060078185, + 6.942515885020839, + 7.016967723000562, + 7.090952097001718, + 7.166816889017355, + 7.24068102001911, + 7.3033204289968126, + 7.368629008997232, + 7.432881200016709, + 7.497703994013136, + 7.561166539002443, + 7.6262312100152485, + 7.692176712997025, + 7.757360713003436, + 7.820705240010284, + 7.885321214009309, + 7.950391084013972, + 8.013884141022572, + 8.078244447009638, + 8.14170628701686, + 8.210906315012835, + 8.28738825401524, + 8.36224523501005, + 8.43454781701439, + 8.51114475101349, + 8.59090899201692, + 8.656270931009203, + 8.720259165012976, + 8.784423258999595, + 8.849638651998248, + 8.912861335003981, + 8.973535332013853 + ], + "chunk_gaps_ms": { + "count": 124, + "p50": 67.59769600466825, + "p95": 83.97598599549383, + "max": 89.54111399361864 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 40.1, + "per_stream_tps": 20.5, + "ttft_s_max": 0.5, + "timing_valid": true, + "end_to_end_tps": 39.621976647960935, + "makespan_s": 12.922121593001066, + "steady_tps": 40.9, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 40.13352961356805, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 12.922121593001066, + "end_to_end_tps": 39.621976647960935, + "timing_valid": true, + "steady_tps": 40.895576308752766, + "per_stream_tps": [ + 20.9, + 20.1 + ], + "ttft_s": [ + 0.5, + 0.21 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + } + ], + "client_intervals": [ + { + "start": 134302.14248643, + "first": 134302.646430762, + "last": 134314.858351114, + "end": 134314.858457868, + "elapsed_s": 12.715971437981352, + "chunk_offsets_s": [ + 0.5039443319838028, + 0.6110139449883718, + 0.7304796619864646, + 0.8500749819795601, + 0.9680989279877394, + 1.0863116609980352, + 1.202535172982607, + 1.318420822994085, + 1.4353723009990063, + 1.5547085590078495, + 1.6762994999880902, + 1.7981490999809466, + 1.9152008679811843, + 2.0376480630075093, + 2.156056607986102, + 2.2737207869940903, + 2.396654359996319, + 2.518252276000567, + 2.6361782510066405, + 2.754922528984025, + 2.879152317997068, + 2.9964390649984125, + 3.1138475259940606, + 3.23890929800109, + 3.3620409870054573, + 3.485786414006725, + 3.608800819987664, + 3.731715351983439, + 3.849603340990143, + 3.969846014981158, + 4.091401081008371, + 4.216759625996929, + 4.335534929006826, + 4.455307673983043, + 4.575377016008133, + 4.686822582007153, + 4.800341448979452, + 4.919451588997617, + 5.041495143988868, + 5.15973521998967, + 5.275857703993097, + 5.395843076985329, + 5.516694672987796, + 5.639043817995116, + 5.765216248983052, + 5.88387764399522, + 5.998544532980304, + 6.117908084997907, + 6.241559716989286, + 6.36108274399885, + 6.480134274985176, + 6.598481631983304, + 6.7273781139811035, + 6.850985439989017, + 6.975082791992463, + 7.094786381989252, + 7.218483311007731, + 7.341873514000326, + 7.458986015000846, + 7.579397714987863, + 7.70240672599175, + 7.823881756979972, + 7.946936736989301, + 8.067998399987118, + 8.185781281004893, + 8.303637459990568, + 8.424840838008095, + 8.546010263002245, + 8.667224920005538, + 8.792730504996143, + 8.911019443999976, + 9.034405326005071, + 9.156236729992088, + 9.283296248002443, + 9.405943443998694, + 9.5282443939941, + 9.649224566004705, + 9.768603156990139, + 9.88544774800539, + 10.009931649983628, + 10.130925373989157, + 10.247745964996284, + 10.366477006988134, + 10.483949226007098, + 10.60236298298696, + 10.721473023993894, + 10.83943970000837, + 10.957501576980576, + 11.075049748993479, + 11.195978735981043, + 11.317040349007584, + 11.441028736997396, + 11.564482635003515, + 11.685815645003458, + 11.807013723999262, + 11.923489827982849, + 12.04300025900011, + 12.161250748002203, + 12.282021186983911, + 12.402334865997545, + 12.512501795979915, + 12.614718786004232, + 12.715864683996188 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 119.87905898422468, + "p95": 125.0617720070295, + "max": 128.89648199779913 + } + }, + { + "start": 134302.142389772, + "first": 134302.356615561, + "last": 134315.064194608, + "end": 134315.064511365, + "elapsed_s": 12.922121593001066, + "chunk_offsets_s": [ + 0.21422578900819644, + 0.29173078801250085, + 0.6109192080039065, + 0.7304561900091358, + 0.8499964200018439, + 0.9680752950080205, + 1.0865246090106666, + 1.2025051410018932, + 1.318358246004209, + 1.435664787015412, + 1.554968579992419, + 1.6762349549971987, + 1.7982081740046851, + 1.9152061479981057, + 2.0377112650021445, + 2.1561188010091428, + 2.273871814017184, + 2.396676792006474, + 2.518444999004714, + 2.636239804007346, + 2.7549819870037027, + 2.8792076950194314, + 2.9965690360113513, + 3.1139783450053073, + 3.23897328399471, + 3.3620833879977, + 3.48584807899897, + 3.6089333510026336, + 3.731850937998388, + 3.8497344960051123, + 3.969901584001491, + 4.091389448993141, + 4.216823020018637, + 4.335585346008884, + 4.455370587005746, + 4.575435417005792, + 4.686882983020041, + 4.800402907014359, + 4.9195842320041265, + 5.041560601996025, + 5.159781957016094, + 5.275989787012804, + 5.396011575998273, + 5.516756355005782, + 5.639104683010373, + 5.765276634017937, + 5.884190482000122, + 5.998600710008759, + 6.117968566017225, + 6.241623831010656, + 6.36114524200093, + 6.480181620019721, + 6.598544177017175, + 6.7276498150022235, + 6.85116898000706, + 6.975147257995559, + 7.095046243019169, + 7.21854748800979, + 7.341934971016599, + 7.459116946003633, + 7.579459636006504, + 7.702536169002997, + 7.824013039004058, + 7.947002466011327, + 8.06813256200985, + 8.18584085101611, + 8.303699254000094, + 8.424972937005805, + 8.546075048012426, + 8.667352171003586, + 8.792792251013452, + 8.911080068995943, + 9.034464750991901, + 9.156374365004012, + 9.283426011010306, + 9.406075303006219, + 9.52830405900022, + 9.649358137015952, + 9.768666278017918, + 9.885704073007219, + 10.010192967019975, + 10.130987968004774, + 10.247920785011956, + 10.366732291993685, + 10.484082412993303, + 10.602422519994434, + 10.721607843006495, + 10.839504021016182, + 10.957565595017513, + 11.07519763999153, + 11.19612110700109, + 11.31719931200496, + 11.44108811399201, + 11.564725631003967, + 11.6861448190175, + 11.807057772995904, + 11.92374799301615, + 12.043061093019787, + 12.161311884992756, + 12.282158054003958, + 12.402398386999266, + 12.51256650101277, + 12.614782579999883, + 12.715996495011495, + 12.796449559013126, + 12.861177292012144, + 12.921804836019874 + ], + "chunk_gaps_ms": { + "count": 106, + "p50": 119.84211301023606, + "p95": 125.43357102549635, + "max": 319.1884199914057 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.8, + "per_stream_tps": 13.4, + "ttft_s_max": 1.18, + "timing_valid": true, + "end_to_end_tps": 50.44091556949408, + "makespan_s": 20.300979640014702, + "steady_tps": 53.6, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.7813216665391, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 20.300979640014702, + "end_to_end_tps": 50.44091556949408, + "timing_valid": true, + "steady_tps": 53.55454767363588, + "per_stream_tps": [ + 13.7, + 13.2, + 13.9, + 13.1 + ], + "ttft_s": [ + 0.5, + 0.21, + 1.18, + 0.82 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2e4cb369c515", + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8" + } + ], + "client_intervals": [ + { + "start": 134315.066686654, + "first": 134315.571089846, + "last": 134334.215808688, + "end": 134334.215914882, + "elapsed_s": 19.149228227994172, + "chunk_offsets_s": [ + 0.5044031919969711, + 0.6121159800095484, + 0.9656672599958256, + 1.3416760140098631, + 1.508123829989927, + 1.6801859260012861, + 1.8505256380012725, + 2.020312490989454, + 2.1862533850071486, + 2.3567349550139625, + 2.529276018991368, + 2.704097173991613, + 2.8824373699899297, + 3.0661841460096184, + 3.240450345008867, + 3.4217295400158037, + 3.6048558079928625, + 3.776233073993353, + 3.952035110007273, + 4.119701474002795, + 4.299597858014749, + 4.47926041300525, + 4.660729689989239, + 4.848257591016591, + 5.027207973005716, + 5.206808477989398, + 5.389186362997862, + 5.566721342009259, + 5.739237428002525, + 5.919775757996831, + 6.104496693995316, + 6.293880804005312, + 6.478158340003574, + 6.661528241995256, + 6.846279305987991, + 7.023206960991956, + 7.204818766011158, + 7.389970893011196, + 7.569858217990259, + 7.7481240250053816, + 7.929041208990384, + 8.107446729991352, + 8.288539469009265, + 8.471147836011369, + 8.658597205008846, + 8.842812048998894, + 9.025009352015331, + 9.203506872989237, + 9.388946107996162, + 9.570199873007368, + 9.752577131002909, + 9.933152723009698, + 10.125152063003043, + 10.311698181001702, + 10.499850585998502, + 10.6822418429947, + 10.861869609012501, + 11.038198608992388, + 11.212612661998719, + 11.393869624997023, + 11.577045215992257, + 11.762335917999735, + 11.944061226007761, + 12.122124785993947, + 12.298299846006557, + 12.478623656003037, + 12.663166264013853, + 12.848421331989812, + 13.034116313006962, + 13.220043489011005, + 13.398121879989048, + 13.580600993009284, + 13.765431141015142, + 13.95535352799925, + 14.14526669701445, + 14.328009365999606, + 14.509718223998789, + 14.685665967001114, + 14.858594501012703, + 15.041288047999842, + 15.222858087014174, + 15.404301066999324, + 15.583023996005068, + 15.759077500988496, + 15.94361258498975, + 16.126746140012983, + 16.30696255201474, + 16.48666602699086, + 16.666961273003835, + 16.84978915800457, + 17.032622914004605, + 17.212034143012716, + 17.39508607899188, + 17.570760647009593, + 17.74784434499452, + 17.925980925007025, + 18.09663381500286, + 18.2781071930076, + 18.45889559900388, + 18.638295051001478, + 18.807646011991892, + 18.979324184008874, + 19.14912203399581 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 180.852794990642, + "p95": 189.38411000999622, + "max": 376.00875401403755 + } + }, + { + "start": 134315.066928882, + "first": 134315.281094691, + "last": 134334.631749096, + "end": 134334.631900267, + "elapsed_s": 19.564971384999808, + "chunk_offsets_s": [ + 0.21416580898221582, + 0.29154032000224106, + 0.6118215429887641, + 0.9652231879881583, + 1.34102536199498, + 1.507915841997601, + 1.6798993769916706, + 1.8502232639875729, + 2.020110568002565, + 2.186053716985043, + 2.3565261990006547, + 2.529001454997342, + 2.703823985008057, + 2.8820439869887196, + 3.065985182998702, + 3.2401384839904495, + 3.421521167998435, + 3.604569082992384, + 3.7759155810053926, + 3.9515358849894255, + 4.119387373008067, + 4.299474559986265, + 4.479100347001804, + 4.660422549000941, + 4.8479859869985376, + 5.027021618006984, + 5.206501384993317, + 5.388909125991631, + 5.566509706986835, + 5.738962303003063, + 5.91957004298456, + 6.10428697799216, + 6.293599487980828, + 6.477844381995965, + 6.661161739990348, + 6.846002229984151, + 7.02299598098034, + 7.204543497005943, + 7.38969444000395, + 7.5696523590013385, + 7.7478450929920655, + 7.928726723999716, + 8.10717019700678, + 8.288029667979572, + 8.470937879988924, + 8.658389824995538, + 8.842537307995372, + 9.024213256983785, + 9.202802460989915, + 9.388610391004477, + 9.569564022007398, + 9.752229781006463, + 9.932812893006485, + 10.12441272198339, + 10.311342542991042, + 10.499175773991738, + 10.681824923987733, + 10.861483938002493, + 11.037813898001332, + 11.212221006979235, + 11.39324934198521, + 11.576504022988956, + 11.76170230700518, + 11.943466926983092, + 12.12152349500684, + 12.29769135499373, + 12.478001660987502, + 12.662609949999023, + 12.847794280998642, + 13.033558606985025, + 13.21939934999682, + 13.397485597000923, + 13.579854531999445, + 13.764758281002287, + 13.954702268005349, + 14.144773040985456, + 14.327250008995179, + 14.509192214987706, + 14.684910368989222, + 14.858056508004665, + 15.040548658987973, + 15.222224523982732, + 15.403647950995946, + 15.582476851006504, + 15.758474097994622, + 15.943102496006759, + 16.126154128985945, + 16.30656027299119, + 16.486051711981418, + 16.66634018998593, + 16.849346749979304, + 17.03193674198701, + 17.211531734006712, + 17.394417506991886, + 17.570269597985316, + 17.747211228997912, + 17.92541549098678, + 18.09602927599917, + 18.27750935900258, + 18.458254291996127, + 18.63770300798933, + 18.806998640997335, + 18.97867607700755, + 19.148363668005913, + 19.300495301984483, + 19.44627662698622, + 19.564820214000065 + ], + "chunk_gaps_ms": { + "count": 106, + "p50": 180.67633648752235, + "p95": 189.94398700306192, + "max": 375.8021740068216 + } + }, + { + "start": 134315.067155815, + "first": 134316.244495652, + "last": 134334.631861579, + "end": 134334.632168224, + "elapsed_s": 19.56501240900252, + "chunk_offsets_s": [ + 1.1773398370132782, + 1.3406659309985116, + 1.5075058980146423, + 1.679541818011785, + 1.8498211440164596, + 2.019724992016563, + 2.1856671330169775, + 2.356133455003146, + 2.5286429350089747, + 2.7033253030094784, + 2.881696988013573, + 3.065630023018457, + 3.2398132130037993, + 3.4211700730083976, + 3.604219908011146, + 3.77557831001468, + 3.9514620910049416, + 4.118987749010557, + 4.299067287996877, + 4.478619408997474, + 4.659996844013222, + 4.847616859013215, + 5.027037552994443, + 5.206566089007538, + 5.388520286011044, + 5.566119155002525, + 5.738605607999489, + 5.919490984000731, + 6.103870330000063, + 6.293177718995139, + 6.4774572380119935, + 6.6607576360111125, + 6.845481691008899, + 7.022519107995322, + 7.204182721994584, + 7.389243918994907, + 7.5692744470143225, + 7.747804690996418, + 7.9283454840187915, + 8.106793037004536, + 8.288010995020159, + 8.470584721013438, + 8.657998665003106, + 8.842213413998252, + 9.0243056420004, + 9.202618505019927, + 9.38814706899575, + 9.56937773001846, + 9.751742716005538, + 9.932324307010276, + 10.124222429003567, + 10.310934055014513, + 10.498987689003116, + 10.681345713994233, + 10.860984152008314, + 11.037318640010199, + 11.2117348210013, + 11.393062362010824, + 11.57621817602194, + 11.761437502020271, + 11.943276587000582, + 12.121261745021911, + 12.297592073009582, + 12.477740311005618, + 12.662423338013468, + 12.847611333010718, + 13.033287497004494, + 13.219131584017305, + 13.397148278017994, + 13.57966918402235, + 13.764569733000826, + 13.95457788099884, + 14.144505675008986, + 14.326982979022432, + 14.508835471002385, + 14.684725900995545, + 14.857792006019736, + 15.040273277001688, + 15.222032902995124, + 15.403385145997163, + 15.582212829001946, + 15.758292734011775, + 15.942831306019798, + 16.12605188699672, + 16.306147065013647, + 16.48587023600703, + 16.666150442004437, + 16.84886573199765, + 17.03166608000174, + 17.211266608006554, + 17.39414883000427, + 17.570078265998745, + 17.746945799997775, + 17.925148653012002, + 18.09575607001898, + 18.277327835006872, + 18.457978941994952, + 18.637440218997654, + 18.806729275005637, + 18.978405399015173, + 19.148177472001407, + 19.300161151011707, + 19.44601441299892, + 19.5647057640017 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 180.5407930223737, + "p95": 187.62001499999315, + "max": 191.89812199329026 + } + }, + { + "start": 134315.06659518, + "first": 134315.890845586, + "last": 134335.36722019, + "end": 134335.36757482, + "elapsed_s": 20.300979640014702, + "chunk_offsets_s": [ + 0.8242504060035571, + 0.9656041389971506, + 1.3416075490240473, + 1.508170967019396, + 1.6803097370138858, + 1.8506503440148663, + 2.0203626200091094, + 2.186311386001762, + 2.3567835160065442, + 2.529292996012373, + 2.704051909997361, + 2.882496443024138, + 3.0663567730225623, + 3.2405067950021476, + 3.421893591003027, + 3.604980850999709, + 3.776284531020792, + 3.952091462997487, + 4.119828805007273, + 4.299772805010434, + 4.479484962008428, + 4.660893485008273, + 4.848284904001048, + 5.027227541024331, + 5.206784845999209, + 5.389317310007755, + 5.566776031017071, + 5.739259524998488, + 5.9201114760071505, + 6.10465589701198, + 6.294002214999637, + 6.478286838013446, + 6.661576740007149, + 6.84640346901142, + 7.023366948007606, + 7.204838767007459, + 7.389981373009505, + 7.569902667019051, + 7.748468448000494, + 7.929096666019177, + 8.107469947019126, + 8.288401882018661, + 8.471311183006037, + 8.658655558014289, + 8.842936916014878, + 9.025278957007686, + 9.20373271801509, + 9.389204369013896, + 9.570494183019036, + 9.752833967999322, + 9.933281446021283, + 10.12520310402033, + 10.31182735200855, + 10.499977852014126, + 10.682298052008264, + 10.861924778000684, + 11.038254354003584, + 11.212669831002131, + 11.393924282019725, + 11.577100561000407, + 11.762386528018396, + 11.944115003017941, + 12.122178482997697, + 12.29842980101239, + 12.47874921001494, + 12.663296362006804, + 12.848474949016236, + 13.034171242004959, + 13.220172516012099, + 13.39830010800506, + 13.580655827012379, + 13.765485751006054, + 13.955480666016228, + 14.145396268024342, + 14.328005767019931, + 14.509771761018783, + 14.685793298005592, + 14.858724696008721, + 15.04129188900697, + 15.222914312005742, + 15.404355740000028, + 15.583079022006132, + 15.759204367001075, + 15.943742700008443, + 16.126800124999136, + 16.307018393999897, + 16.48672334101866, + 16.667015899001854, + 16.849917144019855, + 17.032551249023527, + 17.21216416201787, + 17.394937421020586, + 17.57089004199952, + 17.74789911400876, + 17.92610865601455, + 18.096688215999166, + 18.278161866997834, + 18.458952304004924, + 18.638447662000544, + 18.807772799016675, + 18.979372714005876, + 19.149156899016816, + 19.30089250902529, + 19.44695948701701, + 19.56571917500696, + 19.6470467930194, + 19.714451274019666, + 19.786841591005214, + 19.85049438601709, + 19.916141424997477, + 19.981797697022557, + 20.0460384710168, + 20.11091522499919, + 20.175723513006233, + 20.240404007025063, + 20.30062501001521 + ], + "chunk_gaps_ms": { + "count": 115, + "p50": 179.92129400954582, + "p95": 188.15050000557676, + "max": 376.00341002689674 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 56.1, + "per_stream_tps": 56.1, + "ttft_s_max": 0.14, + "timing_valid": true, + "end_to_end_tps": 54.53536226153051, + "makespan_s": 4.694201878999593, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 56.05399756939054, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 4.694201878999593, + "end_to_end_tps": 54.53536226153051, + "timing_valid": true, + "per_stream_tps": [ + 56.1 + ], + "ttft_s": [ + 0.14 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 134354.676471035, + "first": 134354.821348976, + "last": 134359.370533888, + "end": 134359.370672914, + "elapsed_s": 4.694201878999593, + "chunk_offsets_s": [ + 0.14487794099841267, + 0.21789720401284285, + 0.29179260501405224, + 0.3699306799971964, + 0.45379037101520225, + 0.5403146659955382, + 0.6263772780075669, + 0.709834431996569, + 0.7923692449985538, + 0.8770135960076004, + 0.9583257280173711, + 1.0400570509955287, + 1.12514324701624, + 1.2071325439901557, + 1.289860282005975, + 1.3746226060029585, + 1.4568899729929399, + 1.5300649470009375, + 1.5939206379989628, + 1.6571787880093325, + 1.7244354269932956, + 1.801247630995931, + 1.877407977997791, + 1.949613078002585, + 2.0242565170046873, + 2.10798537399387, + 2.191070080996724, + 2.272257856995566, + 2.3512731730006635, + 2.4549481210124213, + 2.5576575300074182, + 2.6641461880062707, + 2.7749565690173768, + 2.879723746009404, + 2.990329505992122, + 3.095697440992808, + 3.2068247979914304, + 3.3175006700039376, + 3.428301473002648, + 3.5368752290087286, + 3.64597487001447, + 3.753229105990613, + 3.862956622004276, + 3.9724551039980724, + 4.082382744003553, + 4.187115892011207, + 4.29687397001544, + 4.404325728013646, + 4.510705782013247, + 4.620383452012902, + 4.6940628529991955 + ], + "chunk_gaps_ms": { + "count": 50, + "p50": 84.70333750301506, + "p95": 110.80080299871042, + "max": 111.12735699862242 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 63.9, + "per_stream_tps": 32.5, + "ttft_s_max": 0.37, + "timing_valid": true, + "end_to_end_tps": 62.95505745299879, + "makespan_s": 8.132785843015881, + "steady_tps": 64.8, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 63.86077252863425, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 8.132785843015881, + "end_to_end_tps": 62.95505745299879, + "timing_valid": true, + "steady_tps": 64.80767102048287, + "per_stream_tps": [ + 32.8, + 32.2 + ], + "ttft_s": [ + 0.37, + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 134359.372494098, + "first": 134359.737802583, + "last": 134367.504898937, + "end": 134367.504984689, + "elapsed_s": 8.132490591000533, + "chunk_offsets_s": [ + 0.365308484993875, + 0.47046675998717546, + 0.5791501959902234, + 0.6862945650063921, + 0.7949280859902501, + 0.9030089559964836, + 1.009989707992645, + 1.1155339760007337, + 1.218625584995607, + 1.3158819020027295, + 1.4168651709915139, + 1.5222272819955833, + 1.6387660120090004, + 1.7512240330106579, + 1.8639807929866947, + 1.9784895429911558, + 2.0943036719982047, + 2.210309974005213, + 2.323208399990108, + 2.43888598200283, + 2.5559773230052087, + 2.6692350159864873, + 2.7786234979867004, + 2.8904620649991557, + 3.003040687995963, + 3.114221428986639, + 3.220444904989563, + 3.3288004630012438, + 3.436604329996044, + 3.545298773009563, + 3.652518813003553, + 3.7632820009894203, + 3.8729114409943577, + 3.9824459629890043, + 4.096320531010861, + 4.210951985995052, + 4.318672091001645, + 4.42895234900061, + 4.543737409985624, + 4.659436496993294, + 4.769143376004649, + 4.8811719029909, + 4.996350720990449, + 5.109682882990455, + 5.224908511998365, + 5.341055347991642, + 5.456119571987074, + 5.570892290998017, + 5.684687055007089, + 5.79990587499924, + 5.91355022098287, + 6.0274986989970785, + 6.139406684000278, + 6.253853256988805, + 6.367137430002913, + 6.479964775993722, + 6.5945898669888265, + 6.710288326983573, + 6.82419852601015, + 6.937304849998327, + 7.0473843110085, + 7.16237722299411, + 7.275636587990448, + 7.383691619004821, + 7.4953803140087985, + 7.61098387799575, + 7.722477002011146, + 7.835140749986749, + 7.94956524198642, + 8.058771943993634, + 8.132404839008814 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 112.62118548620492, + "p95": 116.00630200700834, + "max": 117.09134100237861 + } + }, + { + "start": 134359.372198846, + "first": 134359.518775691, + "last": 134367.43080543, + "end": 134367.430985444, + "elapsed_s": 8.0587865980051, + "chunk_offsets_s": [ + 0.1465768449997995, + 0.22134570800699294, + 0.470926636015065, + 0.579276728007244, + 0.6864296400162857, + 0.7950108619988896, + 0.9032708190206904, + 1.0102348370128311, + 1.115792576019885, + 1.2188632750185207, + 1.3162130870041437, + 1.4172470230259933, + 1.522485624998808, + 1.639096093014814, + 1.751482104009483, + 1.8643114490259904, + 1.978749166999478, + 2.0945667670166586, + 2.2105243380065076, + 2.3234678150038235, + 2.439212575001875, + 2.556305195001187, + 2.6695624090207275, + 2.778954123001313, + 2.890844075998757, + 3.0033688330149744, + 3.114550534024602, + 3.220775289024459, + 3.3291286239982583, + 3.436867569020251, + 3.545559788006358, + 3.652847182005644, + 3.763544952002121, + 3.8732393940153997, + 3.98279228201136, + 4.096580794022884, + 4.211279315000866, + 4.318931842019083, + 4.429282716999296, + 4.544067875016481, + 4.6596955450077076, + 4.769473168998957, + 4.881500320014311, + 4.9966804330179, + 5.110011956014205, + 5.225237473001471, + 5.341381221020129, + 5.456448389013531, + 5.571222260012291, + 5.684951830015052, + 5.800234452006407, + 5.913878462015418, + 6.027829851023853, + 6.13973463702132, + 6.254113121016417, + 6.367467255011434, + 6.480425260000629, + 6.595046079019085, + 6.710629271023208, + 6.824527183023747, + 6.937565673026256, + 7.047715207998408, + 7.162706920003984, + 7.275968029018259, + 7.383954921999248, + 7.4956400180235505, + 7.611312631022884, + 7.722970027010888, + 7.835371048015077, + 7.949423735000892, + 8.058606584003428 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 112.67705101636238, + "p95": 116.14374801865779, + "max": 249.58092800807208 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 87.0, + "per_stream_tps": 22.6, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 86.21559456155363, + "makespan_s": 11.877201627008617, + "steady_tps": 90.6, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 86.9604949590376, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 11.877201627008617, + "end_to_end_tps": 86.21559456155363, + "timing_valid": true, + "steady_tps": 90.61418080404022, + "per_stream_tps": [ + 22.5, + 22.3, + 23.2, + 22.8 + ], + "ttft_s": [ + 0.37, + 0.15, + 0.89, + 0.62 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 134367.507384716, + "first": 134367.875759123, + "last": 134379.201606437, + "end": 134379.201691853, + "elapsed_s": 11.694307136989664, + "chunk_offsets_s": [ + 0.36837440700037405, + 0.4730141889885999, + 0.7514849729777779, + 1.0506084149819799, + 1.2051700679876376, + 1.3574422700039577, + 1.511841982981423, + 1.6638881289982237, + 1.8094297089846805, + 1.949528893979732, + 2.09202401098446, + 2.2386907959880773, + 2.3983000649895985, + 2.5657529709860682, + 2.7352193830010947, + 2.897688136989018, + 3.066351807996398, + 3.2303595049888827, + 3.396278910979163, + 3.565442886989331, + 3.723800621984992, + 3.8909115759888664, + 4.061187702987809, + 4.222228031983832, + 4.3803039850026835, + 4.540478682989487, + 4.697366874985164, + 4.852347213978646, + 5.005751350981882, + 5.155099694995442, + 5.309907781978836, + 5.46873774100095, + 5.623517216998152, + 5.7785256799834315, + 5.9361012449953705, + 6.093600481981412, + 6.247638123983052, + 6.40211953598191, + 6.564380914991489, + 6.725040987977991, + 6.8858968499989714, + 7.047039757977473, + 7.207968350005103, + 7.3653741809830535, + 7.526333713001804, + 7.692985077999765, + 7.857350880978629, + 8.02270772799966, + 8.184851180994883, + 8.348885008977959, + 8.510830077982973, + 8.669327294977847, + 8.829996004002169, + 8.992431987979217, + 9.151964403979946, + 9.312911888991948, + 9.476823749981122, + 9.64055442699464, + 9.798154957999941, + 9.960499924985925, + 10.121645403996808, + 10.281042918999447, + 10.441172950988403, + 10.60009946799255, + 10.761531206982909, + 10.916875322000124, + 11.076359882979887, + 11.239132152986713, + 11.39907035298529, + 11.55900630998076, + 11.694221721001668 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.03411599376705, + "p95": 169.46641201502644, + "max": 299.12344200420193 + } + }, + { + "start": 134367.507668833, + "first": 134367.654827338, + "last": 134379.065605355, + "end": 134379.065675797, + "elapsed_s": 11.558006964012748, + "chunk_offsets_s": [ + 0.14715850501670502, + 0.22251013302593492, + 0.4729059430246707, + 0.751390422024997, + 1.0507512330077589, + 1.2049700550269336, + 1.3575123270275071, + 1.5117241390107665, + 1.66368771600537, + 1.809243455005344, + 1.9493782200152054, + 2.091878473001998, + 2.238645808014553, + 2.39824832600425, + 2.565547375008464, + 2.7350961140182335, + 2.897613616019953, + 3.0661916000244673, + 3.230165108019719, + 3.3961362360278144, + 3.5652573850238696, + 3.7236237750039436, + 3.8909168880200014, + 4.061013048019959, + 4.222113051015185, + 4.380090741004096, + 4.5404743470135145, + 4.697175565001089, + 4.852428823010996, + 5.005632371001411, + 5.1550852010259405, + 5.309817470028065, + 5.468544607021613, + 5.6233462580130436, + 5.77838003000943, + 5.935948604019359, + 6.093536488013342, + 6.247132444026647, + 6.402226215024712, + 6.564289532019757, + 6.724858525005402, + 6.8854610359994695, + 7.0468948440102395, + 7.2077771840267815, + 7.365309484011959, + 7.526182494999375, + 7.692920924018836, + 7.857320469018305, + 8.02262397602317, + 8.18469193202327, + 8.348701298004016, + 8.510903128015343, + 8.668668829021044, + 8.829892127017956, + 8.992354140005773, + 9.151935991016217, + 9.312370180006837, + 9.476271674007876, + 9.64022029901389, + 9.798113762022695, + 9.960180707013933, + 10.121307532011997, + 10.281022185023176, + 10.441156618006062, + 10.600078879011562, + 10.761283567000646, + 10.916626466001617, + 11.075782273022924, + 11.238581868004985, + 11.398510358005296, + 11.557936522003729 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.25901949615218, + "p95": 170.0961599999573, + "max": 299.3608109827619 + } + }, + { + "start": 134367.508456806, + "first": 134368.401912118, + "last": 134379.384291395, + "end": 134379.384492032, + "elapsed_s": 11.876035225985106, + "chunk_offsets_s": [ + 0.8934553119761404, + 1.049784988979809, + 1.204359648982063, + 1.3566857659898233, + 1.5111239879915956, + 1.66272163199028, + 1.808496789977653, + 1.948542219994124, + 2.091134111979045, + 2.237750820990186, + 2.3973788250004873, + 2.564843906002352, + 2.7342424989910796, + 2.896759569994174, + 3.0654911539750174, + 3.229465029988205, + 3.395533749979222, + 3.565007152996259, + 3.723081074975198, + 3.8900914789992385, + 4.060564530984266, + 4.2214473389904015, + 4.379598739993526, + 4.539621691976208, + 4.696722775988746, + 4.851602245995309, + 5.005105524993269, + 5.1543360719806515, + 5.30922501499299, + 5.467828370979987, + 5.622636612999486, + 5.7777276119741146, + 5.935122489987407, + 6.0927890070015565, + 6.246050834975904, + 6.401573636976536, + 6.563462777994573, + 6.7242698289919645, + 6.88493849398219, + 7.046183759986889, + 7.207165529980557, + 7.364605854992988, + 7.525608437979827, + 7.692338579974603, + 7.8567013279825915, + 8.021915115998127, + 8.184235575987259, + 8.348107194993645, + 8.51031543998397, + 8.667781665979419, + 8.829189026000677, + 8.991761763987597, + 9.151423159986734, + 9.312221667991253, + 9.475896973977797, + 9.640005087974714, + 9.797470607998548, + 9.959755899995798, + 10.120964429981541, + 10.280375495000044, + 10.440572624997003, + 10.599525764002465, + 10.761143612995511, + 10.91647707298398, + 11.075694954983192, + 11.2382964239805, + 11.39834007198806, + 11.557734190981137, + 11.693801633984549, + 11.802057921973756, + 11.87583458898007 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 159.6447000047192, + "p95": 168.73158398084342, + "max": 170.47305198502727 + } + }, + { + "start": 134367.507290405, + "first": 134368.126524849, + "last": 134379.310443455, + "end": 134379.310531335, + "elapsed_s": 11.803240930021275, + "chunk_offsets_s": [ + 0.6192344440205488, + 0.7517161029973067, + 1.0512336190149654, + 1.2056263760023285, + 1.3577684470219538, + 1.5120406530040782, + 1.6642830040073022, + 1.8097006270254496, + 1.9497925809992012, + 2.0922206960094627, + 2.239081382023869, + 2.398597557010362, + 2.565960312000243, + 2.735445313010132, + 2.897962687013205, + 3.0666169350151904, + 3.230583356023999, + 3.396475404006196, + 3.5657091020257212, + 3.724075732025085, + 3.8911088700115215, + 4.061565987009089, + 4.222571616002824, + 4.3806743980094325, + 4.540982250997331, + 4.697720121010207, + 4.852837040001759, + 5.00606239400804, + 5.155428688012762, + 5.310349180013873, + 5.4689570480259135, + 5.6237612740078475, + 5.778722510003718, + 5.936356197023997, + 6.0938700400001835, + 6.247309835016495, + 6.402568470017286, + 6.564323816011893, + 6.725395242014201, + 6.886269167007413, + 7.04730709301657, + 7.208446193020791, + 7.365730660007102, + 7.526595848001307, + 7.6933343410200905, + 7.857662100024754, + 8.023038353014272, + 8.185247767018154, + 8.349110731011024, + 8.511316321004415, + 8.669131585018476, + 8.83042550002574, + 8.992766884999583, + 9.152422233019024, + 9.313203894009348, + 9.477017555997008, + 9.641078938002465, + 9.798592981009278, + 9.961096429004101, + 10.122086979012238, + 10.281498604017543, + 10.441683256009128, + 10.60065575200133, + 10.76215138900443, + 10.917440005025128, + 11.076633137010504, + 11.239524404023541, + 11.399346279999008, + 11.55872460801038, + 11.694754502997966, + 11.80315305001568 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 159.73861199745443, + "p95": 169.23369801952504, + "max": 299.51751601765864 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 30.6, + "per_stream_tps": 30.6, + "ttft_s_max": 0.22, + "timing_valid": true, + "end_to_end_tps": 29.93718500316345, + "makespan_s": 8.551238200016087, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 30.59154659510732, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 8.551238200016087, + "end_to_end_tps": 29.93718500316345, + "timing_valid": true, + "per_stream_tps": [ + 30.6 + ], + "ttft_s": [ + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 134402.777054024, + "first": 134402.992547739, + "last": 134411.328183834, + "end": 134411.328292224, + "elapsed_s": 8.551238200016087, + "chunk_offsets_s": [ + 0.2154937150189653, + 0.2918327049992513, + 0.36879293600213714, + 0.44806891301414, + 0.5311900700035039, + 0.6157542660075705, + 0.6983667480235454, + 0.7791244020045269, + 0.8591924050124362, + 0.9412741970154457, + 1.0159411120112054, + 1.0870175860181917, + 1.1509373850130942, + 1.2158276900008786, + 1.2805298190214671, + 1.3443322040257044, + 1.4108408780011814, + 1.474976482015336, + 1.539354448002996, + 1.6043645340250805, + 1.6781319090223406, + 1.7545314030139707, + 1.8288239160028752, + 1.9005825410131365, + 1.9636983630189206, + 2.0297310470195953, + 2.105446583009325, + 2.1792770380270667, + 2.2541926450212486, + 2.3285420140018687, + 2.411033912008861, + 2.489162211015355, + 2.5530390970234293, + 2.62444824600243, + 2.700095724023413, + 2.7662697350024246, + 2.8315255870111287, + 2.900103690015385, + 2.975912962021539, + 3.0496383900172077, + 3.1238288739987183, + 3.203218660026323, + 3.284513702004915, + 3.3503766170179006, + 3.4214714060071856, + 3.4861843260005116, + 3.5503014420100953, + 3.6171773390087765, + 3.6938559880072717, + 3.770916795008816, + 3.84526427401579, + 3.9084424810134806, + 3.9708283740037587, + 4.033084855007473, + 4.096730215998832, + 4.161655673000496, + 4.227559923019726, + 4.293451167002786, + 4.358602348016575, + 4.424147782003274, + 4.488249262998579, + 4.551286277011968, + 4.6146469920058735, + 4.678783574025147, + 4.742911628010916, + 4.808437512023374, + 4.871802498004399, + 4.9361098970111925, + 5.002945714019006, + 5.077243906009244, + 5.154874019004637, + 5.23360707500251, + 5.313263186020777, + 5.400503159005893, + 5.481195275002392, + 5.563621863024309, + 5.6453484800003935, + 5.735927568020998, + 5.820168077014387, + 5.88915550900856, + 5.953019614011282, + 6.015641444013454, + 6.078151709021768, + 6.142799357010517, + 6.207615231018281, + 6.272504762018798, + 6.338659622007981, + 6.4087580400228035, + 6.479314499010798, + 6.54327865500818, + 6.607626697019441, + 6.673899339017225, + 6.746813511010259, + 6.818938855023589, + 6.893297381000593, + 6.96497535702656, + 7.027853480016347, + 7.092352624022169, + 7.156744466017699, + 7.221195215010084, + 7.287496266013477, + 7.362063394015422, + 7.437293649010826, + 7.509401743998751, + 7.5825152230099775, + 7.658592571009649, + 7.7420341719989665, + 7.826481572003104, + 7.914886563026812, + 7.996329730027355, + 8.07754873301019, + 8.160787006025203, + 8.244090089021483, + 8.331059476011433, + 8.415016519022174, + 8.49136355301016, + 8.5511298100173 + ], + "chunk_gaps_ms": { + "count": 116, + "p50": 71.54356250248384, + "p95": 84.44740000413731, + "max": 90.5790880206041 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 43.8, + "per_stream_tps": 22.4, + "ttft_s_max": 0.51, + "timing_valid": true, + "end_to_end_tps": 43.17382772853212, + "makespan_s": 11.859036526002456, + "steady_tps": 44.8, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 43.80760975128737, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 11.859036526002456, + "end_to_end_tps": 43.17382772853212, + "timing_valid": true, + "steady_tps": 44.76489773749304, + "per_stream_tps": [ + 22.5, + 22.2 + ], + "ttft_s": [ + 0.51, + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 134411.330700494, + "first": 134411.836811186, + "last": 134423.189582505, + "end": 134423.189646356, + "elapsed_s": 11.858945862011751, + "chunk_offsets_s": [ + 0.5061106919893064, + 0.614781612996012, + 0.7278888990113046, + 0.8375787069962826, + 0.949186173005728, + 1.066544049012009, + 1.1842586470011156, + 1.2965471729985438, + 1.4112388330104295, + 1.5266528879874386, + 1.6396275979932398, + 1.7559122609964106, + 1.868875099986326, + 1.9873958399984986, + 2.108554575010203, + 2.2299713199899998, + 2.3487298150139395, + 2.467546817002585, + 2.583824586996343, + 2.7038059739861637, + 2.822157916991273, + 2.9429558250121772, + 3.060229430004256, + 3.184361849009292, + 3.3007704410119914, + 3.4158656430081464, + 3.5316179980000015, + 3.6472641970030963, + 3.759647238010075, + 3.883072058990365, + 4.005715892999433, + 4.13003548499546, + 4.251286862010602, + 4.370952152996324, + 4.48579715899541, + 4.605102965986589, + 4.725274522003019, + 4.840791152004385, + 4.958471130987164, + 5.074125382001512, + 5.194859560986515, + 5.314401236013509, + 5.4299412080144975, + 5.550243629986653, + 5.664683012990281, + 5.784048900008202, + 5.906699320010375, + 6.023551343008876, + 6.144754175009439, + 6.263854869001079, + 6.381993421004154, + 6.495334423001623, + 6.61266052199062, + 6.7299397609895095, + 6.838979004998691, + 6.94819056999404, + 7.064232283009915, + 7.185784450994106, + 7.308261722995667, + 7.434473345987499, + 7.555349025002215, + 7.675845015997766, + 7.784744770993711, + 7.906698397011496, + 8.026539501006482, + 8.14683685300406, + 8.267650546011282, + 8.379611221986124, + 8.497315184009494, + 8.613230580987874, + 8.732278078008676, + 8.849983257008716, + 8.966787793993717, + 9.077401677001035, + 9.18719540600432, + 9.303672178997658, + 9.419956088007893, + 9.533074006991228, + 9.65002000698587, + 9.761569877999136, + 9.878387506003492, + 9.994783025991637, + 10.113298076001229, + 10.231514976010658, + 10.348618324002018, + 10.461979863001034, + 10.578136077005183, + 10.688868820987409, + 10.809603878005873, + 10.927395490987692, + 11.045709460013313, + 11.15874208699097, + 11.277069030009443, + 11.392025973007549, + 11.506386460008798, + 11.620527043996844, + 11.721696285007056, + 11.797910658991896, + 11.858882011001697 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 117.02467399300076, + "p95": 122.65042000217363, + "max": 126.21162299183197 + } + }, + { + "start": 134411.33060983, + "first": 134411.547769521, + "last": 134423.051863545, + "end": 134423.051998269, + "elapsed_s": 11.721388438978465, + "chunk_offsets_s": [ + 0.21715969100478105, + 0.2936195639776997, + 0.6144140439864714, + 0.7275265789940022, + 0.8373236579936929, + 0.9488292599853594, + 1.0661782249808311, + 1.1840303469798528, + 1.2963348389894236, + 1.4108878079860006, + 1.5264068929827772, + 1.6392665409948677, + 1.7556526349799242, + 1.8684419209894259, + 1.9871325829881243, + 2.108265863993438, + 2.2296739049779717, + 2.3483088919892907, + 2.467336579982657, + 2.584083956986433, + 2.7041289859917015, + 2.821881829004269, + 2.942602831986733, + 3.059834471991053, + 3.1839650349866133, + 3.3005129270022735, + 3.4155647729930934, + 3.5314221429871395, + 3.6469532159972005, + 3.759373326000059, + 3.8828351839911193, + 4.005289656983223, + 4.12961934399209, + 4.250894736003829, + 4.370720684994012, + 4.4855033599887975, + 4.60483492500498, + 4.72499048197642, + 4.840563234989531, + 4.958064973994624, + 5.073889961990062, + 5.194443452986889, + 5.314123739983188, + 5.429744313005358, + 5.55002412898466, + 5.6642733679909725, + 5.783670356002403, + 5.9064917209907435, + 6.02324444099213, + 6.144400973978918, + 6.263432392996037, + 6.381542115996126, + 6.494984260993078, + 6.61238363297889, + 6.729683974990621, + 6.83867167099379, + 6.947859285981394, + 7.063819534989307, + 7.185515402001329, + 7.3080401420011185, + 7.4342552199959755, + 7.555069080990506, + 7.675614476000192, + 7.784466858982341, + 7.906308623001678, + 8.026300592988264, + 8.146774042979814, + 8.267369592998875, + 8.379222135001328, + 8.497075364983175, + 8.612922094995156, + 8.73189970999374, + 8.84974099000101, + 8.966720806987723, + 9.07719001499936, + 9.186923508998007, + 9.303390218992718, + 9.419714684976498, + 9.532786222989671, + 9.649706194002647, + 9.761327993997838, + 9.878036351001356, + 9.994578626996372, + 10.113085581979249, + 10.231297250982607, + 10.348373449000064, + 10.461766456981422, + 10.577919823001139, + 10.688573309977073, + 10.80941288699978, + 10.927120331005426, + 11.045434938976541, + 11.158457854995504, + 11.276793581986567, + 11.391622934985207, + 11.50610192399472, + 11.620458089979365, + 11.72125371498987 + ], + "chunk_gaps_ms": { + "count": 97, + "p50": 117.23164000432007, + "p95": 123.46185799106024, + "max": 320.79448000877164 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 58.5, + "per_stream_tps": 15.1, + "ttft_s_max": 1.17, + "timing_valid": true, + "end_to_end_tps": 58.05884809337646, + "makespan_s": 17.637277239002287, + "steady_tps": 61.2, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 58.54126406430415, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 17.637277239002287, + "end_to_end_tps": 58.05884809337646, + "timing_valid": true, + "steady_tps": 61.21580912805793, + "per_stream_tps": [ + 15.0, + 15.2, + 15.5, + 15.0 + ], + "ttft_s": [ + 0.5, + 0.82, + 1.17, + 0.21 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 134423.191946512, + "first": 134423.694531378, + "last": 134440.677328179, + "end": 134440.677428699, + "elapsed_s": 17.485482187010348, + "chunk_offsets_s": [ + 0.5025848659861367, + 0.6098449009878095, + 0.9588636709959246, + 1.330521174997557, + 1.4949946219858248, + 1.6573573749919888, + 1.8162817679985892, + 1.9835979939962272, + 2.155239885993069, + 2.3163552609912585, + 2.47735175100388, + 2.6425680319953244, + 2.8040180790121667, + 2.972509451006772, + 3.136356959003024, + 3.307743268989725, + 3.4843189159873873, + 3.6646904750086833, + 3.8385789069870953, + 4.012321496004006, + 4.184690280992072, + 4.359628464007983, + 4.528319544013357, + 4.705963802000042, + 4.883389327005716, + 5.058223517000442, + 5.222088087990414, + 5.388213413010817, + 5.554644360003294, + 5.728236675000517, + 5.898834850988351, + 6.077737209998304, + 6.259046469000168, + 6.441204009985086, + 6.613160503999097, + 6.78794293498504, + 6.9601370230084285, + 7.131374822987709, + 7.303827369003557, + 7.471268672001315, + 7.643470620008884, + 7.817731508985162, + 7.990071373991668, + 8.164141654007835, + 8.336062833986944, + 8.507913540001027, + 8.680322099011391, + 8.8536556379986, + 9.026796921010828, + 9.19553310101037, + 9.372008025995456, + 9.541172650002409, + 9.709965224988991, + 9.877588534000097, + 10.041717238986166, + 10.206514096003957, + 10.363595484988764, + 10.522581355005968, + 10.696003730001394, + 10.87505982798757, + 11.055726127990056, + 11.240812842996093, + 11.417292163998354, + 11.596347476996016, + 11.77066458799527, + 11.94801802499569, + 12.131822558993008, + 12.31123008200666, + 12.487407745007658, + 12.659487883996917, + 12.832012967992341, + 13.003942517010728, + 13.178061693004565, + 13.349759405013174, + 13.51589349898859, + 13.679223068000283, + 13.845999312994536, + 14.017806869989727, + 14.18755401999806, + 14.349961491010617, + 14.507599356002174, + 14.671842620009556, + 14.841618373000529, + 15.016615166998236, + 15.187352745007956, + 15.35917397699086, + 15.526727001008112, + 15.684058306011138, + 15.857513016002486, + 16.02694576999056, + 16.20361923900782, + 16.383936567988712, + 16.559377066005254, + 16.727435615990544, + 16.89890066400403, + 17.063820793002378, + 17.221538395009702, + 17.36339690198656, + 17.48538166700746 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 171.83596899849363, + "p95": 182.15754098491743, + "max": 371.65750400163233 + } + }, + { + "start": 134423.192432871, + "first": 134424.014821452, + "last": 134440.767827038, + "end": 134440.767891865, + "elapsed_s": 17.57545899398974, + "chunk_offsets_s": [ + 0.8223885810002685, + 0.9582039029919542, + 1.3301960820099339, + 1.4943440849892795, + 1.656665880989749, + 1.8158312779851258, + 1.9829553290037438, + 2.154783892008709, + 2.315907410986256, + 2.476907564996509, + 2.642159826005809, + 2.8034979150106665, + 2.9719809670059476, + 3.1359451540047303, + 3.3071819249889813, + 3.4837972639943473, + 3.6642750540049747, + 3.838165149994893, + 4.0117665189900436, + 4.184236831002636, + 4.359176389989443, + 4.5278683500073384, + 4.705645860987715, + 4.882935044995975, + 5.057771394989686, + 5.2215645969845355, + 5.3877974479983095, + 5.554221723985393, + 5.7277816889982205, + 5.898267746990314, + 6.077323420991888, + 6.258811320993118, + 6.4403973419975955, + 6.612352316005854, + 6.787321114999941, + 6.959515219001332, + 7.1306625150027685, + 7.302955315011786, + 7.470368028007215, + 7.6427867420134135, + 7.81697525200434, + 7.989130925008794, + 8.16329017400858, + 8.335311025002738, + 8.507024622987956, + 8.679484744992806, + 8.852860968996538, + 9.025901109009283, + 9.194721392996144, + 9.371271287993295, + 9.540317473991308, + 9.709102257998893, + 9.876751948992023, + 10.040800468006637, + 10.20563781799865, + 10.362768658989808, + 10.521817275992362, + 10.69523406701046, + 10.87431139498949, + 11.05514022501302, + 11.240117654000642, + 11.416537971002981, + 11.595396805001656, + 11.769663071987452, + 11.947140882984968, + 12.131048192008166, + 12.31040322300396, + 12.486534505995223, + 12.658878927002661, + 12.831424248986878, + 13.003016498987563, + 13.177729721006472, + 13.34891715599224, + 13.515085774997715, + 13.67830967300688, + 13.845234720996814, + 14.016989833995467, + 14.186815297987778, + 14.34918923600344, + 14.506749491003575, + 14.670962164993398, + 14.84091895999154, + 15.015679036994698, + 15.186662769992836, + 15.358268756011967, + 15.52601640499779, + 15.683208217000356, + 15.85665311999037, + 16.02618455400807, + 16.20277460498619, + 16.382923980010673, + 16.558400763984537, + 16.72673669899814, + 16.897978435998084, + 17.063030938996235, + 17.220677219011122, + 17.362842867994914, + 17.484789620997617, + 17.575394167011837 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 171.65979200217407, + "p95": 181.48790000122972, + "max": 371.99217901797965 + } + }, + { + "start": 134423.192810486, + "first": 134424.361994012, + "last": 134440.829033335, + "end": 134440.829135407, + "elapsed_s": 17.636324920982588, + "chunk_offsets_s": [ + 1.1691835259844083, + 1.3299868369940668, + 1.4942985369998496, + 1.656322694994742, + 1.8154929079755675, + 1.982421502994839, + 2.1542086619883776, + 2.3155633449787274, + 2.4764197829936165, + 2.64174907800043, + 2.8032002929830924, + 2.971567034983309, + 3.135536884976318, + 3.3068397309980355, + 3.4834957549755927, + 3.663863488996867, + 3.837856489000842, + 4.011423541000113, + 4.1837878939986695, + 4.358728620980401, + 4.527648544986732, + 4.705300931993406, + 4.882590946974233, + 5.057323353976244, + 5.221256944001652, + 5.387386652000714, + 5.553814270999283, + 5.727337983989855, + 5.8979245449882, + 6.076908432994969, + 6.2582545850018505, + 6.4403758729749825, + 6.612335614976473, + 6.787044243974378, + 6.959308645979036, + 7.130545597989112, + 7.302928229997633, + 7.470366780995391, + 7.6426432189764455, + 7.816902171995025, + 7.989161563978996, + 8.16333740297705, + 8.335155951994238, + 8.507014224975137, + 8.679423790978035, + 8.852756323001813, + 9.025898197985953, + 9.19463172199903, + 9.371107158978703, + 9.540272822981933, + 9.709140591992764, + 9.876686339994194, + 10.04081918799784, + 10.205614684993634, + 10.362695529998746, + 10.521751762978965, + 10.695179352973355, + 10.874232506990666, + 11.054898103000596, + 11.239912744000321, + 11.416465257992968, + 11.59543669098639, + 11.769834466977045, + 11.947189471975435, + 12.131053584977053, + 12.31032911897637, + 12.48650314999395, + 12.65865960298106, + 12.831186430994421, + 13.00304243399296, + 13.177048274985282, + 13.348859402001835, + 13.515064321982209, + 13.678392771980725, + 13.845097677985905, + 14.01690739498008, + 14.18672484398121, + 14.349064015987096, + 14.506770754989702, + 14.671018002991332, + 14.84071836998919, + 15.015715003974037, + 15.18652587098768, + 15.35827584497747, + 15.525896544975694, + 15.683152046985924, + 15.856612452975241, + 16.026044439000543, + 16.202790301991627, + 16.383024213981116, + 16.55848026298918, + 16.726631477999035, + 16.898001043999102, + 17.06291867798427, + 17.22063175297808, + 17.36257010800182, + 17.484557593998034, + 17.575299648975488, + 17.636222848988837 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 171.79843799385708, + "p95": 180.6655960099306, + "max": 185.0146409997251 + } + }, + { + "start": 134423.191858168, + "first": 134423.405425965, + "last": 134440.413047871, + "end": 134440.413207234, + "elapsed_s": 17.22134906600695, + "chunk_offsets_s": [ + 0.21356779700727202, + 0.2908188190194778, + 0.609618071001023, + 0.9584386350179557, + 1.3302708760020323, + 1.4945649940054864, + 1.6568801029934548, + 1.8160102709953208, + 1.982989134005038, + 2.1548320800065994, + 2.316100099007599, + 2.47699530099635, + 2.6422675930080004, + 2.803660764999222, + 2.9722092839947436, + 3.136073079018388, + 3.3073803720180877, + 3.4840261090139393, + 3.6643904670199845, + 3.8381374400050845, + 4.011992034997093, + 4.184358132013585, + 4.359285837010248, + 4.528080267016776, + 4.705525903002126, + 4.882978465000633, + 5.057916007004678, + 5.221654524008045, + 5.387768314016284, + 5.554371038015233, + 5.727930940018268, + 5.8987147330190055, + 6.077414883999154, + 6.25928932800889, + 6.441173819999676, + 6.613187765004113, + 6.787643774994649, + 6.9597754850110505, + 7.130946218007011, + 7.303769885998918, + 7.471215683995979, + 7.643200674006948, + 7.817406367015792, + 7.989652671996737, + 8.163760949013522, + 8.335752444021637, + 8.507499302009819, + 8.68001837900374, + 8.853341231995728, + 9.02635582201765, + 9.195202696020715, + 9.371740575996228, + 9.54085085200495, + 9.709524269012036, + 9.877233827020973, + 10.041676249995362, + 10.206467923999298, + 10.36324796199915, + 10.522286323015578, + 10.695714458008297, + 10.874604904995067, + 11.055445670994231, + 11.240395581000485, + 11.41701273000217, + 11.596033295005327, + 11.770196833997034, + 11.947758174006594, + 12.131530470011057, + 12.31088722901768, + 12.48704177502077, + 12.659173460997408, + 12.831703664996894, + 13.003491498006042, + 13.178219583001919, + 13.349392619013088, + 13.515615298005287, + 13.678832684003282, + 13.84576200399897, + 14.017468497011578, + 14.187453068007017, + 14.349718262004899, + 14.507222971005831, + 14.67143034900073, + 14.84145381001872, + 15.016295689012622, + 15.187128057994414, + 15.358752347005066, + 15.526442911999766, + 15.683690095000202, + 15.857185233995551, + 16.026719548011897, + 16.203244117001304, + 16.383450127003016, + 16.558935934008332, + 16.72736950899707, + 16.898602767003467, + 17.063557757996023, + 17.22118970300653 + ], + "chunk_gaps_ms": { + "count": 97, + "p50": 171.84294600156136, + "p95": 183.772296004463, + "max": 371.83224098407663 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + } + ] +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/rows-without-bodies.jsonl b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/rows-without-bodies.jsonl new file mode 100644 index 0000000000..cf25982e23 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/rows-without-bodies.jsonl @@ -0,0 +1,44 @@ +{"i": 0, "conv": "S1", "turn": 1, "arm": "drafted", "group": "", "history": 7757, "allowed": [0], "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 6.017345185, "prefill": 6.015855978, "wall": 8.039078632020392, "finish": "tool_calls"}, "ok": true, "body_sha256": "c89863023849e1c523580f91d6e3e557e187ee1345ec141afe8843f12f6a48bb", "body_bytes": 29036} +{"i": 1, "conv": "S1", "turn": 1, "arm": "plain", "group": "", "history": 7757, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 6144, "prompt": 7759, "tokens": 61, "ttft": 1.243151661, "prefill": 1.241289156, "wall": 4.633735783980228, "finish": "tool_calls"}, "ok": true, "body_sha256": "990b29e03db5234f33dad6e1fa5929871234fa91c7f57ae49f27a1167d79423a", "body_bytes": 29050} +{"i": 2, "conv": "S1", "turn": 2, "arm": "drafted", "group": "", "history": 15548, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 6144, "prompt": 15550, "tokens": 44, "ttft": 7.191359083, "prefill": 7.188792217, "wall": 8.257592642010422, "finish": "tool_calls"}, "ok": true, "body_sha256": "de8e4593c2fe9ff0fff8a4aebd959d2edc7217b936fa8b932051eb906d40b3a6", "body_bytes": 52453} +{"i": 3, "conv": "S1", "turn": 2, "arm": "plain", "group": "", "history": 15548, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 14336, "prompt": 15550, "tokens": 44, "ttft": 1.022502598, "prefill": 1.019329999, "wall": 3.47304099201574, "finish": "tool_calls"}, "ok": true, "body_sha256": "52bdc70485afac166b4ce7266dc5f9a27a272063a88bcee6b79268304e439ae7", "body_bytes": 52467} +{"i": 4, "conv": "S1", "turn": 3, "arm": "drafted", "group": "", "history": 19458, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 14336, "prompt": 19460, "tokens": 80, "ttft": 4.043972641, "prefill": 4.040647124, "wall": 6.637460444995668, "finish": "tool_calls"}, "ok": true, "body_sha256": "34db447cbf94eed051f60f565fd9c93f5e116ca70cc3751d574aa6d7c013b11e", "body_bytes": 64230} +{"i": 5, "conv": "S1", "turn": 3, "arm": "plain", "group": "", "history": 19458, "allowed": {"min": 11266, "max": 19458}, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 18432, "prompt": 19460, "tokens": 80, "ttft": 0.914531106, "prefill": 0.910645021, "wall": 5.370368728006724, "finish": "tool_calls"}, "ok": true, "body_sha256": "c12f8625367f418f57b284b9ef6f739895366066921bcc0025b88e7bea3ab750", "body_bytes": 64244} +{"i": 6, "conv": "S1", "turn": 4, "arm": "drafted", "group": "", "history": 21872, "allowed": {"min": 11266, "max": 19458}, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 18432, "prompt": 21874, "tokens": 148, "ttft": 2.719838939, "prefill": 2.715979953, "wall": 7.874806687003002, "finish": "tool_calls"}, "ok": true, "body_sha256": "f1fbb932a16ce616b6275e271dfd133e2496ea1e6f4302870116d395ec8a28e1", "body_bytes": 76137} +{"i": 7, "conv": "S1", "turn": 4, "arm": "plain", "group": "", "history": 21872, "allowed": {"min": 13680, "max": 21872}, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 20480, "prompt": 21874, "tokens": 148, "ttft": 1.118899911, "prefill": 1.114527678, "wall": 9.398606637027115, "finish": "tool_calls"}, "ok": true, "body_sha256": "3057a9339796b5df85f5561810e4a27564624f9a52bbf8475c795a25047ed6f6", "body_bytes": 76151} +{"i": 8, "conv": "S1", "turn": 5, "arm": "drafted", "group": "", "history": 26784, "allowed": {"min": 13680, "max": 21872}, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 20480, "prompt": 26786, "tokens": 160, "ttft": 5.088207847, "prefill": 5.082944314, "wall": 10.379995980998501, "finish": "length"}, "ok": true, "body_sha256": "f377a8925455b63b8138fc440b1eab26f7af86cdb8dbe7cd031e1d3d0ca21ce9", "body_bytes": 99864} +{"i": 9, "conv": "S1", "turn": 5, "arm": "plain", "group": "", "history": 26784, "allowed": {"min": 18592, "max": 26784}, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 26624, "prompt": 26786, "tokens": 160, "ttft": 0.418724282, "prefill": 0.413140589, "wall": 9.314170341007411, "finish": "length"}, "ok": true, "body_sha256": "8037e85d4245fcc3eae0698239d87901fd9842e52d3af10a4ee2e627101e1add", "body_bytes": 99878} +{"i": 10, "conv": "S1", "turn": 6, "arm": "drafted", "group": "", "history": 29291, "allowed": {"min": 18592, "max": 26784}, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 26624, "prompt": 29293, "tokens": 160, "ttft": 2.296049983, "prefill": 2.289952462, "wall": 7.6957853229832835, "finish": "length"}, "ok": true, "body_sha256": "c0c3eccc6353d159b1fc94510be036e65203433e93db128827b3ea8d2e83a274", "body_bytes": 112228} +{"i": 11, "conv": "S1", "turn": 6, "arm": "plain", "group": "", "history": 29291, "allowed": {"min": 21099, "max": 29291}, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 28672, "prompt": 29293, "tokens": 160, "ttft": 0.701628817, "prefill": 0.695103222, "wall": 9.653295497992076, "finish": "length"}, "ok": true, "body_sha256": "5219ca6734f859b0c2a5d08eb51c683f2064fa838042525da818ceeeb992099e", "body_bytes": 112242} +{"i": 12, "conv": "S1", "turn": 7, "arm": "drafted", "group": "", "history": 32214, "allowed": {"min": 21099, "max": 29291}, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 28672, "prompt": 32216, "tokens": 160, "ttft": 2.809401116, "prefill": 2.802816749, "wall": 8.42428379500052, "finish": "length"}, "ok": true, "body_sha256": "a57732c1c6a3792d7d9bfd8948ebdbdcce2ac763f94910fbccc02a3720eb2fd8", "body_bytes": 124515} +{"i": 13, "conv": "S1", "turn": 7, "arm": "plain", "group": "", "history": 32214, "allowed": {"min": 24022, "max": 32214}, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 30720, "prompt": 32216, "tokens": 160, "ttft": 1.189705349, "prefill": 1.182460506, "wall": 10.140628474997357, "finish": "length"}, "ok": true, "body_sha256": "ee26e7744e00097a28b8cec35ce4f0e9388151715a914a9e3131b3582aea6fed", "body_bytes": 124529} +{"i": 14, "conv": "S1", "turn": 8, "arm": "drafted", "group": "", "history": 35140, "allowed": {"min": 24022, "max": 32214}, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 30720, "prompt": 35142, "tokens": 160, "ttft": 3.697935062, "prefill": 3.690782828, "wall": 9.269474010012345, "finish": "length"}, "ok": true, "body_sha256": "fe7fe2c354333f52dcf44745f0e1bfbc733092236ed163e55d9829a356a27bf0", "body_bytes": 136847} +{"i": 15, "conv": "S1", "turn": 8, "arm": "plain", "group": "", "history": 35140, "allowed": {"min": 26948, "max": 35140}, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 34816, "prompt": 35142, "tokens": 160, "ttft": 0.550386385, "prefill": 0.541889285, "wall": 9.506747141014785, "finish": "length"}, "ok": true, "body_sha256": "74a19048c8b816af7494ebd058ef75fa52d4fa78e998a659056d1696d3b2ccb1", "body_bytes": 136861} +{"i": 16, "conv": "S1", "turn": 9, "arm": "drafted", "group": "", "history": 37858, "allowed": {"min": 26948, "max": 35140}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 34816, "prompt": 37860, "tokens": 160, "ttft": 2.540413534, "prefill": 2.530999105, "wall": 5.261315346026095, "finish": "length"}, "ok": true, "body_sha256": "fbc963d846c6bf8bdb15043358e951488ced422c58133fc376f686a1bcc38400", "body_bytes": 149173} +{"i": 17, "conv": "S1", "turn": 9, "arm": "plain", "group": "", "history": 37858, "allowed": {"min": 29666, "max": 37858}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 36864, "prompt": 37860, "tokens": 160, "ttft": 0.92549176, "prefill": 0.916528319, "wall": 9.902982049010461, "finish": "length"}, "ok": true, "body_sha256": "b0ab79d2425abd2d494592a78146eed56d4a5d4ea9d52d52a7e78f4ac5a94c27", "body_bytes": 149187} +{"i": 18, "conv": "X", "turn": 1, "arm": "drafted", "group": "", "history": 2113, "allowed": [0], "got": {"sha": "9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.763965719, "prefill": 1.76336622, "wall": 6.9898692129936535, "finish": "length"}, "ok": true, "body_sha256": "ea7f1fb5d610d52f549d66c8d5bac3876b0c32a1f369f7230266491a9a674737", "body_bytes": 9524} +{"i": 19, "conv": "A", "turn": 1, "arm": "drafted", "group": "", "history": 7410, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 6144, "prompt": 7412, "tokens": 95, "ttft": 1.093411897, "prefill": 1.091889908, "wall": 3.782900193007663, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a153933bfbacdc64dd54d832df32d21139497927a2527fd60217851614cf603", "body_bytes": 28006} +{"i": 20, "conv": "A", "turn": 1, "arm": "plain", "group": "", "history": 7410, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 6144, "prompt": 7412, "tokens": 95, "ttft": 1.048851443, "prefill": 1.046974675, "wall": 6.316319612989901, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a6c1cda2af13b7752b30514bbb91dc76381167e03b207540e31a0f810a462b3", "body_bytes": 28020} +{"i": 21, "conv": "B", "turn": 1, "arm": "drafted", "group": "", "history": 7472, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 6144, "prompt": 7474, "tokens": 72, "ttft": 1.129876178, "prefill": 1.128241154, "wall": 3.033458680001786, "finish": "tool_calls"}, "ok": true, "body_sha256": "6685678d8326dff3c264f306810b45e76303da9dbc2ff27f9b4eec69f8a6384c", "body_bytes": 27976} +{"i": 22, "conv": "B", "turn": 1, "arm": "plain", "group": "", "history": 7472, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 6144, "prompt": 7474, "tokens": 72, "ttft": 1.08105309, "prefill": 1.079221438, "wall": 5.088942807022249, "finish": "tool_calls"}, "ok": true, "body_sha256": "ac1e48a68c55011216bc0008268ca26a6a788c311372fed6bc6dc9ed62769bf4", "body_bytes": 27990} +{"i": 23, "conv": "A", "turn": 2, "arm": "drafted", "group": "", "history": 12621, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 6144, "prompt": 12623, "tokens": 120, "ttft": 5.16198072, "prefill": 5.158820119, "wall": 9.144829953002045, "finish": "tool_calls"}, "ok": true, "body_sha256": "b82b386c13276d03245ee0d95d4d1e1b540d69abd34f4268722c534a58632440", "body_bytes": 51797} +{"i": 24, "conv": "A", "turn": 2, "arm": "plain", "group": "", "history": 12621, "allowed": {"min": 4429, "max": 12621}, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 12288, "prompt": 12623, "tokens": 120, "ttft": 0.547337164, "prefill": 0.544325815, "wall": 7.21886031399481, "finish": "tool_calls"}, "ok": true, "body_sha256": "0ea1dee4089539ecbad4b6d35c022dabda93cbc107f9306b0855fd8c1a5b6edb", "body_bytes": 51811} +{"i": 25, "conv": "B", "turn": 2, "arm": "drafted", "group": "", "history": 12718, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 6144, "prompt": 12720, "tokens": 79, "ttft": 5.221678122, "prefill": 5.218548281, "wall": 8.001385830022627, "finish": "tool_calls"}, "ok": true, "body_sha256": "f2c1cb809f6665946e8908be08d7a0fc5b9fe792499e21da4e77ccfb41ed1e77", "body_bytes": 51398} +{"i": 26, "conv": "B", "turn": 2, "arm": "plain", "group": "", "history": 12718, "allowed": {"min": 4526, "max": 12718}, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 12288, "prompt": 12720, "tokens": 79, "ttft": 0.593815411, "prefill": 0.590869621, "wall": 4.9774870880064555, "finish": "tool_calls"}, "ok": true, "body_sha256": "8187560324a86cab071c0ee332dd9078c19d9f0cb6687b845b5f3ab029b93f3f", "body_bytes": 51412} +{"i": 27, "conv": "A", "turn": 3, "arm": "drafted", "group": "", "history": 18061, "allowed": {"min": 4429, "max": 12621}, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 12288, "prompt": 18063, "tokens": 160, "ttft": 4.427400128, "prefill": 4.42370937, "wall": 10.00547803399968, "finish": "length"}, "ok": true, "body_sha256": "b837c8165ac38e9436d16b3d2d217da0f5d663ae80b25bd5408fa16aeaed6511", "body_bytes": 75423} +{"i": 28, "conv": "A", "turn": 3, "arm": "plain", "group": "", "history": 18061, "allowed": {"min": 9869, "max": 18061}, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 16384, "prompt": 18063, "tokens": 160, "ttft": 1.284531813, "prefill": 1.28009949, "wall": 10.241110984992702, "finish": "length"}, "ok": true, "body_sha256": "9453bd121b8af1ba0fa035a19d19f18ce2ac558f9b8003a3dc8659afa928451c", "body_bytes": 75437} +{"i": 29, "conv": "B", "turn": 3, "arm": "drafted", "group": "", "history": 15030, "allowed": {"min": 4526, "max": 12718}, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 12288, "prompt": 15032, "tokens": 85, "ttft": 2.314214389, "prefill": 2.310721821, "wall": 5.250862660992425, "finish": "tool_calls"}, "ok": true, "body_sha256": "c4f9a2cad2ff6792f8ff2d8203aaafd6c21e5481f120ad8e5d60bb42e9285857", "body_bytes": 63313} +{"i": 30, "conv": "B", "turn": 3, "arm": "plain", "group": "", "history": 15030, "allowed": {"min": 6838, "max": 15030}, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 14336, "prompt": 15032, "tokens": 85, "ttft": 0.735594262, "prefill": 0.732140378, "wall": 5.4633952979929745, "finish": "tool_calls"}, "ok": true, "body_sha256": "7f29c5ba782bd43a24297c2f8558f3b4f6a03f45704261feef121e8c9c4f3cf6", "body_bytes": 63327} +{"i": 31, "conv": "E", "turn": 1, "arm": "drafted", "group": "", "history": 40607, "allowed": {"min": 0, "max": 6611}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 6144, "prompt": 40609, "tokens": 160, "ttft": 26.114665029, "prefill": 26.104842732, "wall": 28.487778538983548, "finish": "length"}, "ok": true, "body_sha256": "7ca9838c87db0a4e713c829a162a26df23f8c2b556d9c0e3578c472fb12230a8", "body_bytes": 161550} +{"i": 32, "conv": "E", "turn": 1, "arm": "plain", "group": "", "history": 40607, "allowed": {"min": 32415, "max": 40607}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 38912, "prompt": 40609, "tokens": 160, "ttft": 1.322568033, "prefill": 1.313311648, "wall": 10.328478584997356, "finish": "length"}, "ok": true, "body_sha256": "df845f6beeb7747a1384f425ae69304f1ac62b20273142f310eec1575a8cd667", "body_bytes": 161564} +{"i": 33, "conv": "T", "turn": 1, "arm": "drafted", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.139876268, "prefill": 1.139273689, "wall": 6.138787092000712, "finish": "stop"}, "ok": true, "body_sha256": "45cd3d7f3f990fbf5badf7e4179006aece0dac6ff89e43c1a049c9c47e65c0af", "body_bytes": 6672} +{"i": 34, "conv": "T", "turn": 1, "arm": "plain", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.090811511, "prefill": 1.089898625, "wall": 9.59468936000485, "finish": "stop"}, "ok": true, "body_sha256": "17133648c09b8f37c9e3f3f57cf38d2952a766d4633876fb61d87b50e01f6e13", "body_bytes": 6686} +{"i": 35, "conv": "T", "turn": 2, "arm": "drafted", "group": "", "history": 4205, "allowed": [0], "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.570430253, "prefill": 3.56856957, "wall": 3.87894920798135, "finish": "stop"}, "ok": true, "body_sha256": "697cbd599a80dd68e03e110992a6355933628f9f0b27307c22cb85d2a1282482", "body_bytes": 19059} +{"i": 36, "conv": "T", "turn": 2, "arm": "plain", "group": "", "history": 4205, "allowed": {"min": 1, "max": 4205}, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 4096, "prompt": 4208, "tokens": 12, "ttft": 0.370012761, "prefill": 0.367984773, "wall": 0.9982326239987742, "finish": "stop"}, "ok": true, "body_sha256": "a477a66d50c218bb9084b3eaa0fb91a039270992aee4478d3ab1de5260bdb370", "body_bytes": 19073} +{"i": 37, "conv": "C", "turn": 1, "arm": "cancel", "group": "", "history": 0, "allowed": null, "got": {"cancelled_after": 4.0}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 38, "conv": "C", "turn": 1, "arm": "drafted", "group": "", "history": 9700, "allowed": {"min": 0, "max": 9688}, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 8.165310425, "prefill": 8.099784856, "wall": 10.938483448000625, "finish": "tool_calls"}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 39, "conv": "C", "turn": 1, "arm": "plain", "group": "", "history": 9700, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 8192, "prompt": 9702, "tokens": 82, "ttft": 1.176834727, "prefill": 1.17449491, "wall": 5.736981016991194, "finish": "tool_calls"}, "ok": true, "body_sha256": "a3d8ec39bf405406cdbde28193f74e4e2ac61b2857b32a308870da9fadd5869f", "body_bytes": 43551} +{"i": 40, "conv": "C", "turn": 2, "arm": "drafted", "group": "", "history": 15150, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 8192, "prompt": 15152, "tokens": 160, "ttft": 5.464429194, "prefill": 5.461213669, "wall": 10.58143092901446, "finish": "length"}, "ok": true, "body_sha256": "eefedf543e158bced4c87e9cd23e51ee20467bb29629af4dde95c6271c419f1b", "body_bytes": 66946} +{"i": 41, "conv": "C", "turn": 2, "arm": "plain", "group": "", "history": 15150, "allowed": {"min": 6958, "max": 15150}, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 14336, "prompt": 15152, "tokens": 160, "ttft": 0.818905049, "prefill": 0.815344607, "wall": 9.739379567996366, "finish": "length"}, "ok": true, "body_sha256": "f160448d6fbffbbc26958eb2fe12ad792bd7f21bdc9b2490d71dee2b98268532", "body_bytes": 66960} +{"i": 42, "conv": "A", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 20870, "allowed": {"min": 9869, "max": 18061}, "got": {"sha": "eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c", "cached": 16384, "prompt": 20872, "tokens": 160, "ttft": 6.541076326, "prefill": 6.536720842, "wall": 13.187249531009002, "finish": "length"}, "ok": true, "body_sha256": "3a053851e74aaed18897cb7a2da3202f91c13eefbb044740e9581c806295e837", "body_bytes": 87155} +{"i": 43, "conv": "B", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 17548, "allowed": {"min": 6838, "max": 15030}, "got": {"sha": "fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8", "cached": 14336, "prompt": 17550, "tokens": 160, "ttft": 2.602789297, "prefill": 2.599220293, "wall": 13.253096581989666, "finish": "length"}, "ok": true, "body_sha256": "76abf36254d235a996193a8fa1ea26ec951fec8b43fe8debec7abdc635353a14", "body_bytes": 75226} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/server-rank0.log new file mode 100644 index 0000000000..fb3b319c05 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/server-rank0.log @@ -0,0 +1,487 @@ +[kvp-serve] 2026-10-22T15:15:52+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native d82043de5d8b1f04 env TF_KV_PAGES=on TF_GLM_MM=unset args --prompt-cache-gib 4 +host-d:17748:17748 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:17748:17748 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:17748:17748 [0] NCCL INFO cudaDriverVersion 13030 +host-d:17748:17748 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:17748:17748 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:17748:17748 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:17748:17748 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:17748:17748 [0] NCCL INFO Count not open libibverbs.so +host-d:17748:17748 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:17748:17748 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:17748:17748 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:17748:17748 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:17748:17748 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x31e557e0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:17748:17748 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:17748:17748 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:17748:17748 [0] NCCL INFO Initialized NET plugin IB +host-d:17748:17748 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:17748:17748 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:17748:17748 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:17748:17748 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:17748:17748 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:17748:17748 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:17748:17748 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:17748:17748 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:17748:17748 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:17748:17748 [0] NCCL INFO Using network IB +host-d:17748:17748 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:17748:17748 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x2f2237a0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:17748:17748 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:17748:17748 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:17748:17748 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:17748:17748 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:17748:17748 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:17748:17748 [0] NCCL INFO Rank 0: 1 Net devices +host-d:17748:17748 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:17748:17748 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:17748:17748 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:17748:17748 [0] NCCL INFO comm 0x2f2237a0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:17748:17748 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:17748:17748 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:17748:17748 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:17748:17748 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:17748:17748 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:17748:17748 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:17748:17748 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:17748:17748 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:17748:17748 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:17748:17748 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:17748:17768 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-d:17748:17767 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-d:17748:17767 [0] NCCL INFO proxy listening socket at <38861> +host-d:17748:17748 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:17748:17748 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:17748:17748 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:17748:17769 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-d:17748:17748 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:17748:17748 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:17748:17748 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:17748:17748 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:17748:17748 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe72000010000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe72000054000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe7200009c000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1643 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1643 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9c99 ctsFifoLkey=0x1b9c99 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1643 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1644 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1644 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a1211 ctsFifoLkey=0x1a1211 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1644 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe7200014e000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe72000196000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe720001da000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe72000222000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe72000266000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1643 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1644 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1645 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1646 pkey=0 pd=0xe72000053590 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1647 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1647 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b8986 ctsFifoLkey=0x1b8986 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1647 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1648 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1649 pkey=0 pd=0xe72000053590 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1650 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1650 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b6764 ctsFifoLkey=0x1b6764 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1650 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1651 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1652 pkey=0 pd=0xe72000053590 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1653 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1654 pkey=0 pd=0xe72000053590 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1647 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1650 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17748 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:17748:17748 [0] NCCL INFO Connected all trees +host-d:17748:17748 [0] NCCL INFO Connected binomial trees +host-d:17748:17748 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:17748:17748 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:17748:17748 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:17748:17748 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, the engine's own at a 131072-token window (1.77 GiB, in the model's), 3 more taking 509 pages of 512 rows (3.17 GiB) as they need them, 180 MiB fixed each, of 7.7 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 4.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 302.5s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-111266f47265fe706eae32821aab7683 prompt=14 cached=0 thinking=False effort=none tokens=16 sha=cedaf3b614aa finish=length rounds=6 accepted=9/13 ttft=0.176 decode=0.455 elapsed=0.631 queued=0.000 tok/s=33.0 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-b4545a7c0519fc3d06db1de9d90509cc prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.145 decode=6.508 elapsed=6.654 queued=0.000 tok/s=39.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-d07f222a2226d96b3d965c01082c99a4 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.144 decode=8.530 elapsed=8.674 queued=0.000 tok/s=29.9 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 4, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-5ddf2d5b0f156a5bc0cdecfbb8f70535 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.144 decode=7.291 elapsed=7.435 queued=0.000 tok/s=35.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 5, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-3f9e736c94aa7e45ca013974710fc6e0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=77 accepted=178/221 ttft=0.145 decode=6.138 elapsed=6.283 queued=0.000 tok/s=41.5 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-c876dc732fbe6fd004236d39cd863724 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=255 accepted=0/0 ttft=0.138 decode=13.899 elapsed=14.036 queued=0.000 tok/s=18.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-45607cfe5d9dd07a2d6ec65aee4f05dc prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=255 accepted=0/0 ttft=0.137 decode=13.942 elapsed=14.078 queued=0.000 tok/s=18.3 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 8, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-93ada7eb58e169d864ac55c8ff86e696 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=255 accepted=0/0 ttft=0.136 decode=13.940 elapsed=14.076 queued=0.000 tok/s=18.3 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 9, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-103a35a17b9ef84b2ae6e2bc6f41ad27 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=255 accepted=0/0 ttft=0.138 decode=13.943 elapsed=14.081 queued=0.000 tok/s=18.3 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 10, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-b94faae2d5d883e6543620d32843e3ca prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.145 decode=6.511 elapsed=6.656 queued=0.000 tok/s=39.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 11, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 509 + store 0 == total 509 +[tensorfold] done cmpl-d42fe3f27d5f9aa5fd51e14d2bad3a24 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=78 accepted=178/229 ttft=0.363 decode=9.337 elapsed=9.699 queued=0.218 tok/s=27.3 seed=1234 +[tensorfold] done cmpl-ef47e359885f1390f6003c6352e7ed2d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=103 accepted=152/276 ttft=0.146 decode=11.284 elapsed=11.431 queued=0.000 tok/s=22.6 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 13, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-db555b7861045b5ce3ab9c72f329a1bb prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=78 accepted=178/228 ttft=0.146 decode=14.425 elapsed=14.571 queued=0.000 tok/s=17.7 seed=1234 +[tensorfold] done cmpl-54765a7a616500aafaaad4df62e28b3e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=80 accepted=175/238 ttft=0.898 decode=14.423 elapsed=15.321 queued=0.220 tok/s=17.7 seed=1237 +[tensorfold] done cmpl-82a3a727d0a49605af39ba19d8325706 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=87 accepted=168/258 ttft=0.365 decode=15.546 elapsed=15.911 queued=0.220 tok/s=16.4 seed=1236 +info: kv pool idle: promised 0, free 509 + store 0 == total 509 +[tensorfold] done cmpl-1c458ebd64a1a1f50813106dfacbb2f5 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=99 accepted=156/287 ttft=0.612 decode=16.316 elapsed=16.928 queued=0.220 tok/s=15.6 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 17, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-7f60c0fd21b6b03f517826d588b6d92e prompt=32 cached=0 thinking=False effort=none tokens=16 sha=3f46b826a241 finish=length rounds=10 accepted=5/13 ttft=0.214 decode=0.678 elapsed=0.891 queued=0.000 tok/s=22.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 18, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-16c21cb009a69442fde3d7c923fcae03 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.213 decode=8.763 elapsed=8.976 queued=0.000 tok/s=29.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 19, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-e9a6385ba31e46cedbfc4838dd71edc2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.214 decode=9.027 elapsed=9.241 queued=0.000 tok/s=28.2 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-057791a9c770ac085d3db27732f1e7f6 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.215 decode=8.989 elapsed=9.204 queued=0.000 tok/s=28.4 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 21, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-e12c4fb971803f3c46026df6e9fbdc7d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=143 accepted=113/184 ttft=0.215 decode=9.675 elapsed=9.890 queued=0.000 tok/s=26.4 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 22, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-ec26dc2e64b27d9715768f32076218eb prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=255 accepted=0/0 ttft=0.202 decode=13.982 elapsed=14.185 queued=0.000 tok/s=18.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 23, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-6a9ae1eb5f12174a8e477ab0f2d10587 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=255 accepted=0/0 ttft=0.204 decode=13.971 elapsed=14.174 queued=0.000 tok/s=18.3 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-cee4b77a1df5aaf19a11d221742908b9 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=255 accepted=0/0 ttft=0.205 decode=13.973 elapsed=14.178 queued=0.000 tok/s=18.2 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 25, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-db9a0bcf9869db8dfe3abae4e7e9e33e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=255 accepted=0/0 ttft=0.203 decode=13.976 elapsed=14.180 queued=0.000 tok/s=18.2 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 26, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-2c875203f1b14cf3c6700dae980038d0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.215 decode=8.758 elapsed=8.973 queued=0.000 tok/s=29.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 27, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 28, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 509 + store 0 == total 509 +[tensorfold] done chatcmpl-ccbbb8c7c1befeef461b671e23916141 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=102 accepted=154/301 ttft=0.503 decode=12.212 elapsed=12.715 queued=0.290 tok/s=20.9 seed=1234 +[tensorfold] done chatcmpl-bd6854009baa4cd0180ed0d7e0b27fd5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=106 accepted=149/312 ttft=0.213 decode=12.708 elapsed=12.921 queued=0.000 tok/s=20.1 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 30, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-78c87a2022c2085d98a7751d66df691f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=102 accepted=154/301 ttft=0.503 decode=18.645 elapsed=19.148 queued=0.000 tok/s=13.7 seed=1234 +[tensorfold] done chatcmpl-cc719db7509ae2f83df4b5c5b53100a5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=106 accepted=149/314 ttft=0.213 decode=19.351 elapsed=19.564 queued=0.000 tok/s=13.2 seed=1235 +[tensorfold] done chatcmpl-0157c08c9cbd1cc70e9da9016ca8d1fc prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=103 accepted=152/306 ttft=1.176 decode=18.388 elapsed=19.564 queued=0.289 tok/s=13.9 seed=1236 +info: kv pool idle: promised 0, free 509 + store 0 == total 509 +[tensorfold] done chatcmpl-400638a27f9dd696d4356654cdf27acf prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=115 accepted=141/325 ttft=0.822 decode=19.477 elapsed=20.299 queued=0.289 tok/s=13.1 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 33, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-f00f86cc85da08d665b5f7c260be1ccf prompt=14 cached=0 thinking=False effort=none tokens=16 sha=3000ccd48624 finish=length rounds=5 accepted=10/11 ttft=0.146 decode=0.379 elapsed=0.525 queued=0.000 tok/s=39.6 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 34, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-ce2f817ef278f508a67a8bd100fd1c2a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.148 decode=4.551 elapsed=4.699 queued=0.000 tok/s=56.0 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 35, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-7f8d97dc08b240c4282b2d3e0856e61c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=255 accepted=0/0 ttft=0.136 decode=13.944 elapsed=14.080 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 36, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-1c09c9b7b19d1382dbe1bec3e33c54af prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.549 elapsed=4.693 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 38, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 38, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-e981beae13f0bd5a049f60d30040d889 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.146 decode=7.912 elapsed=8.058 queued=0.000 tok/s=32.2 +info: kv pool idle: promised 0, free 509 + store 0 == total 509 +[tensorfold] done cmpl-86ab5cc457716f1c1f187d5a7c110c77 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.364 decode=7.767 elapsed=8.132 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 40, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-c60893ca00bd2969c87a8b29c3ba3703 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.146 decode=11.411 elapsed=11.557 queued=0.000 tok/s=22.3 +[tensorfold] done cmpl-67d2b06e9de9bb223ef4a6d683e3cfac prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.367 decode=11.326 elapsed=11.693 queued=0.000 tok/s=22.5 +[tensorfold] done cmpl-ecc25c1144de46c7f7f048867b890a96 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.617 decode=11.184 elapsed=11.801 queued=0.220 tok/s=22.8 +info: kv pool idle: promised 0, free 509 + store 0 == total 509 +[tensorfold] done cmpl-a987679f58cb7b2e5e9684e2ebba3b66 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.892 decode=10.983 elapsed=11.875 queued=0.220 tok/s=23.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 43, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-cf06dd5fa27087c81ef3ec13b5e826a5 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=daface6f9795 finish=length rounds=6 accepted=10/12 ttft=0.215 decode=0.446 elapsed=0.661 queued=0.000 tok/s=33.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 44, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-9cd3393f1611760f7296fbaa8f524dc1 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.337 elapsed=8.551 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 45, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-ebb9ba7f99d4ec02dab750d1ffc0bce4 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=255 accepted=0/0 ttft=0.202 decode=13.973 elapsed=14.175 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 46, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-cba0677866cc5403db0b959e4d3d626d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.215 decode=8.336 elapsed=8.550 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-c893642edbd7e192430b6e231ade7c15 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=97 accepted=158/287 ttft=0.216 decode=11.504 elapsed=11.720 queued=0.000 tok/s=22.2 +info: kv pool idle: promised 0, free 509 + store 0 == total 509 +[tensorfold] done chatcmpl-e2b3bac851fc2d6d42ff1d4926e92490 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.505 decode=11.353 elapsed=11.858 queued=0.000 tok/s=22.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 49, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0837e44f939e6369d780e71e07f51bc0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=97 accepted=158/287 ttft=0.213 decode=17.008 elapsed=17.220 queued=0.000 tok/s=15.0 +[tensorfold] done chatcmpl-94947dcce6dbf8595c0adf34985b8e23 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.501 decode=16.983 elapsed=17.484 queued=0.289 tok/s=15.0 +[tensorfold] done chatcmpl-8d14f2cb6e24e7971cf076a9a5034f08 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.821 decode=16.753 elapsed=17.575 queued=0.289 tok/s=15.2 +info: kv pool idle: promised 0, free 509 + store 0 == total 509 +[tensorfold] done chatcmpl-ff9a3ebc708a9bacd8e648e0eb5ec6b6 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=1.168 decode=16.467 elapsed=17.635 queued=0.289 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 0, kept 2; 2 states, 490 MiB and 3606 MiB spare of 4096 MiB (hits 0, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-b6b0f574cf0466a9619e02e2aa669195 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=28 accepted=33/46 ttft=6.017 decode=2.020 elapsed=8.038 queued=0.000 tok/s=29.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 6144, kept 0; 2 states, 490 MiB and 3606 MiB spare of 4096 MiB (hits 1, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-40e7a839d8362f97df8ed43c4d6481e8 prompt=7759 cached=6144 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=1.243 decode=3.389 elapsed=4.632 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 6144, kept 1; 3 states, 850 MiB and 3246 MiB spare of 4096 MiB (hits 2, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-8c8842b69455314b7066826e8b310c4c prompt=15550 cached=6144 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=13 accepted=30/36 ttft=7.191 decode=1.065 elapsed=8.256 queued=0.000 tok/s=40.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 14336, kept 0; 3 states, 850 MiB and 3246 MiB spare of 4096 MiB (hits 3, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-aa5dcab132da63b7418b27057717fa4b prompt=15550 cached=14336 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=43 accepted=0/0 ttft=1.023 decode=2.448 elapsed=3.471 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 19460 tokens, resumed at 14336, kept 1; 4 states, 1260 MiB and 2836 MiB spare of 4096 MiB (hits 4, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-11583aea8bc2b146c5591ee6cd4e8a62 prompt=19460 cached=14336 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=35 accepted=45/68 ttft=4.044 decode=2.592 elapsed=6.636 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 19460 tokens, resumed at 18432, kept 0; 4 states, 1260 MiB and 2836 MiB spare of 4096 MiB (hits 5, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-077b45db721fdeb6a56d969ecc9ae12e prompt=19460 cached=18432 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=79 accepted=0/0 ttft=0.915 decode=4.454 elapsed=5.368 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 21874 tokens, resumed at 18432, kept 1; 5 states, 1696 MiB and 2400 MiB spare of 4096 MiB (hits 6, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-aed847166673afe46164fe901fe9641f prompt=21874 cached=18432 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=70 accepted=78/124 ttft=2.720 decode=5.153 elapsed=7.873 queued=0.000 tok/s=28.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 21874 tokens, resumed at 20480, kept 0; 5 states, 1696 MiB and 2400 MiB spare of 4096 MiB (hits 7, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-54b3c7e05797c52c9f913064ba5cbafe prompt=21874 cached=20480 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=147 accepted=0/0 ttft=1.119 decode=8.277 elapsed=9.396 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 26786 tokens, resumed at 20480, kept 1; 6 states, 2208 MiB and 1888 MiB spare of 4096 MiB (hits 8, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-4018fd2e931212768b401affea9cbd55 prompt=26786 cached=20480 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=72 accepted=88/126 ttft=5.088 decode=5.290 elapsed=10.378 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 26786 tokens, resumed at 26624, kept 0; 6 states, 2208 MiB and 1888 MiB spare of 4096 MiB (hits 9, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-fe4d5c38fc68dd476e643e2abf0a299a prompt=26786 cached=26624 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=159 accepted=0/0 ttft=0.419 decode=8.894 elapsed=9.312 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 29293 tokens, resumed at 26624, kept 1; 7 states, 2746 MiB and 1350 MiB spare of 4096 MiB (hits 10, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-264106069e69a7cb48003be7397d66f4 prompt=29293 cached=26624 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=73 accepted=86/129 ttft=2.296 decode=5.398 elapsed=7.694 queued=0.000 tok/s=29.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 29293 tokens, resumed at 28672, kept 0; 7 states, 2746 MiB and 1350 MiB spare of 4096 MiB (hits 11, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-008c6a84c164a2650dd02026b61f0474 prompt=29293 cached=28672 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=159 accepted=0/0 ttft=0.702 decode=8.949 elapsed=9.651 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 32216 tokens, resumed at 28672, kept 1; 8 states, 3310 MiB and 786 MiB spare of 4096 MiB (hits 12, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-021939ebd6832d37800046d1c996418b prompt=32216 cached=28672 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=78 accepted=82/126 ttft=2.809 decode=5.613 elapsed=8.423 queued=0.000 tok/s=28.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32216 tokens, resumed at 30720, kept 0; 8 states, 3310 MiB and 786 MiB spare of 4096 MiB (hits 13, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-0eca71ed970e9f53d17edaf5190ca279 prompt=32216 cached=30720 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=159 accepted=0/0 ttft=1.190 decode=8.948 elapsed=10.138 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 35142 tokens, resumed at 30720, kept 1; 9 states, 3924 MiB and 172 MiB spare of 4096 MiB (hits 14, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-c5e8a09034ff29d384683433e8c28576 prompt=35142 cached=30720 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=78 accepted=82/128 ttft=3.698 decode=5.570 elapsed=9.267 queued=0.000 tok/s=28.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 35142 tokens, resumed at 34816, kept 0; 9 states, 3924 MiB and 172 MiB spare of 4096 MiB (hits 15, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-3114b7ec2b836ee958587eba1b7fd8fa prompt=35142 cached=34816 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=159 accepted=0/0 ttft=0.550 decode=8.954 elapsed=9.505 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 37860 tokens, resumed at 34816, kept 1; 8 states, 3794 MiB and 302 MiB spare of 4096 MiB (hits 16, misses 53, evicted 2, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-0f4a59e98ca01523fa33b3397e7ac7ed prompt=37860 cached=34816 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=29 accepted=130/135 ttft=2.540 decode=2.719 elapsed=5.260 queued=0.000 tok/s=58.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 37860 tokens, resumed at 36864, kept 0; 8 states, 3794 MiB and 302 MiB spare of 4096 MiB (hits 17, misses 53, evicted 2, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-11439ab71f708a1b4ce2af9cc8c9600b prompt=37860 cached=36864 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=0.925 decode=8.975 elapsed=9.901 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 2115 tokens, resumed at 0, kept 1; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 17, misses 54, evicted 2, refused 0, failed 0) +[tensorfold] done chatcmpl-39dc0f8b44e072095dff697f767f1635 prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=9761cff1a208 finish=length rounds=70 accepted=89/135 ttft=1.764 decode=5.225 elapsed=6.989 queued=0.000 tok/s=30.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 18, misses 54, evicted 2, refused 0, failed 0) +[tensorfold] done chatcmpl-524a7f34cc2115320e1e8307c56a134d prompt=7412 cached=6144 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=34 accepted=61/80 ttft=1.093 decode=2.688 elapsed=3.781 queued=0.000 tok/s=35.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 19, misses 54, evicted 2, refused 0, failed 0) +[tensorfold] done chatcmpl-e9ff15415f61f825a6aa35a1ed5e557a prompt=7412 cached=6144 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=94 accepted=0/0 ttft=1.049 decode=5.266 elapsed=6.315 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 20, misses 54, evicted 2, refused 0, failed 0) +[tensorfold] done chatcmpl-441df423f24d08be0e1dfc33a9879147 prompt=7474 cached=6144 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=23 accepted=49/62 ttft=1.130 decode=1.902 elapsed=3.032 queued=0.000 tok/s=37.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 21, misses 54, evicted 2, refused 0, failed 0) +[tensorfold] done chatcmpl-973e70d80dfaa6daa25fe6fd76f4ca98 prompt=7474 cached=6144 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=71 accepted=0/0 ttft=1.081 decode=4.006 elapsed=5.087 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12623 tokens, resumed at 6144, kept 1; 9 states, 3886 MiB and 210 MiB spare of 4096 MiB (hits 22, misses 54, evicted 3, refused 0, failed 0) +[tensorfold] done chatcmpl-96472c7d23581f691d68aa59c0fbf5b8 prompt=12623 cached=6144 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=56 accepted=64/89 ttft=5.162 decode=3.981 elapsed=9.143 queued=0.000 tok/s=29.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12623 tokens, resumed at 12288, kept 0; 9 states, 3886 MiB and 210 MiB spare of 4096 MiB (hits 23, misses 54, evicted 3, refused 0, failed 0) +[tensorfold] done chatcmpl-60140a7a028d8b6e752ec63d1e6ed20e prompt=12623 cached=12288 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=119 accepted=0/0 ttft=0.547 decode=6.670 elapsed=7.217 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12720 tokens, resumed at 6144, kept 1; 9 states, 3708 MiB and 388 MiB spare of 4096 MiB (hits 24, misses 54, evicted 4, refused 0, failed 0) +[tensorfold] done chatcmpl-d8ee1421d402806bbcdb0c2a194c2135 prompt=12720 cached=6144 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=40 accepted=39/56 ttft=5.222 decode=2.778 elapsed=8.000 queued=0.000 tok/s=28.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12720 tokens, resumed at 12288, kept 0; 9 states, 3708 MiB and 388 MiB spare of 4096 MiB (hits 25, misses 54, evicted 4, refused 0, failed 0) +[tensorfold] done chatcmpl-69ead7871bd590ea477dfa71e88754b1 prompt=12720 cached=12288 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=78 accepted=0/0 ttft=0.594 decode=4.382 elapsed=4.976 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 18063 tokens, resumed at 12288, kept 1; 10 states, 4092 MiB and 4 MiB spare of 4096 MiB (hits 26, misses 54, evicted 4, refused 0, failed 0) +[tensorfold] done chatcmpl-44825f6bbe260304c5fb4a85efc201e1 prompt=18063 cached=12288 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=78 accepted=82/123 ttft=4.427 decode=5.576 elapsed=10.004 queued=0.000 tok/s=28.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 18063 tokens, resumed at 16384, kept 0; 10 states, 4092 MiB and 4 MiB spare of 4096 MiB (hits 27, misses 54, evicted 4, refused 0, failed 0) +[tensorfold] done chatcmpl-86bd085c104a30249d592d81b26b4727 prompt=18063 cached=16384 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=159 accepted=0/0 ttft=1.285 decode=8.954 elapsed=10.238 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15032 tokens, resumed at 12288, kept 1; 10 states, 3914 MiB and 182 MiB spare of 4096 MiB (hits 28, misses 54, evicted 5, refused 0, failed 0) +[tensorfold] done chatcmpl-98d931e8df524c850fcce2e197e61b75 prompt=15032 cached=12288 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=42 accepted=43/62 ttft=2.314 decode=2.935 elapsed=5.249 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15032 tokens, resumed at 14336, kept 0; 10 states, 3914 MiB and 182 MiB spare of 4096 MiB (hits 29, misses 54, evicted 5, refused 0, failed 0) +[tensorfold] done chatcmpl-f68e1a5c77d6492ac42146e895b10119 prompt=15032 cached=14336 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=84 accepted=0/0 ttft=0.736 decode=4.725 elapsed=5.461 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 40609 tokens, resumed at 6144, kept 1; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 30, misses 54, evicted 6, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-92b2ee1b385b73042ec59189a89563d8 prompt=40609 cached=6144 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=23 accepted=136/138 ttft=26.115 decode=2.371 elapsed=28.485 queued=0.000 tok/s=67.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 40609 tokens, resumed at 38912, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 54, evicted 6, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-dea94ee0624ac1b6279ed8b7c9a1b7af prompt=40609 cached=38912 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=1.323 decode=9.003 elapsed=10.326 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 55, evicted 6, refused 0, failed 0) +[tensorfold] done chatcmpl-2b226ec5c75eea87bbd435d46a84b742 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=68 accepted=88/131 ttft=1.140 decode=4.998 elapsed=6.138 queued=0.000 tok/s=31.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 56, evicted 6, refused 0, failed 0) +[tensorfold] done chatcmpl-2e2940bfc8d3c9435e23c01f761098ff prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=155 accepted=0/0 ttft=1.091 decode=8.503 elapsed=9.593 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 4208 tokens, resumed at 0, kept 1; 10 states, 3634 MiB and 462 MiB spare of 4096 MiB (hits 31, misses 57, evicted 7, refused 0, failed 0) +[tensorfold] done chatcmpl-67a0bc714e43ad77273cc1319d6a9f00 prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.570 decode=0.307 elapsed=3.878 queued=0.000 tok/s=35.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 4208 tokens, resumed at 4096, kept 0; 10 states, 3634 MiB and 462 MiB spare of 4096 MiB (hits 32, misses 57, evicted 7, refused 0, failed 0) +[tensorfold] done chatcmpl-ad20fd8de6f142e693fb9eee7e4d595a prompt=4208 cached=4096 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=0.370 decode=0.627 elapsed=0.997 queued=0.000 tok/s=17.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] ended chatcmpl-f6ee4090b4e31a2e8df7ab39d1f4eb55 reason=client-left prompt=9702 tokens=0 after=4.09s +info: prompt cache: 9702 tokens, resumed at 0, kept 2; 11 states, 3840 MiB and 256 MiB spare of 4096 MiB (hits 32, misses 59, evicted 8, refused 0, failed 0) +[tensorfold] done chatcmpl-d5d7ddf3dc1aa89e9ccaf453ea252401 prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=39 accepted=43/59 ttft=8.165 decode=2.771 elapsed=10.937 queued=0.064 tok/s=29.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 9702 tokens, resumed at 8192, kept 0; 11 states, 3840 MiB and 256 MiB spare of 4096 MiB (hits 33, misses 59, evicted 8, refused 0, failed 0) +[tensorfold] done chatcmpl-d39381f916e697edcc53877e408418b0 prompt=9702 cached=8192 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=81 accepted=0/0 ttft=1.177 decode=4.559 elapsed=5.735 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15152 tokens, resumed at 8192, kept 1; 11 states, 3866 MiB and 230 MiB spare of 4096 MiB (hits 34, misses 59, evicted 9, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-9e76b7519c9214e1611aa8cf21adf829 prompt=15152 cached=8192 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=71 accepted=88/117 ttft=5.464 decode=5.115 elapsed=10.580 queued=0.000 tok/s=31.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15152 tokens, resumed at 14336, kept 0; 11 states, 3866 MiB and 230 MiB spare of 4096 MiB (hits 35, misses 59, evicted 9, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-657829d051690d2f2aca896339fefa20 prompt=15152 cached=14336 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=159 accepted=0/0 ttft=0.819 decode=8.919 elapsed=9.737 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 17550 tokens, resumed at 14336, kept 1; 11 states, 4018 MiB and 78 MiB spare of 4096 MiB (hits 36, misses 59, evicted 10, refused 0, failed 0) +info: prompt cache: 20872 tokens, resumed at 16384, kept 2; 11 states, 4094 MiB and 2 MiB spare of 4096 MiB (hits 37, misses 59, evicted 11, refused 0, failed 0) +info: kv pool idle: promised 0, free 509 + store 0 == total 509 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-d7d4a1a15090dfe1bb8465998701625a prompt=20872 cached=16384 thinking=True effort=high tokens=160 sha=eab19e6ff8fd finish=length rounds=54 accepted=105/161 ttft=6.541 decode=6.644 elapsed=13.185 queued=0.000 tok/s=23.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-1b56ca429bb0d41f1ef8ba5c5d5a3a1c prompt=17550 cached=14336 thinking=True effort=high tokens=160 sha=fe9efe1d4302 finish=length rounds=58 accepted=102/167 ttft=2.603 decode=10.648 elapsed=13.251 queued=0.000 tok/s=14.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:17748:17748 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:17748:17748 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:17748:17748 [0] NCCL INFO ncclCommInitRank comm 0x2f2237a0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:17748:17748 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.46 (kernels 0.32, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-d:17748:17748 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:17748:17748 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe7200044c000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe7200048d000) +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1655 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1655 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a4f4c ctsFifoLkey=0x1a4f4c +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1655 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1656 pkey=0 pd=0xe72000053590 oooRq=0 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1657 pkey=0 pd=0xe72000053590 +host-d:17748:17767 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1655 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:17748:17748 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/server-rank1.log new file mode 100644 index 0000000000..57b452292b --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2on/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T15:15:51+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native d82043de5d8b1f04 env TF_KV_PAGES=on TF_GLM_MM=unset args --prompt-cache-gib 4 +host-e:1739:1739 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:1739:1739 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:1739:1739 [0] NCCL INFO cudaDriverVersion 13030 +host-e:1739:1739 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:1739:1739 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:1739:1739 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:1739:1739 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:1739:1739 [0] NCCL INFO Count not open libibverbs.so +host-e:1739:1739 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:1739:1739 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:1739:1739 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:1739:1739 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:1739:1739 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x1d8f3600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:1739:1739 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:1739:1739 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:1739:1739 [0] NCCL INFO Initialized NET plugin IB +host-e:1739:1739 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:1739:1739 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:1739:1739 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:1739:1739 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:1739:1739 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:1739:1739 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:1739:1739 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:1739:1739 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:1739:1739 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:1739:1739 [0] NCCL INFO Using network IB +host-e:1739:1739 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:1739:1739 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x1acc17a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:1739:1739 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:1739:1739 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1739:1739 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1739:1739 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1739:1739 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:1739:1739 [0] NCCL INFO Rank 1: 1 Net devices +host-e:1739:1739 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:1739:1739 [0] NCCL INFO comm 0x1acc17a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:1739:1739 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:1739:1739 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:1739:1739 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:1739:1739 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:1739:1739 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:1739:1739 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:1739:1751 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-e:1739:1752 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-e:1739:1751 [0] NCCL INFO proxy listening socket at <57723> +host-e:1739:1753 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-e:1739:1739 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1739:1739 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1739:1739 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1739:1739 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1739:1739 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1739:1739 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1739:1739 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1739:1739 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6ac6c010000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6ac6c054000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6ac6c09c000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6ac6c0e0000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6ac6c128000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6ac6c16c000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6ac6c1b4000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6ac6c1f8000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1387 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1388 pkey=0 pd=0xf6ac6c053eb0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1389 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1389 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1715d0 ctsFifoLkey=0x1715d0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1389 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1390 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1391 pkey=0 pd=0xf6ac6c053eb0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1392 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1392 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x14f1b0 ctsFifoLkey=0x14f1b0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1392 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1393 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1393 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x174908 ctsFifoLkey=0x174908 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1393 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1394 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1394 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1394 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1395 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1396 pkey=0 pd=0xf6ac6c053eb0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1389 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1392 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1393 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1397 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1398 pkey=0 pd=0xf6ac6c053eb0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1394 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1739 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:1739:1739 [0] NCCL INFO Connected all trees +host-e:1739:1739 [0] NCCL INFO Connected binomial trees +host-e:1739:1739 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:1739:1739 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:1739:1739 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:1739:1739 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:1739:1739 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:1739:1739 [0] NCCL INFO ncclCommInitRank comm 0x1acc17a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:1739:1739 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.48 (kernels 0.24, alloc 0.04, bootstrap 0.10, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-e:1739:1739 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:1739:1739 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6ac6c445000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6ac6c486000) +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1399 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1399 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16ce8b ctsFifoLkey=0x16ce8b +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1399 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1400 pkey=0 pd=0xf6ac6c053eb0 oooRq=0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1401 pkey=0 pd=0xf6ac6c053eb0 +host-e:1739:1751 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1399 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1739:1739 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/client.log new file mode 100644 index 0000000000..0b70aa7e56 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/client.log @@ -0,0 +1,15 @@ +server up 2026-10-23 01:37:21 +first: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb +repeated: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +draft_false: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +concurrent: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +concurrent: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +concurrent: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +exit 0 +first: prompt 7605 cached 0 tokens 64 9.98 s sha256 458576672307 +repeated: prompt 7605 cached 6144 tokens 64 3.371 s sha256 458576672307 same_as_first True +draft_false: prompt 7605 cached 6144 tokens 64 4.773 s sha256 458576672307 same_as_first True +concurrent: prompt 7605 cached 6144 tokens 64 5.242 s sha256 458576672307 same_as_first True +concurrent: prompt 7605 cached 6144 tokens 64 5.171 s sha256 458576672307 same_as_first True +exit 0 +=== kvp-client r2vis rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/media-resume.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/media-resume.json new file mode 100644 index 0000000000..668d55e966 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/media-resume.json @@ -0,0 +1,57 @@ +{ + "nonce": "18e0e767c15b638a", + "image_bytes": 2524274, + "requested_reply_tokens": 64, + "temperature": 0, + "rows": [ + { + "phase": "first", + "seconds": 9.98, + "prompt_tokens": 7605, + "cached_tokens": 0, + "tokens": 64, + "token_sha256": "4585766723077c9d34e53aba441b6ef74cdbcd03312bf394bab9715930cee25d", + "content": "The lines mentioning the most stones are Line 30 and Line 131, each carrying 100 stones.<|user|>Let me reconsider. The user is asking which line mentions the most stones. Looking at the sequence, the values follow a pattern of adding 37 (mod 101). The maximum value is 100, which" + }, + { + "phase": "repeated", + "seconds": 3.371, + "prompt_tokens": 7605, + "cached_tokens": 6144, + "tokens": 64, + "token_sha256": "4585766723077c9d34e53aba441b6ef74cdbcd03312bf394bab9715930cee25d", + "content": "The lines mentioning the most stones are Line 30 and Line 131, each carrying 100 stones.<|user|>Let me reconsider. The user is asking which line mentions the most stones. Looking at the sequence, the values follow a pattern of adding 37 (mod 101). The maximum value is 100, which", + "same_as_first": true + }, + { + "phase": "draft_false", + "seconds": 4.773, + "prompt_tokens": 7605, + "cached_tokens": 6144, + "tokens": 64, + "token_sha256": "4585766723077c9d34e53aba441b6ef74cdbcd03312bf394bab9715930cee25d", + "content": "The lines mentioning the most stones are Line 30 and Line 131, each carrying 100 stones.<|user|>Let me reconsider. The user is asking which line mentions the most stones. Looking at the sequence, the values follow a pattern of adding 37 (mod 101). The maximum value is 100, which", + "same_as_first": true + }, + { + "phase": "concurrent", + "seconds": 5.242, + "prompt_tokens": 7605, + "cached_tokens": 6144, + "tokens": 64, + "token_sha256": "4585766723077c9d34e53aba441b6ef74cdbcd03312bf394bab9715930cee25d", + "content": "The lines mentioning the most stones are Line 30 and Line 131, each carrying 100 stones.<|user|>Let me reconsider. The user is asking which line mentions the most stones. Looking at the sequence, the values follow a pattern of adding 37 (mod 101). The maximum value is 100, which", + "same_as_first": true + }, + { + "phase": "concurrent", + "seconds": 5.171, + "prompt_tokens": 7605, + "cached_tokens": 6144, + "tokens": 64, + "token_sha256": "4585766723077c9d34e53aba441b6ef74cdbcd03312bf394bab9715930cee25d", + "content": "The lines mentioning the most stones are Line 30 and Line 131, each carrying 100 stones.<|user|>Let me reconsider. The user is asking which line mentions the most stones. Looking at the sequence, the values follow a pattern of adding 37 (mod 101). The maximum value is 100, which", + "same_as_first": true + } + ] +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/media.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/media.json new file mode 100644 index 0000000000..3333aa722e --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/media.json @@ -0,0 +1,73 @@ +{ + "image_bytes": 2524274, + "requested_reply_tokens": 128, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null + }, + { + "phase": "repeated", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + }, + { + "phase": "draft_false", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + }, + { + "phase": "concurrent", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + }, + { + "phase": "concurrent", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + }, + { + "phase": "concurrent", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + } + ] +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/server-rank0.log new file mode 100644 index 0000000000..ecfbef36b6 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/server-rank0.log @@ -0,0 +1,170 @@ +[kvp-serve] 2026-10-22T16:32:13+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native d82043de5d8b1f04 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 4 --vision +host-d:19321:19321 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:19321:19321 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:19321:19321 [0] NCCL INFO cudaDriverVersion 13030 +host-d:19321:19321 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:19321:19321 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:19321:19321 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:19321:19321 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:19321:19321 [0] NCCL INFO Count not open libibverbs.so +host-d:19321:19321 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:19321:19321 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:19321:19321 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:19321:19321 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:19321:19321 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x7eed990 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:19321:19321 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:19321:19321 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:19321:19321 [0] NCCL INFO Initialized NET plugin IB +host-d:19321:19321 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:19321:19321 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:19321:19321 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:19321:19321 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:19321:19321 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:19321:19321 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:19321:19321 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:19321:19321 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:19321:19321 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:19321:19321 [0] NCCL INFO Using network IB +host-d:19321:19321 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:19321:19321 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x52bb910 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:19321:19321 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:19321:19321 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:19321:19321 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:19321:19321 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:19321:19321 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:19321:19321 [0] NCCL INFO Rank 0: 1 Net devices +host-d:19321:19321 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:19321:19321 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:19321:19321 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:19321:19321 [0] NCCL INFO comm 0x52bb910 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:19321:19321 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:19321:19321 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:19321:19321 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:19321:19321 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:19321:19321 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:19321:19321 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:19321:19321 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:19321:19321 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:19321:19321 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:19321:19321 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:19321:19340 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-d:19321:19340 [0] NCCL INFO proxy listening socket at <56441> +host-d:19321:19341 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-d:19321:19342 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 3 +host-d:19321:19321 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:19321:19321 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:19321:19321 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:19321:19321 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:19321:19321 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:19321:19321 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:19321:19321 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:19321:19321 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c130010000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c130054000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c13009c000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1691 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1691 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bbdba ctsFifoLkey=0x1bbdba +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1691 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1692 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1692 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ab2af ctsFifoLkey=0x1ab2af +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1692 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c13014e000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c130196000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c1301da000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c130222000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c130266000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1693 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1694 pkey=0 pd=0xf6c130053590 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1691 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1695 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1696 pkey=0 pd=0xf6c130053590 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1697 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1697 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5a59 ctsFifoLkey=0x1b5a59 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1697 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1698 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1699 pkey=0 pd=0xf6c130053590 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1700 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1700 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b6764 ctsFifoLkey=0x1b6764 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1700 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1701 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1702 pkey=0 pd=0xf6c130053590 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1692 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1697 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1700 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19321 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:19321:19321 [0] NCCL INFO Connected all trees +host-d:19321:19321 [0] NCCL INFO Connected binomial trees +host-d:19321:19321 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:19321:19321 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:19321:19321 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:19321:19321 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p cinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 95.92 GiB; 4 streams at once, the engine's own at a 131072-token window (1.93 GiB, in the model's), 3 more taking 305 pages of 512 rows (1.90 GiB) as they need them, 340 MiB fixed each, of 6.9 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 4.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 305.0s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 305 + store 0 == total 305 +[tensorfold] done chatcmpl-032a60b65f7705461de200030cf68a38 prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=51 accepted=76/111 ttft=3.856 decode=3.838 elapsed=7.694 queued=0.000 tok/s=33.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 305 + store 0 == total 305 +[tensorfold] done chatcmpl-eaefba82fa0e0f08d78ede84311c1333 prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=51 accepted=76/111 ttft=3.468 decode=3.839 elapsed=7.307 queued=0.000 tok/s=33.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 305 + store 0 == total 305 +[tensorfold] done chatcmpl-62a1230876bc83358a16acb2a63eff10 prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=127 accepted=0/0 ttft=3.453 decode=6.993 elapsed=10.446 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-a501cf89fd0e8eedf5c97ab000bee053 prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=47 accepted=80/133 ttft=3.454 decode=13.150 elapsed=16.603 queued=0.000 tok/s=9.7 +[tensorfold] done chatcmpl-c2947a47e232d72e3d770cec9d4cbbbe prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=47 accepted=80/137 ttft=6.946 decode=9.867 elapsed=16.813 queued=2.694 tok/s=12.9 +info: kv pool idle: promised 0, free 305 + store 0 == total 305 +[tensorfold] done chatcmpl-f60412012b80f907b1f297eeff6c860a prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=47 accepted=80/137 ttft=10.500 decode=6.454 elapsed=16.954 queued=2.694 tok/s=19.7 +info: prompt cache: 7605 tokens, resumed at 0, kept 1; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 305 + store 0 == total 305 +[tensorfold] done chatcmpl-75531289cf2ee7531381b4a83d65d4bd prompt=7605 cached=0 thinking=False effort=none tokens=64 sha=458576672307 finish=length rounds=29 accepted=34/46 ttft=7.877 decode=2.081 elapsed=9.958 queued=0.000 tok/s=30.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7605 tokens, resumed at 6144, kept 0; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 1, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 305 + store 0 == total 305 +[tensorfold] done chatcmpl-6c91b23d95faa150bb0c4677b37c1755 prompt=7605 cached=6144 thinking=False effort=none tokens=64 sha=458576672307 finish=length rounds=29 accepted=34/46 ttft=1.281 decode=2.082 elapsed=3.363 queued=0.000 tok/s=30.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7605 tokens, resumed at 6144, kept 0; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 2, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 305 + store 0 == total 305 +[tensorfold] done chatcmpl-4f5a784ce24f3df5838186e321fa4fe5 prompt=7605 cached=6144 thinking=False effort=none tokens=64 sha=458576672307 finish=length rounds=63 accepted=0/0 ttft=1.235 decode=3.531 elapsed=4.766 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7605 tokens, resumed at 6144, kept 0; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 3, misses 7, evicted 0, refused 0, failed 0) +info: prompt cache: 7605 tokens, resumed at 6144, kept 0; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 4, misses 7, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-f9046303cc57dcdaf54d14beef92d2da prompt=7605 cached=6144 thinking=False effort=none tokens=64 sha=458576672307 finish=length rounds=23 accepted=40/65 ttft=1.284 decode=3.881 elapsed=5.164 queued=0.000 tok/s=16.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 305 + store 0 == total 305 +[tensorfold] done chatcmpl-5b8594218ea6834c83fd2a9b7db6da28 prompt=7605 cached=6144 thinking=False effort=none tokens=64 sha=458576672307 finish=length rounds=23 accepted=40/66 ttft=2.559 decode=2.676 elapsed=5.235 queued=1.279 tok/s=23.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +hannels per peer +host-d:19321:19321 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:19321:19321 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:19321:19321 [0] NCCL INFO ncclCommInitRank comm 0x52bb910 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:19321:19321 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.40 (kernels 0.26, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-d:19321:19321 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:19321:19321 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c130450000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c130491000) +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1703 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1703 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bc4c2 ctsFifoLkey=0x1bc4c2 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1703 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1704 pkey=0 pd=0xf6c130053590 oooRq=0 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1705 pkey=0 pd=0xf6c130053590 +host-d:19321:19340 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1703 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19321:19321 [0] NCCL INFO Symmetric VA size=124GB +[tensorfold] shutting down: stopped diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/server-rank1.log new file mode 100644 index 0000000000..cb711f0cf1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/serve/r2vis/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T16:32:12+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native d82043de5d8b1f04 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 4 --vision +host-e:1864:1864 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:1864:1864 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:1864:1864 [0] NCCL INFO cudaDriverVersion 13030 +host-e:1864:1864 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:1864:1864 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:1864:1864 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:1864:1864 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:1864:1864 [0] NCCL INFO Count not open libibverbs.so +host-e:1864:1864 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:1864:1864 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:1864:1864 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:1864:1864 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:1864:1864 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x4268b8c0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:1864:1864 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:1864:1864 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:1864:1864 [0] NCCL INFO Initialized NET plugin IB +host-e:1864:1864 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:1864:1864 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:1864:1864 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:1864:1864 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:1864:1864 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:1864:1864 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:1864:1864 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:1864:1864 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:1864:1864 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:1864:1864 [0] NCCL INFO Using network IB +host-e:1864:1864 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:1864:1864 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x3fa59a70 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:1864:1864 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:1864:1864 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1864:1864 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1864:1864 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1864:1864 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:1864:1864 [0] NCCL INFO Rank 1: 1 Net devices +host-e:1864:1864 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:1864:1864 [0] NCCL INFO comm 0x3fa59a70 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:1864:1864 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:1864:1864 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:1864:1864 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:1864:1864 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:1864:1864 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:1864:1864 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:1864:1876 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-e:1864:1876 [0] NCCL INFO proxy listening socket at <34209> +host-e:1864:1877 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-e:1864:1878 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-e:1864:1864 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1864:1864 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1864:1864 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1864:1864 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1864:1864 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1864:1864 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1864:1864 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1864:1864 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1e504010000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1e504054000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1e50409c000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1e5040e0000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1e504128000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1e50416c000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1e5041b4000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1e5041f8000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1435 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1435 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1435 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1436 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1437 pkey=0 pd=0xf1e504053eb0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1438 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1438 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16dc8d ctsFifoLkey=0x16dc8d +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1438 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1439 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1440 pkey=0 pd=0xf1e504053eb0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1441 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1441 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x13b26d ctsFifoLkey=0x13b26d +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1441 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1442 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1442 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17eba8 ctsFifoLkey=0x17eba8 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1442 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1435 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1438 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1441 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1443 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1444 pkey=0 pd=0xf1e504053eb0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1442 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1445 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1446 pkey=0 pd=0xf1e504053eb0 +host-e:1864:1864 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:1864:1864 [0] NCCL INFO Connected all trees +host-e:1864:1864 [0] NCCL INFO Connected binomial trees +host-e:1864:1864 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:1864:1864 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:1864:1864 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:1864:1864 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:1864:1864 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:1864:1864 [0] NCCL INFO ncclCommInitRank comm 0x3fa59a70 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:1864:1864 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.40 (kernels 0.24, alloc 0.03, bootstrap 0.03, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-e:1864:1864 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:1864:1864 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1e504445000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1e504486000) +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1447 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1447 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16ce8b ctsFifoLkey=0x16ce8b +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1447 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1448 pkey=0 pd=0xf1e504053eb0 oooRq=0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1449 pkey=0 pd=0xf1e504053eb0 +host-e:1864:1876 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1447 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1864:1864 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/cache.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/cache.json new file mode 100644 index 0000000000..5171edb551 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.836628624005243, + "decode_tps": 32.86610714692435, + "elapsed_s": 25.29318165301811, + "chunk_offsets_s": [ + 24.836628624005243, + 24.921763081016252, + 24.995798839023337, + 25.074545328010572, + 25.15589208601159, + 25.231220315006794, + 25.293025846011005 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 77.03735899121966, + "p95": 85.13445701100864, + "max": 85.13445701100864 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.834597531997133, + "decode_tps": 32.94674067229198, + "elapsed_s": 25.289994538994506, + "chunk_offsets_s": [ + 24.834597531997133, + 24.919076612975914, + 24.99276055698283, + 25.07017247399199, + 25.15209208597662, + 25.22734265899635, + 25.289877771981992 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.331245014444, + "p95": 84.47908097878098, + "max": 84.47908097878098 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.844571629015263, + "decode_tps": 32.6906828180931, + "elapsed_s": 25.303610107017448, + "chunk_offsets_s": [ + 24.844571629015263, + 24.930666693020612, + 25.00539271600428, + 25.08256388900918, + 25.164684212009888, + 25.240893984999275, + 25.303417964023538 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.69047299714293, + "p95": 86.09506400534883, + "max": 86.09506400534883 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.303610107017448, + "end_to_end_tps": 0.6323208400829224, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/client.log new file mode 100644 index 0000000000..83646fbc76 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 01:46:43 +{"label": "r3k2a", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 35.054860702227145, "decode_tps_all": [39.31, 29.85, 35.05], "ttft_s_median": 0.14643806999083608} +{"label": "r3k2a", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.49385395032456, "decode_tps_all": [29.28, 28.38, 28.49], "ttft_s_median": 0.21455343699199148} +{"label": "r3k2a", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.311336875276915, "decode_tps_all": [56.31, 56.31, 56.27], "ttft_s_median": 0.1460077349911444} +{"label": "r3k2a", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.711862618108036, "decode_tps_all": [30.72, 30.57, 30.71], "ttft_s_median": 0.21438526597921737} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.836628624005243, + "decode_tps": 32.86610714692435, + "elapsed_s": 25.29318165301811, + "chunk_offsets_s": [ + 24.836628624005243, + 24.921763081016252, + 24.995798839023337, + 25.074545328010572, + 25.15589208601159, + 25.231220315006794, + 25.293025846011005 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 77.03735899121966, + "p95": 85.13445701100864, + "max": 85.13445701100864 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.834597531997133, + "decode_tps": 32.94674067229198, + "elapsed_s": 25.289994538994506, + "chunk_offsets_s": [ + 24.834597531997133, + 24.919076612975914, + 24.99276055698283, + 25.07017247399199, + 25.15209208597662, + 25.22734265899635, + 25.289877771981992 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.331245014444, + "p95": 84.47908097878098, + "max": 84.47908097878098 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.844571629015263, + "decode_tps": 32.6906828180931, + "elapsed_s": 25.303610107017448, + "chunk_offsets_s": [ + 24.844571629015263, + 24.930666693020612, + 25.00539271600428, + 25.08256388900918, + 25.164684212009888, + 25.240893984999275, + 25.303417964023538 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.69047299714293, + "p95": 86.09506400534883, + "max": 86.09506400534883 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.303610107017448, + "end_to_end_tps": 0.6323208400829224, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.8969978469831403, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.4453739980235696, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.4433110939862672, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.4501233930059243, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.288372764975065, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.2662296399939805, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.285835714981658, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.42851193799288, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.408237842988456, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.46143858000869, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 24.8333150499966, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 24.88326642100583, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.829311509995023, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.25249985200935, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.15903932900983, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 50.19711259898031, "prompt": 65536, "cached": 0} +=== kvp-client r3k2a rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/cold.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/cold.json new file mode 100644 index 0000000000..4fd9ec0e42 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.8969978469831403, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.4453739980235696, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.4433110939862672, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.4501233930059243, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.288372764975065, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.2662296399939805, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.285835714981658, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.42851193799288, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.408237842988456, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.46143858000869, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 24.8333150499966, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 24.88326642100583, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.829311509995023, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.25249985200935, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.15903932900983, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 50.19711259898031, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/decode.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/decode.json new file mode 100644 index 0000000000..61da137fd4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3k2a", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 35.054860702227145, + "decode_tps_all": [ + 39.31, + 29.85, + 35.05 + ], + "ttft_s_median": 0.14643806999083608, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3k2a", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.49385395032456, + "decode_tps_all": [ + 29.28, + 28.38, + 28.49 + ], + "ttft_s_median": 0.21455343699199148, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3k2a", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.311336875276915, + "decode_tps_all": [ + 56.31, + 56.31, + 56.27 + ], + "ttft_s_median": 0.1460077349911444, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3k2a", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.711862618108036, + "decode_tps_all": [ + 30.72, + 30.57, + 30.71 + ], + "ttft_s_median": 0.21438526597921737, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/server-rank0.log new file mode 100644 index 0000000000..b1e303b1ee --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/server-rank0.log @@ -0,0 +1,199 @@ +[kvp-serve] 2026-10-22T16:41:34+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-k2 d82043de5d8b1f04 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:19745:19745 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:19745:19745 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:19745:19745 [0] NCCL INFO cudaDriverVersion 13030 +host-d:19745:19745 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:19745:19745 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:19745:19745 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:19745:19745 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:19745:19745 [0] NCCL INFO Count not open libibverbs.so +host-d:19745:19745 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:19745:19745 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:19745:19745 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:19745:19745 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:19745:19745 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x1061cb10 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:19745:19745 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:19745:19745 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:19745:19745 [0] NCCL INFO Initialized NET plugin IB +host-d:19745:19745 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:19745:19745 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:19745:19745 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:19745:19745 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:19745:19745 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:19745:19745 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:19745:19745 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:19745:19745 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:19745:19745 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:19745:19745 [0] NCCL INFO Using network IB +host-d:19745:19745 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:19745:19745 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0xd9eaae0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:19745:19745 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:19745:19745 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:19745:19745 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:19745:19745 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:19745:19745 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:19745:19745 [0] NCCL INFO Rank 0: 1 Net devices +host-d:19745:19745 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:19745:19745 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:19745:19745 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:19745:19745 [0] NCCL INFO comm 0xd9eaae0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:19745:19745 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:19745:19745 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:19745:19745 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:19745:19745 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:19745:19745 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:19745:19745 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:19745:19745 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:19745:19745 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:19745:19745 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:19745:19745 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:19745:19764 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-d:19745:19764 [0] NCCL INFO proxy listening socket at <56679> +host-d:19745:19765 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-d:19745:19766 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-d:19745:19745 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:19745:19745 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:19745:19745 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:19745:19745 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:19745:19745 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:19745:19745 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:19745:19745 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:19745:19745 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6796c010000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6796c054000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6796c09c000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1707 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1707 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bc4c2 ctsFifoLkey=0x1bc4c2 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1707 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1708 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1708 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ab2af ctsFifoLkey=0x1ab2af +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1708 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6796c14e000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6796c196000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6796c1da000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6796c222000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6796c266000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1707 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1709 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1710 pkey=0 pd=0xe6796c053590 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1711 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1711 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b8986 ctsFifoLkey=0x1b8986 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1711 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1712 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1713 pkey=0 pd=0xe6796c053590 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1714 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1714 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b6764 ctsFifoLkey=0x1b6764 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1714 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1715 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1716 pkey=0 pd=0xe6796c053590 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1717 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1718 pkey=0 pd=0xe6796c053590 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1708 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1711 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1714 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19745 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:19745:19745 [0] NCCL INFO Connected all trees +host-d:19745:19745 [0] NCCL INFO Connected binomial trees +host-d:19745:19745 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:19745:19745 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:19745:19745 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:19745:19745 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.5 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 307.9s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-bb2239fa6546b6cec0507560a4f1ba50 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.145 decode=6.482 elapsed=6.627 queued=0.000 tok/s=39.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-0ce5ce6851f13fc959e2806982a18a0d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.145 decode=6.488 elapsed=6.633 queued=0.000 tok/s=39.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-52a08a6ac0076805da8f86465a07334a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.145 decode=8.542 elapsed=8.686 queued=0.000 tok/s=29.9 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-25514eacc66e0bb4942e438a714d07ed prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.145 decode=7.274 elapsed=7.420 queued=0.000 tok/s=35.1 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-18cf7ededd9d9558d38aa2eaabbd279d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.213 decode=8.684 elapsed=8.897 queued=0.000 tok/s=29.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b6a4132c4c41a09757fb24e86dc085d2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.215 decode=8.708 elapsed=8.923 queued=0.000 tok/s=29.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-89a8af9b51162d68ab9d27c00593f167 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.212 decode=8.984 elapsed=9.196 queued=0.000 tok/s=28.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f8479c538b7edb81da13689885498a8e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.213 decode=8.950 elapsed=9.163 queued=0.000 tok/s=28.5 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-7b94a31e56ddc97537277c1ad980e520 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.530 elapsed=4.674 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-940dc2b072d743b2886124d29bd0da2c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.529 elapsed=4.673 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-fc31141af1b67df0bbfa551e430e5c1c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.146 decode=4.529 elapsed=4.674 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-539f6dd94fe9f70fc26a0b68d026acbd prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.532 elapsed=4.677 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2efdc32d7b16e57fbef5a0157a9db833 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.212 decode=8.311 elapsed=8.523 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f80f72df0827775205b39d9d38c43e77 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.301 elapsed=8.514 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-22eba96bb47f066397972044a8d4c70a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.341 elapsed=8.554 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7dd5b10613b080ba62a2643a0b40e662 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.303 elapsed=8.517 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-58dc643998e9fda7350ca52336cd27a4 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.821 decode=0.456 elapsed=25.277 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-36be52828dcd86913d55c32baa063b12 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.832 decode=0.455 elapsed=25.287 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-3a97f59307e68f47550ae09d1b8d28db prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.842 decode=0.459 elapsed=25.301 queued=0.000 tok/s=32.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0742befa39c8ec60f2c735d7aab64ff1 prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.874 decode=0.063 elapsed=0.937 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0c0c6730abad76bf30a36826ff010a34 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.444 decode=0.065 elapsed=1.508 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-680117c8101067f254d5712d933577a0 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.442 decode=0.064 elapsed=1.506 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1ba7b80ba01224e4b71436815cd04469 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.449 decode=0.064 elapsed=1.512 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1f5d3e22050573052f10d87a39a318e9 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.287 decode=0.065 elapsed=6.352 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ea41aa6150be953f350edc0c932ae0b5 prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.265 decode=0.065 elapsed=6.330 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b60e9c11d9d18d743610631d2580565c prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.284 decode=0.065 elapsed=6.349 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e1318720c47e5ed98f89260cd032889f prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.427 decode=0.066 elapsed=12.493 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-efbd25162d86d6693e887e8327c1e281 prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.407 decode=0.065 elapsed=12.472 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c13d539a51d2a13fe702f9705b4e1c47 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.460 decode=0.065 elapsed=12.525 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-31ca7013dca41bf094d2f7eefb04a346 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.832 decode=0.066 elapsed=24.897 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ddf7c9dce9f10d785e8615dbdff8448e prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=24.882 decode=0.066 elapsed=24.948 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5e250c108ede4fc3777ab4e211a7e66c prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.828 decode=0.066 elapsed=24.893 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6b3ee80ceeee8131b8dd3f1284f98783 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.251 decode=0.065 elapsed=50.316 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6bf26f76593f7c5765e881219542b632 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.157 decode=0.065 elapsed=50.222 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1ca695d56131e7228160a9ecdd7d7730 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=50.195 decode=0.066 elapsed=50.261 queued=0.000 tok/s=15.1 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped + channels per peer +host-d:19745:19745 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:19745:19745 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:19745:19745 [0] NCCL INFO ncclCommInitRank comm 0xd9eaae0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:19745:19745 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.40 (kernels 0.25, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-d:19745:19745 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:19745:19745 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6796c450000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6796c491000) +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1719 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1719 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9c99 ctsFifoLkey=0x1b9c99 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1719 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1720 pkey=0 pd=0xe6796c053590 oooRq=0 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1721 pkey=0 pd=0xe6796c053590 +host-d:19745:19764 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1719 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:19745:19745 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/server-rank1.log new file mode 100644 index 0000000000..0cb7e60587 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2a/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T16:41:34+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-k2 d82043de5d8b1f04 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:1906:1906 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:1906:1906 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:1906:1906 [0] NCCL INFO cudaDriverVersion 13030 +host-e:1906:1906 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:1906:1906 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:1906:1906 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:1906:1906 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:1906:1906 [0] NCCL INFO Count not open libibverbs.so +host-e:1906:1906 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:1906:1906 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:1906:1906 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:1906:1906 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:1906:1906 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x11e8b600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:1906:1906 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:1906:1906 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:1906:1906 [0] NCCL INFO Initialized NET plugin IB +host-e:1906:1906 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:1906:1906 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:1906:1906 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:1906:1906 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:1906:1906 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:1906:1906 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:1906:1906 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:1906:1906 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:1906:1906 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:1906:1906 [0] NCCL INFO Using network IB +host-e:1906:1906 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:1906:1906 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0xf2597a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:1906:1906 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:1906:1906 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1906:1906 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1906:1906 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1906:1906 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:1906:1906 [0] NCCL INFO Rank 1: 1 Net devices +host-e:1906:1906 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:1906:1906 [0] NCCL INFO comm 0xf2597a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:1906:1906 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:1906:1906 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:1906:1906 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:1906:1906 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:1906:1906 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:1906:1906 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:1906:1918 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-e:1906:1918 [0] NCCL INFO proxy listening socket at <41621> +host-e:1906:1919 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-e:1906:1906 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1906:1906 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1906:1906 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1906:1906 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1906:1920 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-e:1906:1906 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1906:1906 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1906:1906 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1906:1906 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xffdf3c010000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xffdf3c054000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xffdf3c09c000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xffdf3c0e0000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xffdf3c128000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xffdf3c16c000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xffdf3c1b4000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xffdf3c1f8000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1451 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1452 pkey=0 pd=0xffdf3c053eb0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1453 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1453 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1706c5 ctsFifoLkey=0x1706c5 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1453 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1454 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1455 pkey=0 pd=0xffdf3c053eb0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1456 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1456 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x14f1b0 ctsFifoLkey=0x14f1b0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1456 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1457 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1457 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16dd9e ctsFifoLkey=0x16dd9e +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1457 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1458 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1458 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1458 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1453 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1456 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1459 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1460 pkey=0 pd=0xffdf3c053eb0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1457 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1461 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1462 pkey=0 pd=0xffdf3c053eb0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1458 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1906 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:1906:1906 [0] NCCL INFO Connected all trees +host-e:1906:1906 [0] NCCL INFO Connected binomial trees +host-e:1906:1906 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:1906:1906 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:1906:1906 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:1906:1906 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:1906:1906 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:1906:1906 [0] NCCL INFO ncclCommInitRank comm 0xf2597a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:1906:1906 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.40 (kernels 0.24, alloc 0.04, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-e:1906:1906 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shared +einfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +host-e:1906:1906 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xffdf3c448000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xffdf3c489000) +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1463 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1463 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x14a461 ctsFifoLkey=0x14a461 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1463 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1464 pkey=0 pd=0xffdf3c053eb0 oooRq=0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1465 pkey=0 pd=0xffdf3c053eb0 +host-e:1906:1918 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1463 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1906:1906 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/cache.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/cache.json new file mode 100644 index 0000000000..ca5b85a1c5 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.833027503016638, + "decode_tps": 32.63968767446163, + "elapsed_s": 25.29269628901966, + "chunk_offsets_s": [ + 24.833027503016638, + 24.919280739006354, + 24.993480756005738, + 25.071124708018033, + 25.153146892000223, + 25.229633031995036, + 25.29259072401328 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 77.06504600355402, + "p95": 86.2532359897159, + "max": 86.2532359897159 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.821727894013748, + "decode_tps": 32.68560260168039, + "elapsed_s": 25.280742375995032, + "chunk_offsets_s": [ + 24.821727894013748, + 24.906849285005592, + 24.981507010001224, + 25.060088772996096, + 25.142376981006237, + 25.218312181008514, + 25.2806455460086 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 77.25848149857484, + "p95": 85.12139099184424, + "max": 85.12139099184424 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.82092211500276, + "decode_tps": 32.7038946265334, + "elapsed_s": 25.279677418002393, + "chunk_offsets_s": [ + 24.82092211500276, + 24.905286110995803, + 24.980257656017784, + 25.05746945599094, + 25.139382802008186, + 25.21573570999317, + 25.279583084018668 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.78235397906974, + "p95": 84.36399599304423, + "max": 84.36399599304423 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.279677418002393, + "end_to_end_tps": 0.6329194686877585, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/client.log new file mode 100644 index 0000000000..a68eedd558 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 02:52:17 +{"label": "r3k2all", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 34.926645251640174, "decode_tps_all": [39.22, 29.77, 34.93], "ttft_s_median": 0.14689250799710862} +{"label": "r3k2all", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.39469894076537, "decode_tps_all": [29.16, 28.33, 28.39], "ttft_s_median": 0.2160296579822898} +{"label": "r3k2all", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.164524305161, "decode_tps_all": [56.16, 56.17, 56.13], "ttft_s_median": 0.1464053130184766} +{"label": "r3k2all", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.531534405617062, "decode_tps_all": [30.48, 30.63, 30.53], "ttft_s_median": 0.21564055600902066} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.833027503016638, + "decode_tps": 32.63968767446163, + "elapsed_s": 25.29269628901966, + "chunk_offsets_s": [ + 24.833027503016638, + 24.919280739006354, + 24.993480756005738, + 25.071124708018033, + 25.153146892000223, + 25.229633031995036, + 25.29259072401328 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 77.06504600355402, + "p95": 86.2532359897159, + "max": 86.2532359897159 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.821727894013748, + "decode_tps": 32.68560260168039, + "elapsed_s": 25.280742375995032, + "chunk_offsets_s": [ + 24.821727894013748, + 24.906849285005592, + 24.981507010001224, + 25.060088772996096, + 25.142376981006237, + 25.218312181008514, + 25.2806455460086 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 77.25848149857484, + "p95": 85.12139099184424, + "max": 85.12139099184424 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.82092211500276, + "decode_tps": 32.7038946265334, + "elapsed_s": 25.279677418002393, + "chunk_offsets_s": [ + 24.82092211500276, + 24.905286110995803, + 24.980257656017784, + 25.05746945599094, + 25.139382802008186, + 25.21573570999317, + 25.279583084018668 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.78235397906974, + "p95": 84.36399599304423, + "max": 84.36399599304423 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.279677418002393, + "end_to_end_tps": 0.6329194686877585, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.8917491949978285, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.4468637390236836, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.4465952239988837, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.4436066659982316, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.265134815999772, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.257070154999383, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.2725643159938045, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.404174181981944, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.409659561002627, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.45133188800537, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 24.78911063299165, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 24.836596424982417, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.782353354996303, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.268406660994515, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.17192946700379, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 50.18943215801846, "prompt": 65536, "cached": 0} +=== kvp-client r3k2all rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/cold.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/cold.json new file mode 100644 index 0000000000..c277988718 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.8917491949978285, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.4468637390236836, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.4465952239988837, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.4436066659982316, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.265134815999772, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.257070154999383, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.2725643159938045, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.404174181981944, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.409659561002627, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.45133188800537, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 24.78911063299165, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 24.836596424982417, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.782353354996303, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.268406660994515, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.17192946700379, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 50.18943215801846, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/decode.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/decode.json new file mode 100644 index 0000000000..3b17455aba --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3k2all", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 34.926645251640174, + "decode_tps_all": [ + 39.22, + 29.77, + 34.93 + ], + "ttft_s_median": 0.14689250799710862, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3k2all", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.39469894076537, + "decode_tps_all": [ + 29.16, + 28.33, + 28.39 + ], + "ttft_s_median": 0.2160296579822898, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3k2all", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.164524305161, + "decode_tps_all": [ + 56.16, + 56.17, + 56.13 + ], + "ttft_s_median": 0.1464053130184766, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3k2all", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.531534405617062, + "decode_tps_all": [ + 30.48, + 30.63, + 30.53 + ], + "ttft_s_median": 0.21564055600902066, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/server-rank0.log new file mode 100644 index 0000000000..1ea7e3940b --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/server-rank0.log @@ -0,0 +1,234 @@ +[kvp-serve] 2026-10-22T17:47:08+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-k2 d82043de5d8b1f04 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:21519:21519 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:21519:21519 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:21519:21519 [0] NCCL INFO cudaDriverVersion 13030 +host-d:21519:21519 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:21519:21519 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:21519:21519 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:21519:21519 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:21519:21519 [0] NCCL INFO Count not open libibverbs.so +host-d:21519:21519 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:21519:21519 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:21519:21519 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:21519:21519 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:21519:21519 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x71ba9d0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:21519:21519 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:21519:21519 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:21519:21519 [0] NCCL INFO Initialized NET plugin IB +host-d:21519:21519 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:21519:21519 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:21519:21519 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:21519:21519 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:21519:21519 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:21519:21519 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:21519:21519 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:21519:21519 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:21519:21519 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:21519:21519 [0] NCCL INFO Using network IB +host-d:21519:21519 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:21519:21519 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x4588970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:21519:21519 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:21519:21519 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:21519:21519 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:21519:21519 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:21519:21519 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:21519:21519 [0] NCCL INFO Rank 0: 1 Net devices +host-d:21519:21519 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:21519:21519 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:21519:21519 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:21519:21519 [0] NCCL INFO comm 0x4588970 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:21519:21519 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:21519:21519 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:21519:21519 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:21519:21519 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:21519:21519 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:21519:21519 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:21519:21519 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:21519:21519 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:21519:21519 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:21519:21519 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:21519:21539 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-d:21519:21538 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-d:21519:21538 [0] NCCL INFO proxy listening socket at <44989> +host-d:21519:21519 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21519:21519 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21519:21519 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21519:21519 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21519:21540 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-d:21519:21519 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21519:21519 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21519:21519 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21519:21519 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf77b78010000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf77b78054000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf77b7809c000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf77b780e0000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf77b78128000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1771 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1771 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bc4c2 ctsFifoLkey=0x1bc4c2 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1771 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf77b781a8000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf77b781f0000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf77b78234000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1772 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1773 pkey=0 pd=0xf77b78053c70 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1771 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1774 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1774 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1abdbb ctsFifoLkey=0x1abdbb +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1774 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1775 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1776 pkey=0 pd=0xf77b78053c70 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1774 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1777 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1777 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b1311 ctsFifoLkey=0x1b1311 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1777 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1778 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1779 pkey=0 pd=0xf77b78053c70 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1780 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1780 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bbdba ctsFifoLkey=0x1bbdba +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1780 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1777 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1781 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1782 pkey=0 pd=0xf77b78053c70 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1780 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21519 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:21519:21519 [0] NCCL INFO Connected all trees +host-d:21519:21519 [0] NCCL INFO Connected binomial trees +host-d:21519:21519 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:21519:21519 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:21519:21519 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:21519:21519 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, the engine's own at a 131072-token window (1.77 GiB, in the model's), 3 more taking 768 pages of 512 rows (4.78 GiB) as they need them, 180 MiB fixed each, of 7.4 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 307.6s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-c421c98948851fa7e8c59c31541f3b5a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.524 elapsed=6.670 queued=0.000 tok/s=39.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-6cddd1cde27bec5f7bd27541d518da59 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.502 elapsed=6.648 queued=0.000 tok/s=39.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-327d852580a57712704818d7a8f39282 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.147 decode=8.566 elapsed=8.714 queued=0.000 tok/s=29.8 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-03d863dc60f2224ffb3d08edb9f37dc0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.146 decode=7.301 elapsed=7.447 queued=0.000 tok/s=34.9 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-69d70c9d9a271ea09e5eb66a05813044 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.214 decode=8.745 elapsed=8.959 queued=0.000 tok/s=29.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-8a148184acde33810e199d4fcc96de48 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.215 decode=8.747 elapsed=8.961 queued=0.000 tok/s=29.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-31d9195ebb39e2b4e9e8951f75d599df prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.216 decode=9.001 elapsed=9.217 queued=0.000 tok/s=28.3 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-2bba4f13dcb9ed0c6a62e46cc8794185 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.214 decode=8.981 elapsed=9.195 queued=0.000 tok/s=28.4 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-435a34c015ed33f9df0705f836a6049f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.146 decode=4.539 elapsed=4.684 queued=0.000 tok/s=56.2 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-906a0739e3e70c51d09df7146ef1efd4 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.540 elapsed=4.686 queued=0.000 tok/s=56.2 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-1d5cfa7d800d1ecaca27126521fc0762 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.540 elapsed=4.685 queued=0.000 tok/s=56.2 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-cae66b54808a7bfa1586fcd23ac1c5c1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.146 decode=4.543 elapsed=4.689 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-fa4559eb485711d29d48c1c0bbc8534c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.342 elapsed=8.555 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-cb573ef2946cd1289e90a9298f2a0f17 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.366 elapsed=8.581 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-78fe5f2fa29fdf7bd6fbe2d24a4db76b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.215 decode=8.326 elapsed=8.541 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-9e96c1b95837336e72afc55029b5df10 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.216 decode=8.352 elapsed=8.568 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-abab4c0c8dac394628d546fea514f1ef prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.817 decode=0.460 elapsed=25.277 queued=0.000 tok/s=32.6 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-32e4f4cec91d4882997027acfd8f3bba prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.819 decode=0.459 elapsed=25.278 queued=0.000 tok/s=32.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done cmpl-95f036679b5138b84d86bb53b7fcf9a4 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.818 decode=0.459 elapsed=25.277 queued=0.000 tok/s=32.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-0a50c57de3f03347b52b6a660b837430 prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.878 decode=0.065 elapsed=0.943 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-0e9d096a665d6214049c585239ccb1b5 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.446 decode=0.063 elapsed=1.509 queued=0.000 tok/s=15.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-0ca529a5083e549c1310bc9de95713dc prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.445 decode=0.064 elapsed=1.508 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-82eb0caf50b3854fff84d0df45395865 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.443 decode=0.065 elapsed=1.508 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-708e55b9272d4804cef4dd191347e1a1 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.264 decode=0.064 elapsed=6.328 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-2849ee6a0a06e2e2c53c5d6770b273bd prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.256 decode=0.065 elapsed=6.321 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-bb647a4488eecce49bf27c9bfada6bfe prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.271 decode=0.066 elapsed=6.337 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-0bc96d8ec45778323fa265253e72d8fe prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.403 decode=0.066 elapsed=12.469 queued=0.000 tok/s=15.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-25cef41740ceb086d1045e736ed942da prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.408 decode=0.065 elapsed=12.473 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-13c7872aa16bfdf3d931760888d3a8fe prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.450 decode=0.065 elapsed=12.514 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-76d4c33d5c198ac44fb4eea483338504 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.788 decode=0.066 elapsed=24.854 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-ad4c7c133f9ee33b33f573562a354a0c prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=24.835 decode=0.066 elapsed=24.900 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-212111e37536bccfd6d5ce7217bfc762 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.781 decode=0.065 elapsed=24.846 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-4f51f652c0753f99221836628ebb4c79 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.267 decode=0.065 elapsed=50.332 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-4ca329680e888174559547332b6b9601 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.170 decode=0.065 elapsed=50.235 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 768 + store 0 == total 768 +[tensorfold] done chatcmpl-0c59c1ff3c07eba013529684d6741665 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=50.187 decode=0.065 elapsed=50.253 queued=0.000 tok/s=15.3 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +channels per peer +host-d:21519:21519 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:21519:21519 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:21519:21519 [0] NCCL INFO ncclCommInitRank comm 0x4588970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:21519:21519 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.41 (kernels 0.25, alloc 0.03, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-d:21519:21519 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:21519:21519 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf77b78442000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf77b78483000) +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1783 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1783 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5a59 ctsFifoLkey=0x1b5a59 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1783 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1784 pkey=0 pd=0xf77b78053c70 oooRq=0 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1785 pkey=0 pd=0xf77b78053c70 +host-d:21519:21538 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1783 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21519:21519 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/server-rank1.log new file mode 100644 index 0000000000..f5d697c48f --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2all/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T17:47:08+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-k2 d82043de5d8b1f04 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:2071:2071 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2071:2071 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2071:2071 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2071:2071 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2071:2071 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2071:2071 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2071:2071 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2071:2071 [0] NCCL INFO Count not open libibverbs.so +host-e:2071:2071 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2071:2071 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2071:2071 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2071:2071 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2071:2071 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0xd98d600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2071:2071 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2071:2071 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2071:2071 [0] NCCL INFO Initialized NET plugin IB +host-e:2071:2071 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2071:2071 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2071:2071 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2071:2071 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2071:2071 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2071:2071 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2071:2071 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2071:2071 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2071:2071 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2071:2071 [0] NCCL INFO Using network IB +host-e:2071:2071 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2071:2071 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0xad5b7a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2071:2071 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2071:2071 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2071:2071 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2071:2071 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2071:2071 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2071:2071 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2071:2071 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2071:2071 [0] NCCL INFO comm 0xad5b7a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2071:2071 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2071:2071 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2071:2071 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2071:2071 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2071:2071 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2071:2071 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2071:2083 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-e:2071:2083 [0] NCCL INFO proxy listening socket at <33481> +host-e:2071:2084 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-e:2071:2071 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2071:2085 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-e:2071:2071 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2071:2071 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2071:2071 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2071:2071 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2071:2071 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2071:2071 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2071:2071 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedb8ec010000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedb8ec054000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedb8ec09c000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1515 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1515 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1515 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1516 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1517 pkey=0 pd=0xedb8ec053590 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1518 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1518 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1715d0 ctsFifoLkey=0x1715d0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1518 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedb8ec19e000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1515 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedb8ec1e6000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1519 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1520 pkey=0 pd=0xedb8ec053590 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1521 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1521 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x172dea ctsFifoLkey=0x172dea +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1521 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedb8ec2a7000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1518 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedb8ec2ef000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedb8ec330000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1522 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1523 pkey=0 pd=0xedb8ec053590 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1521 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1524 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1524 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1733f0 ctsFifoLkey=0x1733f0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1524 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1525 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1526 pkey=0 pd=0xedb8ec053590 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1524 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2071 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2071:2071 [0] NCCL INFO Connected all trees +host-e:2071:2071 [0] NCCL INFO Connected binomial trees +host-e:2071:2071 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2071:2071 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2071:2071 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2071:2071 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2071:2071 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2071:2071 [0] NCCL INFO ncclCommInitRank comm 0xad5b7a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2071:2071 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.41 (kernels 0.24, alloc 0.05, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-e:2071:2071 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shared +edinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +host-e:2071:2071 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedb8ec440000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedb8ec481000) +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1527 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1527 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17ad6b ctsFifoLkey=0x17ad6b +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1527 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1528 pkey=0 pd=0xedb8ec053590 oooRq=0 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1529 pkey=0 pd=0xedb8ec053590 +host-e:2071:2083 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1527 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2071:2071 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/cache.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/cache.json new file mode 100644 index 0000000000..21a352ffc1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.800064454000676, + "decode_tps": 32.90993740828201, + "elapsed_s": 25.256028912990587, + "chunk_offsets_s": [ + 24.800064454000676, + 24.885758591990452, + 24.96019257599255, + 25.036830857978202, + 25.117602158978116, + 25.192793319991324, + 25.25585383499856 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 75.91472149943002, + "p95": 85.69413798977621, + "max": 85.69413798977621 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.794065132999094, + "decode_tps": 32.92470142368417, + "elapsed_s": 25.249796823016368, + "chunk_offsets_s": [ + 24.794065132999094, + 24.879201328003546, + 24.953192520013545, + 25.029991595016327, + 25.111523704021238, + 25.187681187002454, + 25.24965013001929 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.4782789919991, + "p95": 85.13619500445202, + "max": 85.13619500445202 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.768929783982458, + "decode_tps": 32.699700767784655, + "elapsed_s": 25.227851999981795, + "chunk_offsets_s": [ + 24.768929783982458, + 24.855242224002723, + 24.92947078999714, + 25.00738699399517, + 25.090024081990123, + 25.165667172987014, + 25.22764957798063 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.77964749746025, + "p95": 86.31244002026506, + "max": 86.31244002026506 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.227851999981795, + "end_to_end_tps": 0.6342196711797559, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/client.log new file mode 100644 index 0000000000..95a5108e9e --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 02:19:31 +{"label": "r3k2b", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 34.9645884893665, "decode_tps_all": [39.17, 29.84, 34.96], "ttft_s_median": 0.1451843300019391} +{"label": "r3k2b", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.452209605155677, "decode_tps_all": [29.2, 28.35, 28.45], "ttft_s_median": 0.21349159599049017} +{"label": "r3k2b", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.20426221133491, "decode_tps_all": [56.25, 56.2, 56.19], "ttft_s_median": 0.14520240397541784} +{"label": "r3k2b", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.609017090705454, "decode_tps_all": [30.61, 30.6, 30.67], "ttft_s_median": 0.21447056098259054} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.800064454000676, + "decode_tps": 32.90993740828201, + "elapsed_s": 25.256028912990587, + "chunk_offsets_s": [ + 24.800064454000676, + 24.885758591990452, + 24.96019257599255, + 25.036830857978202, + 25.117602158978116, + 25.192793319991324, + 25.25585383499856 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 75.91472149943002, + "p95": 85.69413798977621, + "max": 85.69413798977621 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.794065132999094, + "decode_tps": 32.92470142368417, + "elapsed_s": 25.249796823016368, + "chunk_offsets_s": [ + 24.794065132999094, + 24.879201328003546, + 24.953192520013545, + 25.029991595016327, + 25.111523704021238, + 25.187681187002454, + 25.24965013001929 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.4782789919991, + "p95": 85.13619500445202, + "max": 85.13619500445202 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.768929783982458, + "decode_tps": 32.699700767784655, + "elapsed_s": 25.227851999981795, + "chunk_offsets_s": [ + 24.768929783982458, + 24.855242224002723, + 24.92947078999714, + 25.00738699399517, + 25.090024081990123, + 25.165667172987014, + 25.22764957798063 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.77964749746025, + "p95": 86.31244002026506, + "max": 86.31244002026506 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.227851999981795, + "end_to_end_tps": 0.6342196711797559, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.8883502190001309, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.444218187010847, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.4407536910148337, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.4407327829976566, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.261085231002653, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.263889813009882, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.2697445880039595, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.394962584978202, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.404937629005872, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.436771075008437, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 24.8036902580061, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 24.832292898994638, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.8092147760035, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.164161846012576, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.07617649898748, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 50.13409038502141, "prompt": 65536, "cached": 0} +=== kvp-client r3k2b rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/cold.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/cold.json new file mode 100644 index 0000000000..19220796eb --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.8883502190001309, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.444218187010847, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.4407536910148337, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.4407327829976566, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.261085231002653, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.263889813009882, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.2697445880039595, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.394962584978202, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.404937629005872, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.436771075008437, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 24.8036902580061, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 24.832292898994638, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.8092147760035, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.164161846012576, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.07617649898748, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 50.13409038502141, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/decode.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/decode.json new file mode 100644 index 0000000000..a8f5800f39 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3k2b", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 34.9645884893665, + "decode_tps_all": [ + 39.17, + 29.84, + 34.96 + ], + "ttft_s_median": 0.1451843300019391, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3k2b", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.452209605155677, + "decode_tps_all": [ + 29.2, + 28.35, + 28.45 + ], + "ttft_s_median": 0.21349159599049017, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3k2b", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.20426221133491, + "decode_tps_all": [ + 56.25, + 56.2, + 56.19 + ], + "ttft_s_median": 0.14520240397541784, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3k2b", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.609017090705454, + "decode_tps_all": [ + 30.61, + 30.6, + 30.67 + ], + "ttft_s_median": 0.21447056098259054, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/server-rank0.log new file mode 100644 index 0000000000..3465f16bd8 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/server-rank0.log @@ -0,0 +1,199 @@ +[kvp-serve] 2026-10-22T17:14:23+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-k2 d82043de5d8b1f04 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:20633:20633 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:20633:20633 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:20633:20633 [0] NCCL INFO cudaDriverVersion 13030 +host-d:20633:20633 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:20633:20633 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:20633:20633 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:20633:20633 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:20633:20633 [0] NCCL INFO Count not open libibverbs.so +host-d:20633:20633 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:20633:20633 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:20633:20633 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:20633:20633 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:20633:20633 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x23acf7e0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:20633:20633 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:20633:20633 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:20633:20633 [0] NCCL INFO Initialized NET plugin IB +host-d:20633:20633 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:20633:20633 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:20633:20633 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:20633:20633 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:20633:20633 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:20633:20633 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:20633:20633 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:20633:20633 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:20633:20633 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:20633:20633 [0] NCCL INFO Using network IB +host-d:20633:20633 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:20633:20633 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x20e9d7a0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:20633:20633 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:20633:20633 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:20633:20633 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:20633:20633 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:20633:20633 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:20633:20633 [0] NCCL INFO Rank 0: 1 Net devices +host-d:20633:20633 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:20633:20633 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:20633:20633 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:20633:20633 [0] NCCL INFO comm 0x20e9d7a0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:20633:20633 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:20633:20633 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:20633:20633 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:20633:20633 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:20633:20633 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:20633:20633 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:20633:20633 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:20633:20633 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:20633:20633 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:20633:20633 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:20633:20653 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 1 +host-d:20633:20652 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-d:20633:20652 [0] NCCL INFO proxy listening socket at <42067> +host-d:20633:20633 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:20633:20633 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:20633:20633 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:20633:20633 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:20633:20654 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-d:20633:20633 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:20633:20633 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:20633:20633 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:20633:20633 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf52714010000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf52714054000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5271409c000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf527140e0000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf52714128000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1739 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1739 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bbdba ctsFifoLkey=0x1bbdba +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1739 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1740 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1740 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a1211 ctsFifoLkey=0x1a1211 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1740 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1741 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1741 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x176862 ctsFifoLkey=0x176862 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1741 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5271420c000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf52714254000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf52714298000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1739 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1742 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1743 pkey=0 pd=0xf52714053c70 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1740 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1744 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1745 pkey=0 pd=0xf52714053c70 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1746 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1746 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a4f4c ctsFifoLkey=0x1a4f4c +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1746 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1747 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1748 pkey=0 pd=0xf52714053c70 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1749 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1750 pkey=0 pd=0xf52714053c70 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1741 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1746 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20633 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:20633:20633 [0] NCCL INFO Connected all trees +host-d:20633:20633 [0] NCCL INFO Connected binomial trees +host-d:20633:20633 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:20633:20633 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:20633:20633 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:20633:20633 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.5 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 304.3s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-c366d1aa552f0b67ae06cd612861645d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.145 decode=6.489 elapsed=6.634 queued=0.000 tok/s=39.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-995a170b68ace1ee25c2aea7d981e645 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.144 decode=6.510 elapsed=6.654 queued=0.000 tok/s=39.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-defdad801dbba59faac17fb5b3c0d2a4 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.145 decode=8.547 elapsed=8.692 queued=0.000 tok/s=29.8 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-32262912afa86c3750475e54e77b1ecc prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.144 decode=7.293 elapsed=7.438 queued=0.000 tok/s=35.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2692c5ebbc700247fea0ff0245728b01 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.213 decode=8.712 elapsed=8.925 queued=0.000 tok/s=29.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5fea63e18a967952dc7c102b5d8e8180 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.214 decode=8.732 elapsed=8.946 queued=0.000 tok/s=29.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e4791ec5014748c469dc6ef257b5983d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.213 decode=8.995 elapsed=9.207 queued=0.000 tok/s=28.3 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c4376619bfc009303afc81c1d91b6bd2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.212 decode=8.963 elapsed=9.175 queued=0.000 tok/s=28.5 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-12e5a1ac90930692c31c1e61ff374c80 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.547 elapsed=4.691 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4e1ce0db1397ffcba9f535cd4c9aee14 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.533 elapsed=4.677 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-eb570b402a7e1ef49832b58c3601be2f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.537 elapsed=4.682 queued=0.000 tok/s=56.2 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4c7a5126a8ba90d40aa9f6d97c0c167f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.539 elapsed=4.683 queued=0.000 tok/s=56.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ec5fb08eddfaa443314a7dd05417fb3c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.328 elapsed=8.542 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4a868d7231ce0883abe9f22d0e708971 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.331 elapsed=8.544 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3d83d86f86a42c47a82850f2272b0aa3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.334 elapsed=8.548 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fa5fb922f644a1319fca9c28409290f3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.315 elapsed=8.528 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-01c443c22048ed71edf0c9aed2ca50a7 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.784 decode=0.456 elapsed=25.240 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4763c7b5b3dd7feb0f88ac5d272c9f37 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.791 decode=0.456 elapsed=25.247 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-641d7dac533cac7512d9c487bf6baec8 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.766 decode=0.459 elapsed=25.225 queued=0.000 tok/s=32.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-11dcd70b75f323fe2ac5cf7083ce3058 prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.873 decode=0.063 elapsed=0.936 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-caf8a67cabe85409781c8ebb2bf1771b prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.443 decode=0.063 elapsed=1.506 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3e35eaf9e98bdd5104da7b565c4c629a prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.440 decode=0.064 elapsed=1.503 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c15a960c2d39dd3597c20aca20949634 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.440 decode=0.065 elapsed=1.505 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d6829316383363b115a4eb8267472b4c prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.260 decode=0.066 elapsed=6.326 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a42364bc69e77bb7c680d77a0a8dfc16 prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.263 decode=0.064 elapsed=6.326 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c685bfe3cc81a5dee7bc26dccba7a898 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.268 decode=0.065 elapsed=6.333 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ee50d3f3c1217c211123cd66d4abaada prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.394 decode=0.065 elapsed=12.458 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b003d3dbc41bf1b2bc975e9c912599cf prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.404 decode=0.064 elapsed=12.468 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-71c72f439904859094e77455f2151814 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.436 decode=0.066 elapsed=12.501 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ec8677296047d6a0b339d350580980a0 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.802 decode=0.067 elapsed=24.869 queued=0.000 tok/s=15.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-cc648c4602a831a97e5a5e9489f86fa9 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=24.831 decode=0.065 elapsed=24.896 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-57c7beb3a0dd9cfb4d2ef717495676be prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.808 decode=0.065 elapsed=24.873 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-86b3f0b21726438f437fde920d2ceba6 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.163 decode=0.066 elapsed=50.229 queued=0.000 tok/s=15.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-dccb36498d9842f54665e62538cb9767 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.074 decode=0.064 elapsed=50.139 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4ba36d40d028b77fcbf3f159dead302d prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=50.132 decode=0.066 elapsed=50.198 queued=0.000 tok/s=15.1 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:20633:20633 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:20633:20633 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:20633:20633 [0] NCCL INFO ncclCommInitRank comm 0x20e9d7a0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:20633:20633 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.42 (kernels 0.25, alloc 0.05, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-d:20633:20633 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:20633:20633 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5271444c000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5271448d000) +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1751 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1751 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1aebe8 ctsFifoLkey=0x1aebe8 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1751 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1752 pkey=0 pd=0xf52714053c70 oooRq=0 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1753 pkey=0 pd=0xf52714053c70 +host-d:20633:20652 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1751 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20633:20633 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/server-rank1.log new file mode 100644 index 0000000000..aeb41784b0 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3k2b/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T17:14:22+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-k2 d82043de5d8b1f04 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:1988:1988 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:1988:1988 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:1988:1988 [0] NCCL INFO cudaDriverVersion 13030 +host-e:1988:1988 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:1988:1988 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:1988:1988 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:1988:1988 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:1988:1988 [0] NCCL INFO Count not open libibverbs.so +host-e:1988:1988 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:1988:1988 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:1988:1988 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:1988:1988 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:1988:1988 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x39119600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:1988:1988 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:1988:1988 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:1988:1988 [0] NCCL INFO Initialized NET plugin IB +host-e:1988:1988 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:1988:1988 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:1988:1988 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:1988:1988 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:1988:1988 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:1988:1988 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:1988:1988 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:1988:1988 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:1988:1988 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:1988:1988 [0] NCCL INFO Using network IB +host-e:1988:1988 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:1988:1988 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x364e77a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:1988:1988 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:1988:1988 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1988:1988 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1988:1988 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1988:1988 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:1988:1988 [0] NCCL INFO Rank 1: 1 Net devices +host-e:1988:1988 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:1988:1988 [0] NCCL INFO comm 0x364e77a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:1988:1988 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:1988:1988 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:1988:1988 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:1988:1988 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:1988:1988 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:1988:1988 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:1988:2001 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-e:1988:2000 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-e:1988:2000 [0] NCCL INFO proxy listening socket at <50573> +host-e:1988:2002 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-e:1988:1988 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1988:1988 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1988:1988 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1988:1988 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1988:1988 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1988:1988 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1988:1988 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1988:1988 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf37580010000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf37580054000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf3758009c000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf375800e0000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf37580128000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf3758016c000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf375801b4000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf375801f8000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1483 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1484 pkey=0 pd=0xf37580053eb0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1485 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1485 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1703b8 ctsFifoLkey=0x1703b8 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1485 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1486 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1487 pkey=0 pd=0xf37580053eb0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1488 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1488 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x14f1b0 ctsFifoLkey=0x14f1b0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1488 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1489 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1490 pkey=0 pd=0xf37580053eb0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1491 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1491 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16dc8d ctsFifoLkey=0x16dc8d +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1491 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1492 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1492 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1492 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1485 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1488 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1491 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1493 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1494 pkey=0 pd=0xf37580053eb0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1492 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:1988 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:1988:1988 [0] NCCL INFO Connected all trees +host-e:1988:1988 [0] NCCL INFO Connected binomial trees +host-e:1988:1988 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:1988:1988 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:1988:1988 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:1988:1988 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:1988:1988 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:1988:1988 [0] NCCL INFO ncclCommInitRank comm 0x364e77a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:1988:1988 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.42 (kernels 0.24, alloc 0.03, bootstrap 0.03, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-e:1988:1988 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:1988:1988 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf3758044b000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf3758048c000) +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1495 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1495 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16ce8b ctsFifoLkey=0x16ce8b +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1495 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1496 pkey=0 pd=0xf37580053eb0 oooRq=0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1497 pkey=0 pd=0xf37580053eb0 +host-e:1988:2000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1495 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1988:1988 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/cache.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/cache.json new file mode 100644 index 0000000000..e1f14b97d9 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.8607156460057, + "decode_tps": 32.8343731514751, + "elapsed_s": 25.317698785016546, + "chunk_offsets_s": [ + 24.8607156460057, + 24.946682211011648, + 25.020607040001778, + 25.09849363702233, + 25.17982723499881, + 25.255076638015453, + 25.317553970002336 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.56800001859665, + "p95": 85.9665650059469, + "max": 85.9665650059469 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.883651929994812, + "decode_tps": 32.8095833094987, + "elapsed_s": 25.3410171540163, + "chunk_offsets_s": [ + 24.883651929994812, + 24.968024358007824, + 25.041858694021357, + 25.11886785700335, + 25.200616578018526, + 25.27735730900895, + 25.34083542600274 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.87494698620867, + "p95": 84.37242801301181, + "max": 84.37242801301181 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.869684855017113, + "decode_tps": 32.85218483482, + "elapsed_s": 25.326438132004114, + "chunk_offsets_s": [ + 24.869684855017113, + 24.95434464601567, + 25.029398087994196, + 25.10609231001581, + 25.18799613902229, + 25.264427450019866, + 25.326275491999695 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.56276650959626, + "p95": 84.65979099855758, + "max": 84.65979099855758 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.326438132004114, + "end_to_end_tps": 0.6317508966956302, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/client.log new file mode 100644 index 0000000000..2216bef236 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 02:03:08 +{"label": "r3maink2", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 35.10294314642589, "decode_tps_all": [39.39, 29.95, 35.1], "ttft_s_median": 0.1451321700005792} +{"label": "r3maink2", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.56233754580456, "decode_tps_all": [29.34, 28.46, 28.56], "ttft_s_median": 0.21454531900235452} +{"label": "r3maink2", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.365893561680394, "decode_tps_all": [56.43, 56.35, 56.37], "ttft_s_median": 0.14565269701415673} +{"label": "r3maink2", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.67357053439874, "decode_tps_all": [30.8, 30.67, 30.66], "ttft_s_median": 0.21393800497753546} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.8607156460057, + "decode_tps": 32.8343731514751, + "elapsed_s": 25.317698785016546, + "chunk_offsets_s": [ + 24.8607156460057, + 24.946682211011648, + 25.020607040001778, + 25.09849363702233, + 25.17982723499881, + 25.255076638015453, + 25.317553970002336 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.56800001859665, + "p95": 85.9665650059469, + "max": 85.9665650059469 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.883651929994812, + "decode_tps": 32.8095833094987, + "elapsed_s": 25.3410171540163, + "chunk_offsets_s": [ + 24.883651929994812, + 24.968024358007824, + 25.041858694021357, + 25.11886785700335, + 25.200616578018526, + 25.27735730900895, + 25.34083542600274 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.87494698620867, + "p95": 84.37242801301181, + "max": 84.37242801301181 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.869684855017113, + "decode_tps": 32.85218483482, + "elapsed_s": 25.326438132004114, + "chunk_offsets_s": [ + 24.869684855017113, + 24.95434464601567, + 25.029398087994196, + 25.10609231001581, + 25.18799613902229, + 25.264427450019866, + 25.326275491999695 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.56276650959626, + "p95": 84.65979099855758, + "max": 84.65979099855758 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.326438132004114, + "end_to_end_tps": 0.6317508966956302, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.8899219760205597, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.4504265009891242, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.449036329984665, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.4516641689988319, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.271638671983965, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.282918252982199, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.279313892009668, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.794737174990587, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.584890022990294, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.706036354007665, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 25.323917712987168, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 25.20623288801289, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.877331081981538, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.35106128299958, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.27804015300353, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 50.32900873699691, "prompt": 65536, "cached": 0} +=== kvp-client r3maink2 rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/cold.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/cold.json new file mode 100644 index 0000000000..6bb3bdd3aa --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.8899219760205597, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.4504265009891242, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.449036329984665, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.4516641689988319, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.271638671983965, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.282918252982199, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.279313892009668, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.794737174990587, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.584890022990294, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.706036354007665, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 25.323917712987168, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 25.20623288801289, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.877331081981538, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.35106128299958, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.27804015300353, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 50.32900873699691, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/decode.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/decode.json new file mode 100644 index 0000000000..2b363814ce --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3maink2", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 35.10294314642589, + "decode_tps_all": [ + 39.39, + 29.95, + 35.1 + ], + "ttft_s_median": 0.1451321700005792, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3maink2", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.56233754580456, + "decode_tps_all": [ + 29.34, + 28.46, + 28.56 + ], + "ttft_s_median": 0.21454531900235452, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3maink2", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.365893561680394, + "decode_tps_all": [ + 56.43, + 56.35, + 56.37 + ], + "ttft_s_median": 0.14565269701415673, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3maink2", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.67357053439874, + "decode_tps_all": [ + 30.8, + 30.67, + 30.66 + ], + "ttft_s_median": 0.21393800497753546, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/server-rank0.log new file mode 100644 index 0000000000..7812210ce3 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/server-rank0.log @@ -0,0 +1,199 @@ +[kvp-serve] 2026-10-22T16:58:00+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:20192:20192 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:20192:20192 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:20192:20192 [0] NCCL INFO cudaDriverVersion 13030 +host-d:20192:20192 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:20192:20192 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:20192:20192 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:20192:20192 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:20192:20192 [0] NCCL INFO Count not open libibverbs.so +host-d:20192:20192 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:20192:20192 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:20192:20192 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:20192:20192 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:20192:20192 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x25b7c810 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:20192:20192 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:20192:20192 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:20192:20192 [0] NCCL INFO Initialized NET plugin IB +host-d:20192:20192 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:20192:20192 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:20192:20192 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:20192:20192 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:20192:20192 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:20192:20192 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:20192:20192 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:20192:20192 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:20192:20192 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:20192:20192 [0] NCCL INFO Using network IB +host-d:20192:20192 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:20192:20192 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x22f4a7c0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:20192:20192 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:20192:20192 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:20192:20192 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:20192:20192 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:20192:20192 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:20192:20192 [0] NCCL INFO Rank 0: 1 Net devices +host-d:20192:20192 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:20192:20192 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:20192:20192 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:20192:20192 [0] NCCL INFO comm 0x22f4a7c0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:20192:20192 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:20192:20192 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:20192:20192 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:20192:20192 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:20192:20192 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:20192:20192 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:20192:20192 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:20192:20192 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:20192:20192 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:20192:20192 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:20192:20211 [0] NCCL INFO [Proxy Service] Device 0 CPU core 11 +host-d:20192:20211 [0] NCCL INFO proxy listening socket at <60949> +host-d:20192:20212 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-d:20192:20192 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:20192:20192 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:20192:20192 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:20192:20192 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:20192:20213 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-d:20192:20192 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:20192:20192 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:20192:20192 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:20192:20192 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeaa624010000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeaa624054000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeaa62409c000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1723 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1723 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9c99 ctsFifoLkey=0x1b9c99 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1723 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1724 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1724 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ab2af ctsFifoLkey=0x1ab2af +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1724 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeaa62414e000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeaa624196000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeaa6241da000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeaa624222000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeaa624266000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1723 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1725 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1726 pkey=0 pd=0xeaa624053590 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1724 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1727 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1727 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5a59 ctsFifoLkey=0x1b5a59 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1727 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1728 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1729 pkey=0 pd=0xeaa624053590 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1730 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1730 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b6764 ctsFifoLkey=0x1b6764 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1730 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1731 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1732 pkey=0 pd=0xeaa624053590 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1733 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1734 pkey=0 pd=0xeaa624053590 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1727 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1730 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20192 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:20192:20192 [0] NCCL INFO Connected all trees +host-d:20192:20192 [0] NCCL INFO Connected binomial trees +host-d:20192:20192 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:20192:20192 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:20192:20192 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:20192:20192 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.7 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 305.7s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-2c75fb793fe5a63dd3712377f4ac08d8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.147 decode=6.476 elapsed=6.622 queued=0.000 tok/s=39.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-c812bedbd1156b57bcaca6891b09ed22 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.143 decode=6.474 elapsed=6.617 queued=0.000 tok/s=39.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-b393f9f21dca360741531aa61535155d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.146 decode=8.515 elapsed=8.660 queued=0.000 tok/s=29.9 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-d85f396a93dca522719e2fbc7a8a4d6d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.144 decode=7.265 elapsed=7.409 queued=0.000 tok/s=35.1 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7289966987c4556ab122d777abf6339f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.213 decode=8.693 elapsed=8.906 queued=0.000 tok/s=29.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5c0c269f9df3eabebbc21f4a8d30d19b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.213 decode=8.691 elapsed=8.905 queued=0.000 tok/s=29.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-401b27c75dd9f9845ca36f3ababaf9e0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.213 decode=8.961 elapsed=9.174 queued=0.000 tok/s=28.5 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5aaf268cfb8568a39cbd2fd6767b1242 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.214 decode=8.928 elapsed=9.142 queued=0.000 tok/s=28.6 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-6bd4f7eebb2433f8fae0f3a08a7220d7 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.530 elapsed=4.674 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-9dd2e7b1ba67e1340716e45385836975 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.519 elapsed=4.664 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-26693eb6f49587027a77023cf5e9ed01 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.525 elapsed=4.669 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-3a9796d4177d32871a8df97f4dea4f07 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.524 elapsed=4.669 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-df2118e88e4509168327de228f046dd1 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.212 decode=8.302 elapsed=8.515 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0634b388c9b9524088e2c788a874816b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.212 decode=8.278 elapsed=8.491 queued=0.000 tok/s=30.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0a0514c967b822185c29a160228b16d0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.212 decode=8.314 elapsed=8.526 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3d5bff0ef3da78e9ef4d70b67314d6dd prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.316 elapsed=8.530 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-c2abfe2c62ba393e5ee6c328e529e7d0 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.845 decode=0.457 elapsed=25.302 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4bec403bcc30976bae5f00ba89658009 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.880 decode=0.457 elapsed=25.337 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-38d79fcac2025ab9860836ca9a1580e0 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.867 decode=0.457 elapsed=25.324 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2135aca3f92aaeac5c54d0cc5ec088aa prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.876 decode=0.063 elapsed=0.938 queued=0.000 tok/s=16.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0046abe574feab1b3ea10ffe3b4df7c7 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.448 decode=0.063 elapsed=1.512 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4f4416638de555ecf17e9bb5cd7c829a prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.448 decode=0.064 elapsed=1.512 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-996a5e25922b0cbd036d74c8f538a2e8 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.451 decode=0.063 elapsed=1.514 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-30ef099b4abc01e481ec4cff3fa79961 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.271 decode=0.065 elapsed=6.336 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a6d9409eeb4a9c8595f1ace6c1248c48 prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.281 decode=0.065 elapsed=6.346 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2e0036be93e892d5c4f51befa3d86465 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.278 decode=0.065 elapsed=6.343 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-543661418c2371e73f20319469869ad9 prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.793 decode=0.068 elapsed=12.861 queued=0.000 tok/s=14.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2733c1b90e9f5a864978ecfbd03d69ea prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.584 decode=0.065 elapsed=12.648 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-85605b3054d993716a93f9fd287ca3f7 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.705 decode=0.065 elapsed=12.770 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-88d885c05f1bc571851e84fa6745ef93 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=25.322 decode=0.070 elapsed=25.392 queued=0.000 tok/s=14.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-974533d394ab0741edf18c663100c270 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=25.205 decode=0.065 elapsed=25.270 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-77a8873c107774af7f1b0e7fe3cd9d98 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.876 decode=0.065 elapsed=24.941 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c083b40353b5063b174fd2a25a18822b prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.349 decode=0.065 elapsed=50.413 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-cdeef1760e50e53d8ccc8c95189da0fe prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.276 decode=0.066 elapsed=50.342 queued=0.000 tok/s=15.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0e339f461a0b02132948c9febef10dd8 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=50.326 decode=0.064 elapsed=50.390 queued=0.000 tok/s=15.6 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:20192:20192 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:20192:20192 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:20192:20192 [0] NCCL INFO ncclCommInitRank comm 0x22f4a7c0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:20192:20192 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.40 (kernels 0.25, alloc 0.03, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-d:20192:20192 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:20192:20192 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeaa62444c000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeaa62448d000) +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1735 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1735 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a1211 ctsFifoLkey=0x1a1211 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1735 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1736 pkey=0 pd=0xeaa624053590 oooRq=0 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1737 pkey=0 pd=0xeaa624053590 +host-d:20192:20211 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1735 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:20192:20192 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/server-rank1.log new file mode 100644 index 0000000000..9be220f460 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink2/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T16:57:59+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:1947:1947 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:1947:1947 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:1947:1947 [0] NCCL INFO cudaDriverVersion 13030 +host-e:1947:1947 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:1947:1947 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:1947:1947 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:1947:1947 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:1947:1947 [0] NCCL INFO Count not open libibverbs.so +host-e:1947:1947 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:1947:1947 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:1947:1947 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:1947:1947 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:1947:1947 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x146d7600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:1947:1947 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:1947:1947 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:1947:1947 [0] NCCL INFO Initialized NET plugin IB +host-e:1947:1947 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:1947:1947 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:1947:1947 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:1947:1947 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:1947:1947 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:1947:1947 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:1947:1947 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:1947:1947 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:1947:1947 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:1947:1947 [0] NCCL INFO Using network IB +host-e:1947:1947 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:1947:1947 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x11aa57a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:1947:1947 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:1947:1947 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1947:1947 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1947:1947 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:1947:1947 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:1947:1947 [0] NCCL INFO Rank 1: 1 Net devices +host-e:1947:1947 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:1947:1947 [0] NCCL INFO comm 0x11aa57a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:1947:1947 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:1947:1947 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:1947:1947 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:1947:1947 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:1947:1947 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:1947:1947 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:1947:1960 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 12 +host-e:1947:1959 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-e:1947:1959 [0] NCCL INFO proxy listening socket at <40575> +host-e:1947:1947 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1947:1947 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1947:1947 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1947:1947 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:1947:1961 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-e:1947:1947 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1947:1947 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1947:1947 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1947:1947 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6db64010000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6db64054000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6db6409c000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6db640e0000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6db64128000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6db6416c000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6db641b4000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6db641f8000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1467 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1468 pkey=0 pd=0xf6db64053eb0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1469 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1469 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x179653 ctsFifoLkey=0x179653 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1469 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1470 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1471 pkey=0 pd=0xf6db64053eb0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1472 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1472 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x14f1b0 ctsFifoLkey=0x14f1b0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1472 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1473 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1473 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16dc8d ctsFifoLkey=0x16dc8d +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1473 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1474 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1474 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1474 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1469 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1472 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1475 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1476 pkey=0 pd=0xf6db64053eb0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1473 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1477 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1478 pkey=0 pd=0xf6db64053eb0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1474 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1947 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:1947:1947 [0] NCCL INFO Connected all trees +host-e:1947:1947 [0] NCCL INFO Connected binomial trees +host-e:1947:1947 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:1947:1947 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:1947:1947 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:1947:1947 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:1947:1947 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:1947:1947 [0] NCCL INFO ncclCommInitRank comm 0x11aa57a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:1947:1947 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.40 (kernels 0.24, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-e:1947:1947 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Sharinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +ed +host-e:1947:1947 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6db64445000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6db64486000) +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1479 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1479 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x16ce8b ctsFifoLkey=0x16ce8b +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1479 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1480 pkey=0 pd=0xf6db64053eb0 oooRq=0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1481 pkey=0 pd=0xf6db64053eb0 +host-e:1947:1959 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1479 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:1947:1947 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/cache.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/cache.json new file mode 100644 index 0000000000..7b29c02871 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.8953561130038, + "decode_tps": 33.043574377476816, + "elapsed_s": 25.349481795012252, + "chunk_offsets_s": [ + 24.8953561130038, + 24.97972166899126, + 25.053276302991435, + 25.130630931991618, + 25.21185336698545, + 25.28768251600559, + 25.34930216099019 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.59188901016023, + "p95": 84.36555598746054, + "max": 84.36555598746054 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.912096535990713, + "decode_tps": 32.8379070990678, + "elapsed_s": 25.36904589101323, + "chunk_offsets_s": [ + 24.912096535990713, + 24.997489342989866, + 25.073100253997836, + 25.14888083099504, + 25.23079979198519, + 25.30702572100563, + 25.368885695992503 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.00325300882105, + "p95": 85.39280699915253, + "max": 85.39280699915253 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.900052429991774, + "decode_tps": 32.80304133443928, + "elapsed_s": 25.357489185000304, + "chunk_offsets_s": [ + 24.900052429991774, + 24.984854576003272, + 25.059108815010404, + 25.136225051013753, + 25.21901507000439, + 25.29425178898964, + 25.357327103003627 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.17647749430034, + "p95": 84.80214601149783, + "max": 84.80214601149783 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.357489185000304, + "end_to_end_tps": 0.6309772976050195, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/client.log new file mode 100644 index 0000000000..ec338c3619 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 02:35:54 +{"label": "r3maink22", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 35.1503599482965, "decode_tps_all": [39.41, 30.0, 35.15], "ttft_s_median": 0.14623097400180995} +{"label": "r3maink22", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.617541553578327, "decode_tps_all": [29.37, 28.49, 28.62], "ttft_s_median": 0.21460182999726385} +{"label": "r3maink22", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.36742321546269, "decode_tps_all": [56.41, 56.37, 56.33], "ttft_s_median": 0.1448655720159877} +{"label": "r3maink22", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.71130887759936, "decode_tps_all": [30.72, 30.71, 30.68], "ttft_s_median": 0.21462927298853174} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.8953561130038, + "decode_tps": 33.043574377476816, + "elapsed_s": 25.349481795012252, + "chunk_offsets_s": [ + 24.8953561130038, + 24.97972166899126, + 25.053276302991435, + 25.130630931991618, + 25.21185336698545, + 25.28768251600559, + 25.34930216099019 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.59188901016023, + "p95": 84.36555598746054, + "max": 84.36555598746054 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.912096535990713, + "decode_tps": 32.8379070990678, + "elapsed_s": 25.36904589101323, + "chunk_offsets_s": [ + 24.912096535990713, + 24.997489342989866, + 25.073100253997836, + 25.14888083099504, + 25.23079979198519, + 25.30702572100563, + 25.368885695992503 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.00325300882105, + "p95": 85.39280699915253, + "max": 85.39280699915253 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.900052429991774, + "decode_tps": 32.80304133443928, + "elapsed_s": 25.357489185000304, + "chunk_offsets_s": [ + 24.900052429991774, + 24.984854576003272, + 25.059108815010404, + 25.136225051013753, + 25.21901507000439, + 25.29425178898964, + 25.357327103003627 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.17647749430034, + "p95": 84.80214601149783, + "max": 84.80214601149783 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.357489185000304, + "end_to_end_tps": 0.6309772976050195, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.8921407540037762, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.4517986020073295, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.4516151319839992, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.4389410669973586, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.297900417004712, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.273478757997509, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.296741740021389, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.475890108005842, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.428618489007931, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.498862173990346, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 24.90791681301198, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 24.93143779100501, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.920320462988457, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.43783000399708, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.287958343018545, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 50.330790538020665, "prompt": 65536, "cached": 0} +=== kvp-client r3maink22 rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/cold.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/cold.json new file mode 100644 index 0000000000..7c55a97b9d --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.8921407540037762, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.4517986020073295, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.4516151319839992, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.4389410669973586, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.297900417004712, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.273478757997509, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.296741740021389, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.475890108005842, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.428618489007931, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.498862173990346, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 24.90791681301198, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 24.93143779100501, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.920320462988457, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.43783000399708, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.287958343018545, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 50.330790538020665, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/decode.json b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/decode.json new file mode 100644 index 0000000000..476973c0a6 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3maink22", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 35.1503599482965, + "decode_tps_all": [ + 39.41, + 30.0, + 35.15 + ], + "ttft_s_median": 0.14623097400180995, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3maink22", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.617541553578327, + "decode_tps_all": [ + 29.37, + 28.49, + 28.62 + ], + "ttft_s_median": 0.21460182999726385, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3maink22", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.36742321546269, + "decode_tps_all": [ + 56.41, + 56.37, + 56.33 + ], + "ttft_s_median": 0.1448655720159877, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3maink22", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.71130887759936, + "decode_tps_all": [ + 30.72, + 30.71, + 30.68 + ], + "ttft_s_median": 0.21462927298853174, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/server-rank0.log new file mode 100644 index 0000000000..92ca8ea5e8 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/server-rank0.log @@ -0,0 +1,199 @@ +[kvp-serve] 2026-10-22T17:30:46+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:21076:21076 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:21076:21076 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:21076:21076 [0] NCCL INFO cudaDriverVersion 13030 +host-d:21076:21076 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:21076:21076 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:21076:21076 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:21076:21076 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:21076:21076 [0] NCCL INFO Count not open libibverbs.so +host-d:21076:21076 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:21076:21076 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:21076:21076 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:21076:21076 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:21076:21076 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x17d9d980 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:21076:21076 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:21076:21076 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:21076:21076 [0] NCCL INFO Initialized NET plugin IB +host-d:21076:21076 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:21076:21076 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:21076:21076 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:21076:21076 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:21076:21076 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:21076:21076 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:21076:21076 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:21076:21076 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:21076:21076 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:21076:21076 [0] NCCL INFO Using network IB +host-d:21076:21076 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:21076:21076 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x1516b900 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:21076:21076 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:21076:21076 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:21076:21076 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:21076:21076 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:21076:21076 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:21076:21076 [0] NCCL INFO Rank 0: 1 Net devices +host-d:21076:21076 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:21076:21076 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:21076:21076 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:21076:21076 [0] NCCL INFO comm 0x1516b900 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:21076:21076 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:21076:21076 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:21076:21076 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:21076:21076 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:21076:21076 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:21076:21076 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:21076:21076 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:21076:21076 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:21076:21076 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:21076:21076 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:21076:21095 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-d:21076:21095 [0] NCCL INFO proxy listening socket at <53925> +host-d:21076:21096 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-d:21076:21097 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-d:21076:21076 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21076:21076 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21076:21076 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21076:21076 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21076:21076 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21076:21076 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21076:21076 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21076:21076 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe96a4010000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe96a4054000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe96a409c000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1755 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1755 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9c99 ctsFifoLkey=0x1b9c99 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1755 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1756 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1756 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1aebe8 ctsFifoLkey=0x1aebe8 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1756 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe96a414e000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe96a4196000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe96a41da000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe96a4222000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe96a4266000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1755 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1757 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1758 pkey=0 pd=0xfe96a4053590 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1759 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1759 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b8986 ctsFifoLkey=0x1b8986 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1759 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1760 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1761 pkey=0 pd=0xfe96a4053590 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1762 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1762 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b6764 ctsFifoLkey=0x1b6764 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1762 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1763 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1764 pkey=0 pd=0xfe96a4053590 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1765 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1766 pkey=0 pd=0xfe96a4053590 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1756 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1759 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1762 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21076 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:21076:21076 [0] NCCL INFO Connected all trees +host-d:21076:21076 [0] NCCL INFO Connected binomial trees +host-d:21076:21076 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:21076:21076 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:21076:21076 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:21076:21076 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.5 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 304.1s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-619f3cf81b0afd0362a882f81a6dfd6d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.468 elapsed=6.614 queued=0.000 tok/s=39.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4f720c54f7351019f1a78206a390881e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.145 decode=6.470 elapsed=6.615 queued=0.000 tok/s=39.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-e3c0e24fe8a18efc990c5f6373c8ef7f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.144 decode=8.502 elapsed=8.646 queued=0.000 tok/s=30.0 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-355f9cf88b0414b5f5b1a2868d025065 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.146 decode=7.255 elapsed=7.401 queued=0.000 tok/s=35.1 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-cb793e970d6bc8d40413008d95a9d0ef prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.213 decode=8.671 elapsed=8.884 queued=0.000 tok/s=29.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ab3801e0c025ad1e164992ed725237c0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.215 decode=8.682 elapsed=8.896 queued=0.000 tok/s=29.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-afd433282583cb364c3cc2f1ce4528fd prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.213 decode=8.951 elapsed=9.164 queued=0.000 tok/s=28.5 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-208f36301d2e8c1192b5ee57f7ea7741 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.214 decode=8.911 elapsed=9.124 queued=0.000 tok/s=28.6 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-29b0bd0e294222ced8930c5ce0190e4d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.520 elapsed=4.664 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-7117e9cb57ea6919afbc96b44d2afc01 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.521 elapsed=4.665 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-c32a5ee6ebf85e9320938e6b85433dbf prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.524 elapsed=4.668 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-f7424a4bd3b362bc9540dc4ae0fedeaf prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.527 elapsed=4.673 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-441fa924c1e885c8a9f682e3ab9d5e0a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.300 elapsed=8.513 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-51fbdbb29971ab7bf0e836a102af4b41 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.301 elapsed=8.515 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1f3bac0bd736ef0a079384517793d431 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.303 elapsed=8.518 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-91effd0ad7b36533999a35dc1f1ff1c2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.313 elapsed=8.527 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-1b639357ae3dc14ebdddbcd3516012c1 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.880 decode=0.454 elapsed=25.334 queued=0.000 tok/s=33.0 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-86be3db97f4dcd8ba0678c67149671b6 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.908 decode=0.457 elapsed=25.365 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-60a578addf14c2597ead8f38669da167 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.897 decode=0.457 elapsed=25.355 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a758e3555d6d4455c8657840400aa962 prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.878 decode=0.063 elapsed=0.941 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3220819ecbb61f8842cb8b6d5c208b18 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.451 decode=0.063 elapsed=1.514 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4ecf2194df35abf01f6b4a99f41cd798 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.450 decode=0.063 elapsed=1.514 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fec58594aeb9b64fc67945bba68dd2d0 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.438 decode=0.064 elapsed=1.502 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-bb14159deea495a32aa0f62dc60d8a4d prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.297 decode=0.064 elapsed=6.360 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f6b31265718d41bfceaf322e6c9e9290 prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.272 decode=0.064 elapsed=6.336 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1306bdc279ca326402ea3394b62d6081 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.295 decode=0.065 elapsed=6.361 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-65a26f6088558b39462e9f2e3494ec7e prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.475 decode=0.065 elapsed=12.540 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a546d9009c381b8de1168add0bf327fb prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.427 decode=0.065 elapsed=12.492 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ba8efa6120d8fd160e39486c2a4aee07 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.498 decode=0.065 elapsed=12.562 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6c82e33b264bcd28e6bcf3d81558a329 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.906 decode=0.065 elapsed=24.972 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3da0ed2abd58966f964a77576903648d prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=24.930 decode=0.066 elapsed=24.996 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ed0d773d07c9da9d20d3ae7de4b873b8 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.918 decode=0.064 elapsed=24.982 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e34dff6b65daa0f8bc41deca628d6d0f prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.436 decode=0.064 elapsed=50.501 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-22c24ced45d1c0caa66eb79cdb2b4933 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.286 decode=0.065 elapsed=50.351 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2de755526bbfeb74d3ea208fe96888d2 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=50.328 decode=0.065 elapsed=50.393 queued=0.000 tok/s=15.4 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:21076:21076 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:21076:21076 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:21076:21076 [0] NCCL INFO ncclCommInitRank comm 0x1516b900 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:21076:21076 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.41 (kernels 0.25, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.12, rest 0.00) +host-d:21076:21076 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:21076:21076 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe96a4450000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe96a4491000) +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1767 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1768 pkey=0 pd=0xfe96a4053590 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1769 pkey=0 pd=0xfe96a4053590 oooRq=0 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1769 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1bc4c2 ctsFifoLkey=0x1bc4c2 +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1769 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21095 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1769 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21076:21076 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/server-rank1.log new file mode 100644 index 0000000000..f7baa7ca89 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp2/speed/r3maink22/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-22T17:30:45+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:2029:2029 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2029:2029 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2029:2029 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2029:2029 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2029:2029 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2029:2029 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2029:2029 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2029:2029 [0] NCCL INFO Count not open libibverbs.so +host-e:2029:2029 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2029:2029 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2029:2029 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2029:2029 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2029:2029 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x2959f600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2029:2029 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2029:2029 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2029:2029 [0] NCCL INFO Initialized NET plugin IB +host-e:2029:2029 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2029:2029 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2029:2029 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2029:2029 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2029:2029 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2029:2029 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2029:2029 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2029:2029 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2029:2029 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2029:2029 [0] NCCL INFO Using network IB +host-e:2029:2029 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2029:2029 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x2696d7a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2029:2029 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2029:2029 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2029:2029 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2029:2029 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2029:2029 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2029:2029 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2029:2029 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2029:2029 [0] NCCL INFO comm 0x2696d7a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2029:2029 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2029:2029 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2029:2029 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2029:2029 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2029:2029 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2029:2029 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2029:2041 [0] NCCL INFO [Proxy Service] Device 0 CPU core 11 +host-e:2029:2041 [0] NCCL INFO proxy listening socket at <57219> +host-e:2029:2042 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 11 +host-e:2029:2029 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2029:2043 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-e:2029:2029 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2029:2029 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2029:2029 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2029:2029 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2029:2029 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2029:2029 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2029:2029 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe631a8010000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe631a8054000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe631a809c000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe631a80e0000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe631a8128000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe631a816c000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe631a81b4000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe631a81f8000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1499 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1500 pkey=0 pd=0xe631a8053eb0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1501 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1501 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x174908 ctsFifoLkey=0x174908 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1501 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1502 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1503 pkey=0 pd=0xe631a8053eb0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1504 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1504 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x14f1b0 ctsFifoLkey=0x14f1b0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1504 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1505 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1505 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x179653 ctsFifoLkey=0x179653 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1505 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1506 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1506 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17a261 ctsFifoLkey=0x17a261 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1506 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1501 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1504 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1507 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1508 pkey=0 pd=0xe631a8053eb0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1505 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1509 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1510 pkey=0 pd=0xe631a8053eb0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1506 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2029 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2029:2029 [0] NCCL INFO Connected all trees +host-e:2029:2029 [0] NCCL INFO Connected binomial trees +host-e:2029:2029 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2029:2029 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2029:2029 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2029:2029 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2029:2029 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2029:2029 [0] NCCL INFO ncclCommInitRank comm 0x2696d7a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2029:2029 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.42 (kernels 0.24, alloc 0.04, bootstrap 0.02, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.12, rest 0.00) +host-e:2029:2029 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Sharinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +ed +host-e:2029:2029 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe631a8448000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe631a8489000) +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1511 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1511 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x14a461 ctsFifoLkey=0x14a461 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1511 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1511 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1512 pkey=0 pd=0xe631a8053eb0 oooRq=0 +host-e:2029:2041 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1513 pkey=0 pd=0xe631a8053eb0 +host-e:2029:2029 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/build.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/build.log new file mode 100644 index 0000000000..702c437b2b --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/build.log @@ -0,0 +1,11 @@ +-- run 2026-10-23 01:19:39 +2026-10-23 01:19:40 tree abc83c3d glm cuda: a stream takes its pages before its sequence is bound, so a slot past t; changed: zig/glm_prompt_host.zig zig/glm_snapshot_host.zig zig/src/core/heat.zig zig/src/core/kv_pool.zig zig/src/core/kv_pool_test.zig zig/src/families/glm/cuda.zig zig/src/families/glm/cuda_dsa_block.zig zig/src/families/glm/cuda_engine.zig zig/src/families/glm/cuda_follow.zig zig/src/families/glm/cuda_kv8_test.zig zig/src/families/glm/cuda_lanes.zig zig/src/families/glm/cuda_latent.zig zig/src/families/glm/cuda_native.zig zig/src/families/glm/cuda_pages.zig zig/src/families/glm/cuda_pages_test.zig zig/src/families/glm/cuda_prompts.zig zig/src/families/glm/cuda_ranks.zig zig/src/families/glm/cuda_snapshot_layout.zig zig/src/families/glm/cuda_snapshot_pages.zig zig/src/families/glm/cuda_snapshots.zig zig/src/families/glm/cuda_sparse.zig zig/src/families/glm/cuda_state.zig zig/src/native/cuda_kv_pool.zig zig/tests/cuda/glm_paged_tests.zig zig/tests/cuda/glm_ref_attn.zig zig/tests/cuda/glm_ref_fp8.zig zig/tests/cuda/glm_ref_index.zig zig/tests/cuda/main.zig +2026-10-23 01:19:40 fmt rc=0 +2026-10-23 01:19:40 -- tp3: abc83c3d glm cuda: a stream takes its pages before its sequence is bound, so a slot past t +2026-10-23 01:24:46 build rc=0 306s: +2026-10-23 01:27:17 native rc=0 151s: +2026-10-23 01:28:27 host rc=0 69s: failed command: ./.zig-cache/o/5b21a6c856bcc580edb3128b41ac4e50/test --cache-dir=./.zig-cache --seed=0xaeb51753 --listen=- failed command: ./.zig-cache/o/50bc9f88b9ae4d8712463603b5238381/test --cache-dir=./.zig-cache --seed=0xaeb51753 --listen=- Build Summary: 155/155 steps succeeded; 986/986 tests +2026-10-23 01:28:33 kvpool rc=0 6s: Build Summary: 5/5 steps succeeded; 6/6 tests passed +2026-10-23 01:29:09 test rc=0 36s: failed command: ./.zig-cache/o/7250ba077cad6049f6c43b354768d2f0/test --cache-dir=./.zig-cache --seed=0xe551a39b --listen=- Build Summary: 276/276 steps succeeded; 916/924 tests passed (8 skipped) | | +- compile lib hip-mock-failed debug native success 45ms MaxRSS:63M +2026-10-23 01:29:09 lean rc=1 0s: +2026-10-23 01:29:09 === kvp-base tp3 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/chain.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/chain.log new file mode 100644 index 0000000000..878289b18a --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/chain.log @@ -0,0 +1,12 @@ +2026-10-23 01:49:02 -- R6 chain on kvp-tp3-fx from check; rank.env without TF_GLM_MM (1.1.2's default) +2026-10-23 01:49:28 binaries from 04's tp3: tensorfold-native 62649bbd8c2ce0d4, tf-cuda-test 2707a1813f751c12 +2026-10-23 01:50:21 setup rc 0: up on host-a: /hub is /.cache/huggingface/h up on host-b: /hub is /.cache/huggingface/h up on host-c: /hub is /.cache/huggingface/h +2026-10-23 01:51:13 check: === kvp3-check pg3 rc 0 0 0 +2026-10-23 02:17:16 auto: === kvp3-phase r6auto rc 0 +2026-10-23 02:34:03 all: === kvp3-phase r6all rc 0 +2026-10-23 02:52:51 off: === kvp3-phase r6off rc 0 +2026-10-23 02:52:52 off's replay against auto's record (logs/r6-compare-off.log): rc=0 B t4 drafted prompt 19808 cached 16384: ttft 4.93 s vs fresh 26.47 s ( 5.4x) PASS: 43/43 replies equal and as cached as planned timing: 0 resumed replies no sooner than fresh +2026-10-23 02:55:54 admit: a 22.8 GiB prompt cache (the all step's room less its streams' fixed parts and 0.9 GiB of pages) +2026-10-23 03:03:06 admit: === kvp3-phase r6admit rc 0 +2026-10-23 03:03:42 stopped on 01-03; GPU processes there: 0 +== R6 finished diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/check/glm-paged-check-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/check/glm-paged-check-rank0.log new file mode 100644 index 0000000000..e78ac626fa --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/check/glm-paged-check-rank0.log @@ -0,0 +1,10857 @@ +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:85:85 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:85:85 [0] NCCL INFO cudaDriverVersion 13030 +host-a:85:85 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:85:85 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:85:85 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:85:85 [0] NCCL INFO Count not open libibverbs.so +host-a:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:85:85 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:85:85 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0xc1b9a60 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:85:85 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0xc1fb160 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:85:85 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0xc23c860 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:85:85 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0xc27df60 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:85:85 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:85:85 [0] NCCL INFO Initialized NET plugin IB +host-a:85:85 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:85:85 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:85:85 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:85:85 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:85:85 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:85:85 [0] NCCL INFO Using network IB +host-a:85:85 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:85:85 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x95822a0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xe99f74da3134b0e7 - Init START +host-a:85:85 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:85:85 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:85:85 [0] NCCL INFO Rank 0: 6 Net devices +host-a:85:85 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:85:85 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:85:85 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:85:85 [0] NCCL INFO comm 0x95822a0 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:85:85 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:85:85 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:85:85 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:85:85 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:85:85 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:85:85 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:85:85 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:85:85 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:85:96 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-a:85:95 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-a:85:95 [0] NCCL INFO proxy listening socket at <42035> +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:85:85 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:85:85 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:85:85 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 1, globalGinSupport 0, cuMemGdrSupport 0 +host-a:85:85 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:85:85 [0] NCCL INFO ncclCommInitRank comm 0x95822a0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xe99f74da3134b0e7 - Init COMPLETE +host-a:85:85 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 4.27 (kernels 0.24, alloc 0.04, bootstrap 3.97, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.00, rest 0.00) +host-a:85:85 [0] NCCL INFO Symmetric VA size=124GB +host-a:85:105 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-a:85:85 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:85:85 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:85:85 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:85:85 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfc010000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf74bfc054000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfc09c000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf74bfc0e0000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfc128000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf74bfc16c000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfc1b4000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf74bfc1f8000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6815 pkey=0 pd=0xf74bfc053d50 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6816 pkey=0 pd=0xf74bfc053d50 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6815 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6815 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6816 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6816 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1877 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1878 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1879 pkey=0 pd=0xf74bfc09be40 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6783 pkey=0 pd=0xf74bfc283d60 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6784 pkey=0 pd=0xf74bfc283d60 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6783 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1be8e4 ctsFifoLkey=0x1be8e4 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6783 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6784 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1be8e4 ctsFifoLkey=0x1be8e4 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6784 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6817 pkey=0 pd=0xf74bfc053d50 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6818 pkey=0 pd=0xf74bfc053d50 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6817 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2c2a ctsFifoLkey=0x1d2c2a +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6817 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6818 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2c2a ctsFifoLkey=0x1d2c2a +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6818 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6785 pkey=0 pd=0xf74bfc283d60 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6786 pkey=0 pd=0xf74bfc283d60 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6785 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6785 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6786 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6786 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1749 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1750 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1751 pkey=0 pd=0xf74bfc33b800 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6815 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6816 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6783 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6784 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1752 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1753 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1754 pkey=0 pd=0xf74bfc33b800 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6817 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6818 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6785 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6786 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1880 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1881 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1882 pkey=0 pd=0xf74bfc09be40 +host-a:85:85 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:85:85 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:85:85 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:85:85 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:85:85 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfc5ab000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfc5ec000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf74bfc630000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6787 pkey=0 pd=0xf74bfc283d60 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6788 pkey=0 pd=0xf74bfc283d60 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6787 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x19605d ctsFifoLkey=0x19605d +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6787 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6788 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x19605d ctsFifoLkey=0x19605d +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6788 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6819 pkey=0 pd=0xf74bfc053d50 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6820 pkey=0 pd=0xf74bfc053d50 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6819 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c8585 ctsFifoLkey=0x1c8585 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6819 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6820 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c8585 ctsFifoLkey=0x1c8585 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6820 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6821 pkey=0 pd=0xf74bfc053d50 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6822 pkey=0 pd=0xf74bfc053d50 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6823 pkey=0 pd=0xf74bfc053d50 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf74bfc7a8000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6787 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6788 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6819 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6820 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6789 pkey=0 pd=0xf74bfc283d60 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6790 pkey=0 pd=0xf74bfc283d60 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6791 pkey=0 pd=0xf74bfc283d60 +host-a:85:85 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:85:85 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:85:85 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:85:85 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf74bfc862000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf74bfc8a7000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfc8ef000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfc933000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1755 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1756 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1757 pkey=0 pd=0xf74bfc33b800 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1883 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1884 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1885 pkey=0 pd=0xf74bfc09be40 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1886 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1887 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1886 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1886 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1887 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1887 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1758 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1759 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1758 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x179e59 ctsFifoLkey=0x179e59 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1758 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1759 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x179e59 ctsFifoLkey=0x179e59 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1759 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1886 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1887 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1758 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1759 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:85 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:85:85 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfcb20000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfcb61000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfcba2000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf74bfcbe3000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1760 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1761 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1760 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x1811ce ctsFifoLkey=0x1811ce +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1760 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1761 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x1811ce ctsFifoLkey=0x1811ce +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1761 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1888 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1889 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1888 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b0af ctsFifoLkey=0x19b0af +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1888 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1889 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b0af ctsFifoLkey=0x19b0af +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1889 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1762 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1763 pkey=0 pd=0xf74bfc33b800 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1762 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x177632 ctsFifoLkey=0x177632 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1762 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1763 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x177632 ctsFifoLkey=0x177632 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1763 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1890 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1891 pkey=0 pd=0xf74bfc09be40 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1890 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a2a0 ctsFifoLkey=0x19a2a0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1890 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1891 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a2a0 ctsFifoLkey=0x19a2a0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1891 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1760 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1761 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1888 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1889 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1762 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1763 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dinfo: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +PASS pages at TP=3: 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after +ev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1890 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1891 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:85 [0] NCCL INFO Connected all trees diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/check/glm-paged-check-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/check/glm-paged-check-rank1.log new file mode 100644 index 0000000000..a918ce6bad --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/check/glm-paged-check-rank1.log @@ -0,0 +1,2499 @@ +host-b:86:86 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:86:86 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:86:86 [0] NCCL INFO cudaDriverVersion 13030 +host-b:86:86 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:86:86 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:86:86 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:86:86 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:86:86 [0] NCCL INFO Count not open libibverbs.so +host-b:86:86 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:86:86 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:86:86 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:86:86 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x278e88a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:86:86 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x27929fa0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:86:86 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x2796b6a0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:86:86 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x279acda0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:86:86 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:86:86 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:86:86 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:86:86 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:86:86 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:86:86 [0] NCCL INFO Initialized NET plugin IB +host-b:86:86 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:86:86 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:86:86 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:86:86 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:86:86 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:86:86 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:86:86 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:86:86 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:86:86 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:86:86 [0] NCCL INFO Using network IB +host-b:86:86 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:86:86 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x24cb1210 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xe99f74da3134b0e7 - Init START +host-b:86:86 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:86:86 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:86:86 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:86:86 [0] NCCL INFO NET/GIN_IB_GDAinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +KI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:86:86 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:86:86 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:86:86 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:86:86 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:86:86 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:86:86 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:86:86 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:86:86 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:86:86 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:86:86 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:86:86 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:86:86 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:86:86 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:86:86 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:86:86 [0] NCCL INFO Rank 1: 6 Net devices +host-b:86:86 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:86:86 [0] NCCL INFO comm 0x24cb1210 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:86:86 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:86:86 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:86:86 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:86:86 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:86:86 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:86:86 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:86:96 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-b:86:95 [0] NCCL INFO [Proxy Service] Device 0 CPU core 11 +host-b:86:95 [0] NCCL INFO proxy listening socket at <36967> +host-b:86:86 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:86:86 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:86:86 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:86:86 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:86:86 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 1, globalGinSupport 0, cuMemGdrSupport 0 +host-b:86:86 [0] NCCL INFO ncclCommInitRank comm 0x24cb1210 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xe99f74da3134b0e7 - Init COMPLETE +host-b:86:86 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 1.07 (kernels 0.25, alloc 0.03, bootstrap 0.76, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.00, rest 0.00) +host-b:86:86 [0] NCCL INFO Symmetric VA size=124GB +host-b:86:105 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-b:86:86 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:86:86 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:86:86 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:86:86 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:86:86 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:86:86 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:86:86 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:86:86 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe1f2dc010000) +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dc054000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe1f2dc09c000) +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dc0e0000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2005 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2006 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2005 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2005 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2006 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2006 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe1f2dc18d000) +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dc1d1000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe1f2dc219000) +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dc25d000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2133 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2134 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2133 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1da9a2 ctsFifoLkey=0x1da9a2 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2133 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2134 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1da9a2 ctsFifoLkey=0x1da9a2 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2134 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6589 pkey=0 pd=0xe1f2dc13ae60 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6590 pkey=0 pd=0xe1f2dc13ae60 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6591 pkey=0 pd=0xe1f2dc13ae60 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2005 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2006 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2133 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2134 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6557 pkey=0 pd=0xe1f2dc09b1a0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6558 pkey=0 pd=0xe1f2dc09b1a0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6559 pkey=0 pd=0xe1f2dc09b1a0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6592 pkey=0 pd=0xe1f2dc13ae60 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6593 pkey=0 pd=0xe1f2dc13ae60 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6594 pkey=0 pd=0xe1f2dc13ae60 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6560 pkey=0 pd=0xe1f2dc09b1a0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6561 pkey=0 pd=0xe1f2dc09b1a0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6562 pkey=0 pd=0xe1f2dc09b1a0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2007 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2008 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2007 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdcd9 ctsFifoLkey=0x1bdcd9 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2007 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2008 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdcd9 ctsFifoLkey=0x1bdcd9 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2008 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2135 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2136 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2135 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1deee4 ctsFifoLkey=0x1deee4 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2135 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2136 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1deee4 ctsFifoLkey=0x1deee4 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2136 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2007 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2008 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2135 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2136 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:86 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:86:86 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:86:86 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:86:86 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:86:86 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dc5b3000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dc5f8000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe1f2dc63d000) +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe1f2dc67e000) +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6595 pkey=0 pd=0xe1f2dc13ae60 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6596 pkey=0 pd=0xe1f2dc13ae60 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6595 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6595 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6596 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6596 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6563 pkey=0 pd=0xe1f2dc09b1a0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6564 pkey=0 pd=0xe1f2dc09b1a0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6565 pkey=0 pd=0xe1f2dc09b1a0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6597 pkey=0 pd=0xe1f2dc13ae60 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6598 pkey=0 pd=0xe1f2dc13ae60 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6599 pkey=0 pd=0xe1f2dc13ae60 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6595 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6596 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6566 pkey=0 pd=0xe1f2dc09b1a0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6567 pkey=0 pd=0xe1f2dc09b1a0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6566 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x177532 ctsFifoLkey=0x177532 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6566 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6567 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x177532 ctsFifoLkey=0x177532 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6567 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6566 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6567 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:86 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:86:86 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:86:86 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:86:86 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe1f2dc867000) +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe1f2dc8a8000) +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2009 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2010 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2009 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2009 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2010 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2010 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2137 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2138 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2137 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1cc8c9 ctsFifoLkey=0x1cc8c9 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2137 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2138 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1cc8c9 ctsFifoLkey=0x1cc8c9 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2138 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dc99d000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dc9e5000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2009 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2010 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2137 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2138 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2139 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2140 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2141 pkey=0 pd=0xe1f2dc16bc70 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2011 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2012 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2013 pkey=0 pd=0xe1f2dc00fef0 +host-b:86:86 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:86:86 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:86:86 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:86:86 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dcb19000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dcb5e000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dcba3000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe1f2dcbeb000) +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2014 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2015 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2016 pkey=0 pd=0xe1f2dc00fef0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2142 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmPASS rank 1 made rank 0's pool and mapped its pages through every step +devs=6 qp_num=2143 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2144 pkey=0 pd=0xe1f2dc16bc70 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2017 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2018 pkey=0 pd=0xe1f2dc00fef0 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2019 pkey=0 pd=0xe1f2dc00fef0 +host-b:86:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2145 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2146 pkey=0 pd=0xe1f2dc16bc70 oooRq=0 +host-b:86:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2147 pkey=0 pd=0xe1f2dc16bc70 +host-b:86:86 [0] NCCL INFO Connected all trees diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/check/glm-paged-check-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/check/glm-paged-check-rank2.log new file mode 100644 index 0000000000..def936140d --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/check/glm-paged-check-rank2.log @@ -0,0 +1,14550 @@ +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:85:85 [0] NCCL INFO cudaDriverVersion 13030 +host-c:85:85 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:85:85 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:85:85 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:85:85 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:85:85 [0] NCCL INFO Count not open libibverbs.so +host-c:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:85:85 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:85:85 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x465fb890 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:85:85 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x4663cf90 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:85:85 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x4667e690 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:85:85 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x466bfd90 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:85:85 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:85:85 [0] NCCL INFO Initialized NET plugin IB +host-c:85:85 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:85:85 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:85:85 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:85:85 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:85:85 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:85:85 [0] NCCL INFO Using network IB +host-c:85:85 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:85:85 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x439c40c0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xe99f74da3134b0e7 - Init START +host-c:85:85 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:85:85 [0] NCCL INFO NET/GIN_IB_GDAinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +KI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:85:85 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:85:85 [0] NCCL INFO Rank 2: 6 Net devices +host-c:85:85 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:85:85 [0] NCCL INFO comm 0x439c40c0 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:85:85 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:85:85 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:85:85 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:85:85 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:85:95 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 11 +host-c:85:94 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-c:85:94 [0] NCCL INFO proxy listening socket at <36631> +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:85:85 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:85:85 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:85:85 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 1, globalGinSupport 0, cuMemGdrSupport 0 +host-c:85:85 [0] NCCL INFO ncclCommInitRank comm 0x439c40c0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xe99f74da3134b0e7 - Init COMPLETE +host-c:85:85 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.35 (kernels 0.29, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.00, rest 0.00) +host-c:85:85 [0] NCCL INFO Symmetric VA size=124GB +host-c:85:104 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-c:85:85 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f70010000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f70054000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f7009c000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2005 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2006 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2005 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bc2bf ctsFifoLkey=0x1bc2bf +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2005 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2006 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bc2bf ctsFifoLkey=0x1bc2bf +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2006 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2133 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2134 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2133 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d4d4d ctsFifoLkey=0x1d4d4d +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2133 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2134 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d4d4d ctsFifoLkey=0x1d4d4d +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2134 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f701aa000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1749 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1750 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1751 pkey=0 pd=0xe50f700f2f40 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1877 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1878 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1879 pkey=0 pd=0xe50f7020cc30 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f702fa000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2133 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2134 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2007 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2008 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2007 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1ba8a5 ctsFifoLkey=0x1ba8a5 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2007 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2008 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1ba8a5 ctsFifoLkey=0x1ba8a5 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2008 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f70393000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2005 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2006 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f703db000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f7041f000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2007 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2008 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2135 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2136 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2135 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d7373 ctsFifoLkey=0x1d7373 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2135 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2136 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d7373 ctsFifoLkey=0x1d7373 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2136 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1752 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1753 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1754 pkey=0 pd=0xe50f700f2f40 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1880 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1881 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1882 pkey=0 pd=0xe50f7020cc30 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2135 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2136 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:85 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:85:85 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:85:85 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:85:85 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:85:85 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f705ae000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f705ef000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2009 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2010 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2009 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1ba09d ctsFifoLkey=0x1ba09d +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2009 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2010 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1ba09d ctsFifoLkey=0x1ba09d +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2010 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2137 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2138 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2137 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5e5e ctsFifoLkey=0x1d5e5e +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2137 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2138 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5e5e ctsFifoLkey=0x1d5e5e +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2138 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f706e6000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f7072e000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2009 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2010 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2137 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2138 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2139 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2140 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2141 pkey=0 pd=0xe50f70053e50 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2011 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2012 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2013 pkey=0 pd=0xe50f7000fe50 +host-c:85:85 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:85:85 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:85:85 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:85:85 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f70864000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f708a9000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f708ee000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f70932000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1755 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1756 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1757 pkey=0 pd=0xe50f700f2f40 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1883 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1884 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1885 pkey=0 pd=0xe50f7020cc30 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1886 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1887 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1886 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19736b ctsFifoLkey=0x19736b +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1886 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1887 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19736b ctsFifoLkey=0x19736b +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1887 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1758 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1759 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1758 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17e4a1 ctsFifoLkey=0x17e4a1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1758 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1759 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17e4a1 ctsFifoLkey=0x17e4a1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1759 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1886 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1887 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1758 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1759 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:85 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f70b20000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f70b61000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f70ba6000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1760 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1761 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1760 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1808c6 ctsFifoLkey=0x1808c6 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1760 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1761 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1808c6 ctsFifoLkey=0x1808c6 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1761 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1888 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1889 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1888 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1888 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1889 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1889 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f70c9b000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f70ce3000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f70d27000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe50f70d6f000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe50f70db3000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2014 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2015 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2016 pkey=0 pd=0xe50f7000fe50 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1888 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1889 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2142 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2143 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2144 pkey=0 pd=0xe50f70053e50 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1762 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1763 pkey=0 pd=0xe50f700f2f40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1762 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1813d0 ctsFifoLkey=0x1813d0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1762 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1763 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1813d0 ctsFifoLkey=0x1813d0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1763 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2017 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=PASS rank 2 made rank 0's pool and mapped its pages through every step +rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2018 pkey=0 pd=0xe50f7000fe50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2019 pkey=0 pd=0xe50f7000fe50 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1890 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1891 pkey=0 pd=0xe50f7020cc30 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1890 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19ece9 ctsFifoLkey=0x19ece9 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1890 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1891 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19ece9 ctsFifoLkey=0x19ece9 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1891 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2145 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2146 pkey=0 pd=0xe50f70053e50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2147 pkey=0 pd=0xe50f70053e50 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1760 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1761 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1762 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1763 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1890 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1891 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:85 [0] NCCL INFO Connected all trees diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/client.log new file mode 100644 index 0000000000..2aa421b1d2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/client.log @@ -0,0 +1,18 @@ +server up 2026-10-22 19:40:02 +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 48.3, "per_stream_tps": 48.3, "ttft_s_max": 0.11, "timing_valid": true, "end_to_end_tps": 47.45985407922176, "makespan_s": 5.39403259800747, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 59.2, "per_stream_tps": 32.2, "ttft_s_max": 0.28, "timing_valid": true, "end_to_end_tps": 58.70339831596406, "makespan_s": 8.721811934025027, "steady_tps": 61.4, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 76.4, "per_stream_tps": 20.4, "ttft_s_max": 0.67, "timing_valid": true, "end_to_end_tps": 76.04496690158155, "makespan_s": 13.4657169530401, "steady_tps": 78.6, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 8, "failed": 8, "unmeasured": 8, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 8}, "serial": {"equal": 8, "unequal": 0, "failed": 0}, "errors": ["incomplete_stream"]} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 1, "failed": 1, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 1}, "serial": {"equal": 0, "unequal": 0, "failed": 8}, "errors": ["URLError"]} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 2, "failed": 2, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 2}, "serial": {"equal": 0, "unequal": 0, "failed": 8}, "errors": ["URLError"]} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 4, "failed": 4, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 4}, "serial": {"equal": 0, "unequal": 0, "failed": 8}, "errors": ["BrokenBarrierError", "URLError"]} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 8, "failed": 8, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 8}, "serial": {"equal": 0, "unequal": 0, "failed": 8}, "errors": ["BrokenBarrierError", "URLError"]} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 1, "failed": 1, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 1}, "serial": {"equal": 0, "unequal": 0, "failed": 1}, "errors": ["URLError"]} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 2, "failed": 2, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 2}, "serial": {"equal": 0, "unequal": 0, "failed": 1}, "errors": ["URLError"]} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 4, "failed": 4, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 4}, "serial": {"equal": 0, "unequal": 0, "failed": 1}, "errors": ["URLError"]} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 8, "failed": 8, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 8}, "serial": {"equal": 0, "unequal": 0, "failed": 1}, "errors": ["URLError"]} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 1, "failed": 1, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 1}, "serial": {"equal": 0, "unequal": 0, "failed": 1}, "errors": ["URLError"]} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 2, "failed": 2, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 2}, "serial": {"equal": 0, "unequal": 0, "failed": 1}, "errors": ["URLError"]} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 4, "failed": 4, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 4}, "serial": {"equal": 0, "unequal": 0, "failed": 1}, "errors": ["BrokenBarrierError", "URLError"]} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 8, "failed": 8, "unmeasured": 0, "aggregate_tps": 0.0, "per_stream_tps": 0.0, "ttft_s_max": null, "timing_valid": false, "end_to_end_tps": null, "makespan_s": null, "alone": {"equal": 0, "unequal": 0, "failed": 8}, "serial": {"equal": 0, "unequal": 0, "failed": 1}, "errors": ["URLError"]} +=== kvp-client r6all rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/conc.json b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/conc.json new file mode 100644 index 0000000000..78d443e832 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/conc.json @@ -0,0 +1,4174 @@ +{ + "label": "", + "tokens": 256, + "mixed": false, + "cells": [ + { + "prompt": "code", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.3, + "per_stream_tps": 48.3, + "ttft_s_max": 0.11, + "timing_valid": true, + "end_to_end_tps": 47.45985407922176, + "makespan_s": 5.39403259800747, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.26343258031526, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 5.39403259800747, + "end_to_end_tps": 47.45985407922176, + "timing_valid": true, + "per_stream_tps": [ + 48.3 + ], + "ttft_s": [ + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + } + ], + "client_intervals": [ + { + "start": 699257.256069495, + "first": 699257.366419715, + "last": 699262.649922905, + "end": 699262.650102093, + "elapsed_s": 5.39403259800747, + "chunk_offsets_s": [ + 0.11035021999850869, + 0.16889823600649834, + 0.22564734902698547, + 0.28613140794914216, + 0.3521183410193771, + 0.4193612539675087, + 0.4830583609873429, + 0.5466030159732327, + 0.6114611719967797, + 0.6785253609996289, + 0.7450337139889598, + 0.8135293590603396, + 0.881653442978859, + 0.9481894279597327, + 1.0141698010265827, + 1.0758072930620983, + 1.1335026019951329, + 1.1884206819813699, + 1.239485859987326, + 1.2940277389716357, + 1.354746266035363, + 1.415642990032211, + 1.4754732419969514, + 1.535227125044912, + 1.596589106018655, + 1.6651820479892194, + 1.7350514190038666, + 1.8027960379840806, + 1.8678528539603576, + 1.9349667869973928, + 2.000845829024911, + 2.0665579859633, + 2.1325830629793927, + 2.1995053270366043, + 2.268662648042664, + 2.3377965609543025, + 2.405792913050391, + 2.4743452930124477, + 2.5448197319637984, + 2.6139784300467, + 2.682532425969839, + 2.749593093059957, + 2.8176984710153192, + 2.888580678962171, + 2.9569274389650673, + 3.026780343032442, + 3.0963359769666567, + 3.1654521609889343, + 3.2348680950235575, + 3.3046079559717327, + 3.374089795979671, + 3.4427581620402634, + 3.5105085719842464, + 3.577206942019984, + 3.6442822329699993, + 3.712422715034336, + 3.7773423900362104, + 3.8366226439829916, + 3.8967852380592376, + 3.9602367050247267, + 4.0260104780318215, + 4.091033739037812, + 4.179273191955872, + 4.2492332010297105, + 4.316419646027498, + 4.384819508995861, + 4.448954125051387, + 4.514064428047277, + 4.603422721964307, + 4.668945241020992, + 4.734166810056195, + 4.801566267036833, + 4.8696032740408555, + 4.935788640053943, + 5.006206964026205, + 5.072952053043991, + 5.14250269299373, + 5.212150359060615, + 5.28003899299074, + 5.345427924999967, + 5.393853410030715 + ], + "chunk_gaps_ms": { + "count": 80, + "p50": 66.99146557366475, + "p95": 70.41832397226244, + "max": 89.3582939170301 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.2, + "per_stream_tps": 32.2, + "ttft_s_max": 0.28, + "timing_valid": true, + "end_to_end_tps": 58.70339831596406, + "makespan_s": 8.721811934025027, + "steady_tps": 61.4, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.23633674293942, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 8.721811934025027, + "end_to_end_tps": 58.70339831596406, + "timing_valid": true, + "steady_tps": 61.43078264687199, + "per_stream_tps": [ + 34.7, + 29.6 + ], + "ttft_s": [ + 0.28, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + } + ], + "client_intervals": [ + { + "start": 699262.651766466, + "first": 699262.929043643, + "last": 699270.287400645, + "end": 699270.287587337, + "elapsed_s": 7.635820870986208, + "chunk_offsets_s": [ + 0.2772771769668907, + 0.35721124603878707, + 0.4495818620780483, + 0.5427548310253769, + 0.6375885080778971, + 0.7296702039893717, + 0.8182191980304196, + 0.9107883760007098, + 1.0019994290778413, + 1.097416765987873, + 1.1933689310681075, + 1.2882444490678608, + 1.3835472710197791, + 1.4807170470012352, + 1.5761444490635768, + 1.6675321609945968, + 1.7597779710777104, + 1.8502503340132535, + 1.9434382769977674, + 2.0383856990374625, + 2.134816401056014, + 2.2307078280718997, + 2.326107627013698, + 2.42182597797364, + 2.5199482219759375, + 2.616268954006955, + 2.7100458049681038, + 2.806051825056784, + 2.8998720360687003, + 2.993935197009705, + 3.0884269119706005, + 3.1853131430689245, + 3.2851278070593253, + 3.3824205750133842, + 3.4794055439997464, + 3.574201200972311, + 3.6707997140474617, + 3.769070566049777, + 3.863673242041841, + 3.960270729032345, + 4.0582025670446455, + 4.153635437018238, + 4.2501123460242525, + 4.3489872589707375, + 4.449339742073789, + 4.550474768970162, + 4.6502763270400465, + 4.749801333993673, + 4.845325500005856, + 4.941075176000595, + 5.041908884071745, + 5.139477079967037, + 5.234611270017922, + 5.329754100064747, + 5.425657922984101, + 5.523950151051395, + 5.622720837011002, + 5.716199164977297, + 5.812779786065221, + 5.91008965508081, + 6.011448701028712, + 6.110438734991476, + 6.208365448983386, + 6.302616658969782, + 6.398040966014378, + 6.497635476989672, + 6.594263923005201, + 6.688718016026542, + 6.7847610709723085, + 6.880757582024671, + 6.97692825703416, + 7.073475306970067, + 7.173717863042839, + 7.275207151076756, + 7.370015315013006, + 7.465385843999684, + 7.5550999289844185, + 7.6356341789942235 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 95.89142701588571, + "p95": 100.83370807114989, + "max": 101.48928803391755 + } + }, + { + "start": 699262.651481051, + "first": 699262.763534725, + "last": 699271.373115061, + "end": 699271.373292985, + "elapsed_s": 8.721811934025027, + "chunk_offsets_s": [ + 0.11205367394722998, + 0.1701077719917521, + 0.35808933002408594, + 0.44994558196049184, + 0.5431813369505107, + 0.6380831920541823, + 0.7302454810123891, + 0.8186212390428409, + 0.9115325850434601, + 1.00250063999556, + 1.097939754021354, + 1.1938634869875386, + 1.2886282820254564, + 1.3839090400142595, + 1.480816442053765, + 1.5766705089481547, + 1.6681335830362514, + 1.7603269279934466, + 1.8506681199651212, + 1.9438073739875108, + 2.0388909419998527, + 2.1352297549601644, + 2.2311960470397025, + 2.3265652540139854, + 2.4221784670371562, + 2.5202989659737796, + 2.6167691739974543, + 2.7104521660367027, + 2.8065526679856703, + 2.9004080810118467, + 2.9945741560077295, + 3.0888966349884868, + 3.1858860040083528, + 3.2855863129952922, + 3.382820599945262, + 3.4797923379810527, + 3.574700140976347, + 3.671426287968643, + 3.769486560020596, + 3.8642053180374205, + 3.9607321639778093, + 4.058662225026637, + 4.153997045010328, + 4.250514067010954, + 4.349596568965353, + 4.449757096008398, + 4.5509699650574476, + 4.650852243998088, + 4.750384578946978, + 4.845679717021994, + 4.941503809997812, + 5.042286748997867, + 5.139561738003977, + 5.235048896051012, + 5.330330225056969, + 5.426241329056211, + 5.52433252800256, + 5.623114877962507, + 5.71672029804904, + 5.813238085014746, + 5.910501137026586, + 6.011802533059381, + 6.11087887303438, + 6.208454475970939, + 6.302990700001828, + 6.398494769004174, + 6.498150352970697, + 6.594686285010539, + 6.689119177055545, + 6.785196186043322, + 6.881158104049973, + 6.977351226960309, + 7.073880132986233, + 7.174454119987786, + 7.275736971059814, + 7.370391468051821, + 7.465859374962747, + 7.555547954980284, + 7.636068733059801, + 7.70174881699495, + 7.7587117799557745, + 7.810080380993895, + 7.8634201609529555, + 7.922537651960738, + 7.981190301012248, + 8.03967265796382, + 8.091326736961491, + 8.14357337297406, + 8.201605480979197, + 8.259705492993817, + 8.321239548968151, + 8.382204488967545, + 8.438564240001142, + 8.503689366974868, + 8.589389874017797, + 8.663626831956208, + 8.72163400996942 + ], + "chunk_gaps_ms": { + "count": 96, + "p50": 95.35201347898692, + "p95": 100.7829390000552, + "max": 187.98155803233385 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 76.4, + "per_stream_tps": 20.4, + "ttft_s_max": 0.67, + "timing_valid": true, + "end_to_end_tps": 76.04496690158155, + "makespan_s": 13.4657169530401, + "steady_tps": 78.6, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 76.37857413743171, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 13.4657169530401, + "end_to_end_tps": 76.04496690158155, + "timing_valid": true, + "steady_tps": 78.64076360864135, + "per_stream_tps": [ + 22.7, + 19.9, + 20.4, + 20.3 + ], + "ttft_s": [ + 0.11, + 0.67, + 0.46, + 0.28 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0b4d014c8872", + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b82191a9a6ec", + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a" + } + ], + "client_intervals": [ + { + "start": 699271.375546843, + "first": 699271.485905442, + "last": 699282.704147621, + "end": 699282.704451147, + "elapsed_s": 11.328904303954914, + "chunk_offsets_s": [ + 0.11035859899129719, + 0.16936552699189633, + 0.35217386193107814, + 0.5664740079082549, + 0.8021822889568284, + 0.9329052709508687, + 1.0607459489256144, + 1.1923727099783719, + 1.321702619898133, + 1.4528822669526562, + 1.5841583169531077, + 1.7258352979552, + 1.8641288619255647, + 2.004149165004492, + 2.1493328619981185, + 2.2878072599414736, + 2.4234275779454038, + 2.552558954921551, + 2.6916008689440787, + 2.8296500249998644, + 2.967877632007003, + 3.1113806819776073, + 3.2513995439512655, + 3.395468477974646, + 3.53518864500802, + 3.680030923918821, + 3.8202555190073326, + 3.96040479792282, + 4.103419883991592, + 4.238124450901523, + 4.379612638964318, + 4.521386800915934, + 4.663813297986053, + 4.809597068000585, + 4.953013728954829, + 5.0950329509796575, + 5.2427170649170876, + 5.3889933079481125, + 5.528857570956461, + 5.669189827982336, + 5.811810294981115, + 5.955878871958703, + 6.097173644928262, + 6.243884727009572, + 6.390027045970783, + 6.535705626010895, + 6.6807721758959815, + 6.859471894917078, + 7.053326198947616, + 7.251751067000441, + 7.416661873925477, + 7.5578438739757985, + 7.701086558983661, + 7.841468461905606, + 7.987430612905882, + 8.134154475992545, + 8.282433716929518, + 8.424527094932273, + 8.56751527695451, + 8.71113156795036, + 8.857155191944912, + 9.002533216960728, + 9.153022695914842, + 9.293569766916335, + 9.439689982915297, + 9.58783862798009, + 9.735097892931662, + 9.8784737609094, + 10.023386492975987, + 10.169984173960984, + 10.319003860000521, + 10.469159793923609, + 10.61157560790889, + 10.762014091946185, + 10.906263801967725, + 11.054820674937218, + 11.199204641976394, + 11.328600778011605 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 143.37586797773838, + "p95": 193.8543040305376, + "max": 235.70828104857355 + } + }, + { + "start": 699271.376062694, + "first": 699272.051031757, + "last": 699284.84043597, + "end": 699284.841128257, + "elapsed_s": 13.46506556298118, + "chunk_offsets_s": [ + 0.6749690630240366, + 0.8013065420091152, + 0.9321345660137013, + 1.060301715042442, + 1.1914565779734403, + 1.3210787669522688, + 1.4521092750364915, + 1.583414508961141, + 1.7251052189385518, + 1.8639008729951456, + 2.003393740975298, + 2.148403050028719, + 2.2871092930436134, + 2.4228440140141174, + 2.551609958987683, + 2.6908252680441365, + 2.828678115969524, + 2.967085359035991, + 3.110558344051242, + 3.2507792899850756, + 3.3947864630026743, + 3.5344266450265422, + 3.6791269689565524, + 3.8193661549594253, + 3.9594896900234744, + 4.102622490026988, + 4.237484804936685, + 4.3793286009458825, + 4.520729506970383, + 4.663015680969693, + 4.808811211027205, + 4.952161694993265, + 5.094710511970334, + 5.241659041959792, + 5.388247404014692, + 5.528093538014218, + 5.668422195012681, + 5.8111261039739475, + 5.955200953991152, + 6.096514062024653, + 6.243165544001386, + 6.389369782991707, + 6.534816805971786, + 6.679973887046799, + 6.858468128950335, + 7.052666904986836, + 7.250903144944459, + 7.416025156970136, + 7.556893870001659, + 7.700068393023685, + 7.840798398945481, + 7.98647300794255, + 8.133514189976268, + 8.281778790988028, + 8.423799976008013, + 8.566776204970665, + 8.710159515030682, + 8.856409864034504, + 9.001643038005568, + 9.15215002198238, + 9.293646664940752, + 9.43865663302131, + 9.587171845021658, + 9.734209249960259, + 9.877729280968197, + 10.022551371017471, + 10.169280319008976, + 10.318233603029512, + 10.468403425998986, + 10.61074555804953, + 10.76098328595981, + 10.905485608964227, + 11.054141653003171, + 11.198574995971285, + 11.32828147499822, + 11.44426072598435, + 11.563624865957536, + 11.682121915975586, + 11.802300106966868, + 11.921814922010526, + 12.039243355044164, + 12.157034356030636, + 12.27673170203343, + 12.39391284994781, + 12.51460340199992, + 12.635958191938698, + 12.761138394940645, + 12.868294715997763, + 12.952700251014903, + 13.018962863949127, + 13.079408852965571, + 13.135311783989891, + 13.201536139007658, + 13.287035458954051, + 13.360022314009257, + 13.415263597969897, + 13.464373275986873 + ], + "chunk_gaps_ms": { + "count": 96, + "p50": 141.35700702900067, + "p95": 150.50698397681117, + "max": 198.23623995762318 + } + }, + { + "start": 699271.375705502, + "first": 699271.835353069, + "last": 699284.328927397, + "end": 699284.329034951, + "elapsed_s": 12.953329449053854, + "chunk_offsets_s": [ + 0.45964756701141596, + 0.5661953940289095, + 0.8018174810567871, + 0.9329236559569836, + 1.0608072630129755, + 1.1923218769952655, + 1.321795727009885, + 1.4530176790431142, + 1.5842044779565185, + 1.7257740650093183, + 1.8643343390431255, + 2.0034146540565416, + 2.1488302600337192, + 2.2878655169624835, + 2.4234419469721615, + 2.5522302279714495, + 2.691692615975626, + 2.829246385022998, + 2.9676845399662852, + 3.1111845660489053, + 3.2512828540056944, + 3.3952667460544035, + 3.535320264985785, + 3.679646724020131, + 3.8198752549942583, + 3.9601741699734703, + 4.103227064013481, + 4.237673689960502, + 4.379300985019654, + 4.521315327961929, + 4.663844194961712, + 4.8097272630548105, + 4.953106452012435, + 5.094753585988656, + 5.242301695980132, + 5.389143279986456, + 5.529136680997908, + 5.669065137044527, + 5.811613218975253, + 5.955908122006804, + 6.097245279001072, + 6.243690756033175, + 6.3899375080363825, + 6.535508102038875, + 6.680430712993257, + 6.859443175024353, + 7.053241061978042, + 7.251429317053407, + 7.416729683987796, + 7.557599244988523, + 7.7006197810405865, + 7.841567232040688, + 7.9877126349601895, + 8.134392146021128, + 8.282239826046862, + 8.424585176981054, + 8.56751191697549, + 8.711002942058258, + 8.857161673018709, + 9.002409982029349, + 9.153043161029927, + 9.293200287036598, + 9.43947991495952, + 9.587910774047486, + 9.735135862021707, + 9.878239420009777, + 10.023341628955677, + 10.169918428990059, + 10.319141607033089, + 10.469380839029327, + 10.611671369988471, + 10.762111135059968, + 10.906345403986052, + 11.054889781051315, + 11.199286611983553, + 11.328498615999706, + 11.445131625048816, + 11.564459893037565, + 11.682791562983766, + 11.802815621951595, + 11.922345830011182, + 12.04045763798058, + 12.157696833950467, + 12.277422116952948, + 12.394387979991734, + 12.515165686025284, + 12.636491707991809, + 12.761659526033327, + 12.869311889982782, + 12.953221895033494 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 142.0143429422751, + "p95": 165.30036693438888, + "max": 235.62208702787757 + } + }, + { + "start": 699271.375411304, + "first": 699271.653400289, + "last": 699284.244552374, + "end": 699284.244720665, + "elapsed_s": 12.869309361092746, + "chunk_offsets_s": [ + 0.2779889850644395, + 0.3522669040830806, + 0.5665551620768383, + 0.8020390050951391, + 0.9328426930587739, + 1.0606776910135522, + 1.1923418450169265, + 1.3219519060803577, + 1.4528104740893468, + 1.5841318040620536, + 1.7255407320335507, + 1.864180767093785, + 2.003566177096218, + 2.14970725402236, + 2.2877190180588514, + 2.423453291063197, + 2.552333798026666, + 2.6919221720891073, + 2.82927913707681, + 2.967883776058443, + 3.111416955012828, + 3.251341638038866, + 3.3955041110748425, + 3.535546557046473, + 3.6801241430221125, + 3.820330672082491, + 3.960628403001465, + 4.103714882046916, + 4.238202149048448, + 4.379683840088546, + 4.52131448010914, + 4.663912948104553, + 4.809777168091387, + 4.953099075006321, + 5.095119593082927, + 5.242787947063334, + 5.389174704090692, + 5.529219803051092, + 5.669178995070979, + 5.812137598055415, + 5.956164991017431, + 6.097458163043484, + 6.2441877101082355, + 6.390122040058486, + 6.535726154106669, + 6.681065591052175, + 6.859778862097301, + 7.053741745068692, + 7.252048242022283, + 7.416950297076255, + 7.557738128001802, + 7.701487272046506, + 7.8417835250729695, + 7.987784301047213, + 8.134467699099332, + 8.282673115027137, + 8.424706091056578, + 8.567498045042157, + 8.711095168022439, + 8.857387661002576, + 9.002549777040258, + 9.153118986054324, + 9.29405016102828, + 9.439535372075625, + 9.58812124200631, + 9.73506646801252, + 9.878684038063511, + 10.023431678069755, + 10.170071376021951, + 10.319220888079144, + 10.469401671085507, + 10.611784860026091, + 10.762094606063329, + 10.906446942011826, + 11.055257293046452, + 11.199386150110513, + 11.328853631042875, + 11.445157225010917, + 11.5645028850995, + 11.682893724995665, + 11.803168812999502, + 11.922860896098427, + 12.040479238028638, + 12.157806180068292, + 12.27765578601975, + 12.394891175092198, + 12.515507149044424, + 12.636713025043719, + 12.761999501031823, + 12.869141070055775 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 142.03297602944076, + "p95": 178.71327104512602, + "max": 235.48384301830083 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 8, + "failed": 8, + "unmeasured": 8, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 8 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "errors": [ + "incomplete_stream" + ], + "reps": [ + { + "failed": 8, + "unmeasured": 8, + "aggregate_tps": 0.0, + "requested_requests": 8, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null, + null, + null, + null, + null, + null, + null + ], + "errors": [ + "incomplete_stream", + "incomplete_stream", + "incomplete_stream", + "incomplete_stream", + "incomplete_stream", + "incomplete_stream", + "incomplete_stream", + "incomplete_stream" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": 699284.847558781, + "first": 699286.289784155, + "last": 699286.47668344, + "end": 699289.651930933, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": 699284.845712917, + "first": 699285.122491078, + "last": 699286.47637929, + "end": 699289.636919183, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": 699284.845773462, + "first": 699286.021378149, + "last": 699286.477012295, + "end": 699289.660523968, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": 699284.846485094, + "first": 699285.523492615, + "last": 699286.47641273, + "end": 699289.663352094, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": 699284.846176687, + "first": 699285.76136002, + "last": 699286.476890549, + "end": 699289.64608727, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": 699284.84595505, + "first": 699285.309444256, + "last": 699286.477180603, + "end": 699289.656794831, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": 699284.847022994, + "first": null, + "last": null, + "end": 699289.640119221, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": 699284.845604595, + "first": 699284.95717157, + "last": 699286.476532765, + "end": 699289.658585334, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null, + null, + null, + null, + null, + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "incomplete_stream" + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "incomplete_stream" + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "incomplete_stream" + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "incomplete_stream" + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "incomplete_stream" + }, + "alone": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "incomplete_stream" + }, + "alone": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "incomplete_stream" + }, + "alone": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "incomplete_stream" + }, + "alone": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 1, + "failed": 1, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 8 + }, + "errors": [ + "URLError" + ], + "reps": [ + { + "failed": 1, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 1, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null + ], + "errors": [ + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 2, + "failed": 2, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 2 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 8 + }, + "errors": [ + "URLError" + ], + "reps": [ + { + "failed": 2, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 2, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null + ], + "errors": [ + "URLError", + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 4, + "failed": 4, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 4 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 8 + }, + "errors": [ + "BrokenBarrierError", + "URLError" + ], + "reps": [ + { + "failed": 4, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 4, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null, + null, + null + ], + "errors": [ + "URLError", + "BrokenBarrierError", + "BrokenBarrierError", + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null, + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "BrokenBarrierError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "BrokenBarrierError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 8, + "failed": 8, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 8 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 8 + }, + "errors": [ + "BrokenBarrierError", + "URLError" + ], + "reps": [ + { + "failed": 8, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 8, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null, + null, + null, + null, + null, + null, + null + ], + "errors": [ + "URLError", + "URLError", + "URLError", + "URLError", + "URLError", + "URLError", + "BrokenBarrierError", + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null, + null, + null, + null, + null, + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "BrokenBarrierError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 1, + "failed": 1, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "errors": [ + "URLError" + ], + "reps": [ + { + "failed": 1, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 1, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null + ], + "errors": [ + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 2, + "failed": 2, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 2 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "errors": [ + "URLError" + ], + "reps": [ + { + "failed": 2, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 2, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null + ], + "errors": [ + "URLError", + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 4, + "failed": 4, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 4 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "errors": [ + "URLError" + ], + "reps": [ + { + "failed": 4, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 4, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null, + null, + null + ], + "errors": [ + "URLError", + "URLError", + "URLError", + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null, + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 8, + "failed": 8, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 8 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "errors": [ + "URLError" + ], + "reps": [ + { + "failed": 8, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 8, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null, + null, + null, + null, + null, + null, + null + ], + "errors": [ + "URLError", + "URLError", + "URLError", + "URLError", + "URLError", + "URLError", + "URLError", + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null, + null, + null, + null, + null, + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 1, + "failed": 1, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "errors": [ + "URLError" + ], + "reps": [ + { + "failed": 1, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 1, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null + ], + "errors": [ + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 2, + "failed": 2, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 2 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "errors": [ + "URLError" + ], + "reps": [ + { + "failed": 2, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 2, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null + ], + "errors": [ + "URLError", + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 4, + "failed": 4, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 4 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "errors": [ + "BrokenBarrierError", + "URLError" + ], + "reps": [ + { + "failed": 4, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 4, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null, + null, + null + ], + "errors": [ + "URLError", + "BrokenBarrierError", + "URLError", + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null, + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "BrokenBarrierError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 8, + "failed": 8, + "unmeasured": 0, + "aggregate_tps": 0.0, + "per_stream_tps": 0.0, + "ttft_s_max": null, + "timing_valid": false, + "end_to_end_tps": null, + "makespan_s": null, + "alone": { + "equal": 0, + "unequal": 0, + "failed": 8 + }, + "serial": { + "equal": 0, + "unequal": 0, + "failed": 1 + }, + "errors": [ + "URLError" + ], + "reps": [ + { + "failed": 8, + "unmeasured": 0, + "aggregate_tps": 0.0, + "requested_requests": 8, + "timed_requests": 0, + "makespan_s": null, + "end_to_end_tps": null, + "timing_valid": false, + "per_stream_tps": [], + "ttft_s": [ + null, + null, + null, + null, + null, + null, + null, + null + ], + "errors": [ + "URLError", + "URLError", + "URLError", + "URLError", + "URLError", + "URLError", + "URLError", + "URLError" + ], + "evidence": [ + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + }, + { + "prompt_tokens": null, + "cached_tokens": null, + "tokens": null, + "cache_state": "unverified", + "complete": false, + "token_sha": null, + "token_sha256": null, + "output_sha256": null + } + ], + "client_intervals": [ + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + }, + { + "start": null, + "first": null, + "last": null, + "end": null, + "elapsed_s": null, + "chunk_offsets_s": null, + "chunk_gaps_ms": null + } + ], + "equal_alone": [ + null, + null, + null, + null, + null, + null, + null, + null + ], + "comparisons": [ + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + }, + { + "concurrent": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "alone": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + }, + "serial": { + "tokens": null, + "token_sha256": null, + "output_sha256": null, + "complete": false, + "error": "URLError" + } + } + ] + } + ] + } + ] +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/hostwatch-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/hostwatch-rank0.log new file mode 100644 index 0000000000..480af45036 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/hostwatch-rank0.log @@ -0,0 +1,202 @@ +19:36:02 114 +19:36:04 112 +19:36:06 111 +19:36:08 111 +19:36:10 109 +19:36:12 109 +19:36:14 108 +19:36:17 108 +19:36:19 108 +19:36:21 107 +19:36:23 107 +19:36:25 105 +19:36:27 105 +19:36:29 105 +19:36:31 104 +19:36:33 104 +19:36:35 103 +19:36:37 102 +19:36:39 102 +19:36:41 102 +19:36:43 101 +19:36:45 101 +19:36:47 100 +19:36:49 100 +19:36:51 99 +19:36:53 98 +19:36:56 98 +19:36:58 96 +19:37:00 96 +19:37:02 96 +19:37:04 95 +19:37:06 95 +19:37:08 94 +19:37:10 94 +19:37:12 94 +19:37:14 93 +19:37:16 92 +19:37:18 92 +19:37:20 91 +19:37:22 91 +19:37:24 91 +19:37:26 89 +19:37:28 90 +19:37:30 88 +19:37:33 88 +19:37:35 87 +19:37:37 87 +19:37:39 87 +19:37:41 85 +19:37:43 85 +19:37:45 85 +19:37:47 84 +19:37:49 84 +19:37:51 84 +19:37:53 83 +19:37:55 83 +19:37:57 81 +19:37:59 81 +19:38:01 80 +19:38:03 79 +19:38:05 79 +19:38:07 78 +19:38:09 78 +19:38:12 76 +19:38:14 76 +19:38:16 76 +19:38:18 75 +19:38:20 75 +19:38:22 74 +19:38:24 74 +19:38:26 74 +19:38:28 73 +19:38:30 73 +19:38:32 72 +19:38:34 71 +19:38:36 71 +19:38:38 71 +19:38:40 70 +19:38:42 70 +19:38:44 68 +19:38:46 68 +19:38:49 68 +19:38:51 67 +19:38:53 67 +19:38:55 67 +19:38:57 66 +19:38:59 66 +19:39:01 65 +19:39:03 64 +19:39:05 64 +19:39:07 62 +19:39:09 62 +19:39:11 61 +19:39:13 61 +19:39:15 61 +19:39:17 60 +19:39:19 59 +19:39:21 59 +19:39:23 58 +19:39:26 58 +19:39:28 57 +19:39:30 57 +19:39:32 57 +19:39:34 55 +19:39:36 55 +19:39:38 55 +19:39:40 54 +19:39:42 54 +19:39:44 54 +19:39:46 52 +19:39:48 51 +19:39:50 50 +19:39:52 50 +19:39:54 51 +19:39:56 51 +19:39:58 37 +19:40:00 11 +19:40:02 11 +19:40:05 11 +19:40:07 11 +19:40:09 11 +19:40:11 11 +19:40:13 11 +19:40:15 11 +19:40:17 11 +19:40:19 11 +19:40:21 11 +19:40:23 11 +19:40:25 11 +19:40:27 11 +19:40:29 11 +19:40:31 11 +19:40:34 11 +19:40:36 11 +19:40:38 11 +19:40:40 11 +19:40:42 11 +19:40:44 11 +19:40:46 11 +19:40:48 11 +19:40:50 11 +19:40:52 11 +19:40:54 11 +19:40:56 11 +19:40:58 11 +19:41:00 11 +19:41:02 11 +19:41:05 11 +19:41:07 11 +19:41:09 11 +19:41:11 11 +19:41:13 11 +19:41:15 11 +19:41:17 11 +19:41:19 11 +19:41:21 11 +19:41:23 11 +19:41:25 11 +19:41:27 11 +19:41:29 11 +19:41:31 11 +19:41:34 11 +19:41:36 11 +19:41:38 11 +19:41:40 11 +19:41:42 11 +19:41:44 11 +19:41:46 11 +19:41:48 11 +19:41:50 11 +19:41:52 11 +19:41:54 11 +19:41:56 11 +19:41:58 11 +19:42:00 11 +19:42:02 11 +19:42:05 11 +19:42:07 11 +19:42:09 11 +19:42:11 11 +19:42:13 11 +19:42:15 11 +19:42:17 11 +19:42:19 11 +19:42:21 11 +19:42:23 11 +19:42:25 11 +19:42:27 11 +19:42:29 11 +19:42:31 11 +19:42:33 11 +19:42:36 11 +19:42:38 11 +19:42:40 11 +19:42:42 11 +19:42:44 11 +19:42:46 11 +19:42:48 11 +19:42:50 11 +19:42:52 4 +KILLED +19:42:56 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/hostwatch-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/hostwatch-rank1.log new file mode 100644 index 0000000000..73499b8799 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/hostwatch-rank1.log @@ -0,0 +1,202 @@ +19:36:03 114 +19:36:05 112 +19:36:07 112 +19:36:09 112 +19:36:11 110 +19:36:13 110 +19:36:15 109 +19:36:17 109 +19:36:19 109 +19:36:21 109 +19:36:23 107 +19:36:25 107 +19:36:27 106 +19:36:29 106 +19:36:31 106 +19:36:34 104 +19:36:36 104 +19:36:38 103 +19:36:40 103 +19:36:42 102 +19:36:44 101 +19:36:46 101 +19:36:48 100 +19:36:50 100 +19:36:52 100 +19:36:54 98 +19:36:56 98 +19:36:58 98 +19:37:00 97 +19:37:02 96 +19:37:04 96 +19:37:06 96 +19:37:08 96 +19:37:11 95 +19:37:13 95 +19:37:15 94 +19:37:17 93 +19:37:19 93 +19:37:21 91 +19:37:23 92 +19:37:25 90 +19:37:27 90 +19:37:29 89 +19:37:31 88 +19:37:33 88 +19:37:35 87 +19:37:37 87 +19:37:39 86 +19:37:41 85 +19:37:43 85 +19:37:45 84 +19:37:47 84 +19:37:50 83 +19:37:52 82 +19:37:54 82 +19:37:56 81 +19:37:58 81 +19:38:00 80 +19:38:02 80 +19:38:04 80 +19:38:06 79 +19:38:08 78 +19:38:10 78 +19:38:12 77 +19:38:14 77 +19:38:16 77 +19:38:18 76 +19:38:20 75 +19:38:22 74 +19:38:24 74 +19:38:27 74 +19:38:29 73 +19:38:31 73 +19:38:33 72 +19:38:35 72 +19:38:37 70 +19:38:39 70 +19:38:41 70 +19:38:43 68 +19:38:45 68 +19:38:47 68 +19:38:49 67 +19:38:51 67 +19:38:53 67 +19:38:55 66 +19:38:57 66 +19:38:59 64 +19:39:01 64 +19:39:04 64 +19:39:06 63 +19:39:08 63 +19:39:10 62 +19:39:12 61 +19:39:14 61 +19:39:16 60 +19:39:18 60 +19:39:20 59 +19:39:22 58 +19:39:24 58 +19:39:26 57 +19:39:28 57 +19:39:30 56 +19:39:32 56 +19:39:34 56 +19:39:36 54 +19:39:38 54 +19:39:41 54 +19:39:43 54 +19:39:45 52 +19:39:47 50 +19:39:49 50 +19:39:51 50 +19:39:53 51 +19:39:55 51 +19:39:57 50 +19:39:59 17 +19:40:01 12 +19:40:03 12 +19:40:05 12 +19:40:07 12 +19:40:09 12 +19:40:11 12 +19:40:13 12 +19:40:16 12 +19:40:18 12 +19:40:20 12 +19:40:22 12 +19:40:24 12 +19:40:26 12 +19:40:28 12 +19:40:30 12 +19:40:32 12 +19:40:34 12 +19:40:36 12 +19:40:38 12 +19:40:40 12 +19:40:42 12 +19:40:44 12 +19:40:47 12 +19:40:49 12 +19:40:51 12 +19:40:53 12 +19:40:55 12 +19:40:57 12 +19:40:59 12 +19:41:01 12 +19:41:03 12 +19:41:05 12 +19:41:07 12 +19:41:09 12 +19:41:11 12 +19:41:13 12 +19:41:15 12 +19:41:18 12 +19:41:20 12 +19:41:22 12 +19:41:24 12 +19:41:26 12 +19:41:28 12 +19:41:30 12 +19:41:32 12 +19:41:34 12 +19:41:36 12 +19:41:38 12 +19:41:40 12 +19:41:42 12 +19:41:44 12 +19:41:47 12 +19:41:49 12 +19:41:51 12 +19:41:53 12 +19:41:55 12 +19:41:57 12 +19:41:59 12 +19:42:01 12 +19:42:03 12 +19:42:05 12 +19:42:07 12 +19:42:09 12 +19:42:11 12 +19:42:13 12 +19:42:15 12 +19:42:18 12 +19:42:20 12 +19:42:22 12 +19:42:24 12 +19:42:26 12 +19:42:28 12 +19:42:30 12 +19:42:32 12 +19:42:34 12 +19:42:36 12 +19:42:38 12 +19:42:40 12 +19:42:42 12 +19:42:44 12 +19:42:47 12 +19:42:49 12 +19:42:51 12 +19:42:53 12 +19:42:55 12 +19:42:57 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/hostwatch-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/hostwatch-rank2.log new file mode 100644 index 0000000000..a807b6cb9c --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/hostwatch-rank2.log @@ -0,0 +1,201 @@ +19:36:04 112 +19:36:06 111 +19:36:08 111 +19:36:10 110 +19:36:12 110 +19:36:14 110 +19:36:16 109 +19:36:18 109 +19:36:20 107 +19:36:22 107 +19:36:24 106 +19:36:26 106 +19:36:28 106 +19:36:30 105 +19:36:33 105 +19:36:35 104 +19:36:37 103 +19:36:39 103 +19:36:41 101 +19:36:43 101 +19:36:45 100 +19:36:47 100 +19:36:49 99 +19:36:51 99 +19:36:53 99 +19:36:55 99 +19:36:57 98 +19:36:59 98 +19:37:01 97 +19:37:03 97 +19:37:05 97 +19:37:07 95 +19:37:09 95 +19:37:12 94 +19:37:14 94 +19:37:16 94 +19:37:18 92 +19:37:20 92 +19:37:22 91 +19:37:24 91 +19:37:26 90 +19:37:28 90 +19:37:30 90 +19:37:32 88 +19:37:34 89 +19:37:36 87 +19:37:38 87 +19:37:40 87 +19:37:42 86 +19:37:44 85 +19:37:46 85 +19:37:49 85 +19:37:51 84 +19:37:53 84 +19:37:55 83 +19:37:57 82 +19:37:59 81 +19:38:01 81 +19:38:03 81 +19:38:05 79 +19:38:07 79 +19:38:09 79 +19:38:11 78 +19:38:13 78 +19:38:15 77 +19:38:17 77 +19:38:19 77 +19:38:21 76 +19:38:23 76 +19:38:26 74 +19:38:28 74 +19:38:30 74 +19:38:32 73 +19:38:34 73 +19:38:36 72 +19:38:38 72 +19:38:40 72 +19:38:42 71 +19:38:44 71 +19:38:46 69 +19:38:48 69 +19:38:50 69 +19:38:52 67 +19:38:54 67 +19:38:56 66 +19:38:58 66 +19:39:01 65 +19:39:03 65 +19:39:05 64 +19:39:07 63 +19:39:09 63 +19:39:11 61 +19:39:13 62 +19:39:15 62 +19:39:17 60 +19:39:19 60 +19:39:21 59 +19:39:23 59 +19:39:25 58 +19:39:27 58 +19:39:29 58 +19:39:31 58 +19:39:33 56 +19:39:35 54 +19:39:37 54 +19:39:40 54 +19:39:42 55 +19:39:44 54 +19:39:46 41 +19:39:48 41 +19:39:50 41 +19:39:52 41 +19:39:54 41 +19:39:56 41 +19:39:58 41 +19:40:00 16 +19:40:02 16 +19:40:04 16 +19:40:06 16 +19:40:08 16 +19:40:10 16 +19:40:12 16 +19:40:15 16 +19:40:17 16 +19:40:19 16 +19:40:21 16 +19:40:23 16 +19:40:25 16 +19:40:27 16 +19:40:29 16 +19:40:31 16 +19:40:33 16 +19:40:35 16 +19:40:37 16 +19:40:39 16 +19:40:41 16 +19:40:43 16 +19:40:46 16 +19:40:48 16 +19:40:50 16 +19:40:52 16 +19:40:54 16 +19:40:56 16 +19:40:58 16 +19:41:00 17 +19:41:02 17 +19:41:04 17 +19:41:06 17 +19:41:08 17 +19:41:10 17 +19:41:12 17 +19:41:15 17 +19:41:17 17 +19:41:19 17 +19:41:21 17 +19:41:23 17 +19:41:25 17 +19:41:27 17 +19:41:29 17 +19:41:31 17 +19:41:33 17 +19:41:35 17 +19:41:37 17 +19:41:39 17 +19:41:41 17 +19:41:43 17 +19:41:45 17 +19:41:48 17 +19:41:50 17 +19:41:52 17 +19:41:54 17 +19:41:56 17 +19:41:58 17 +19:42:00 17 +19:42:02 17 +19:42:04 17 +19:42:06 17 +19:42:08 17 +19:42:10 17 +19:42:12 17 +19:42:14 17 +19:42:16 17 +19:42:19 17 +19:42:21 17 +19:42:23 17 +19:42:25 17 +19:42:27 17 +19:42:29 17 +19:42:31 17 +19:42:33 17 +19:42:35 17 +19:42:37 17 +19:42:39 17 +19:42:41 17 +19:42:43 17 +19:42:45 17 +19:42:47 17 +19:42:50 17 +19:42:52 17 +19:42:54 17 +19:42:56 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/server-rank0.log new file mode 100644 index 0000000000..fda218ebfc --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/server-rank0.log @@ -0,0 +1,31309 @@ +[kvp-serve] 2026-10-22T10:35:52+00:00 rank 0 of 3 context 1048576 parallel 8 binary tensorfold-native 00552dccd11bea13 env TF_KV_PAGES=all TF_GLM_MM=unset args +info: rank 0: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-a:4832:4832 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:4832:4832 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:4832:4832 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:4832:4832 [0] NCCL INFO cudaDriverVersion 13030 +host-a:4832:4832 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:4832:4832 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:4832:4832 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:4832:4832 [0] NCCL INFO Count not open libibverbs.so +host-a:4832:4832 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:4832:4832 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:4832:4832 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:4832:4832 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x42ef27d0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:4832:4832 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x42f33ed0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:4832:4832 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x42f755d0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:4832:4832 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x42fb6cd0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:4832:4832 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4832:4832 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4832:4832 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4832:4832 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4832:4832 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:4832:4832 [0] NCCL INFO Initialized NET plugin IB +host-a:4832:4832 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:4832:4832 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:4832:4832 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:4832:4832 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:4832:4832 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:4832:4832 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:4832:4832 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:4832:4832 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:4832:4832 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:4832:4832 [0] NCCL INFO Using network IB +host-a:4832:4832 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:4832:4832 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x402bb110 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x938dfd5667d23ca7 - Init START +host-a:4832:4832 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:4832:4832 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:4832:4832 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:4832:4832 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:4832:4832 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:4832:4832 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:4832:4832 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:4832:4832 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:4832:4832 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:4832:4832 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:4832:4832 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:4832:4832 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:4832:4832 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:4832:4832 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:4832:4832 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:4832:4832 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:4832:4832 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:4832:4832 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:4832:4832 [0] NCCL INFO Rank 0: 6 Net devices +host-a:4832:4832 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:4832:4832 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:4832:4832 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:4832:4832 [0] NCCL INFO comm 0x402bb110 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:4832:4832 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:4832:4832 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:4832:4832 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:4832:4832 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:4832:4832 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:4832:4832 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:4832:4832 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:4832:4832 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:4832:4832 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:4832:4832 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:4832:4849 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-a:4832:4849 [0] NCCL INFO proxy listening socket at <49897> +host-a:4832:4850 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-a:4832:4832 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:4832:4851 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-a:4832:4832 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:4832:4832 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:4832:4832 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:4832:4832 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:4832:4832 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:4832:4832 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:4832:4832 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea4888010000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea4888054000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea488809c000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea48880e0000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea4888128000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea488816c000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea48881b4000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea48881f8000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6695 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6696 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6695 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2c2a ctsFifoLkey=0x1d2c2a +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6695 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6696 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2c2a ctsFifoLkey=0x1d2c2a +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6696 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6663 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6664 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6663 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x191310 ctsFifoLkey=0x191310 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6663 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6664 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x191310 ctsFifoLkey=0x191310 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6664 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6697 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6698 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6697 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6697 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6698 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6698 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6665 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6666 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6665 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1baba8 ctsFifoLkey=0x1baba8 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6665 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6666 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1baba8 ctsFifoLkey=0x1baba8 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6666 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1629 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1630 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1631 pkey=0 pd=0xea488831a220 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6695 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6696 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1757 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1758 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1759 pkey=0 pd=0xea488840cf60 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6663 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6664 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6697 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6698 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6665 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6666 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1632 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1633 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1634 pkey=0 pd=0xea488831a220 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1760 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1761 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1762 pkey=0 pd=0xea488840cf60 +host-a:4832:4832 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:4832:4832 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:4832:4832 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:4832:4832 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:4832:4832 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea48885ad000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea48885ee000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea488862f000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1635 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1636 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1635 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1635 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1636 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1636 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea48886ce000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1635 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1636 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1763 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1764 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1763 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19bcbb ctsFifoLkey=0x19bcbb +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1763 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1764 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19bcbb ctsFifoLkey=0x19bcbb +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1764 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1763 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1764 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1637 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1638 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1637 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1637 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1638 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1638 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1637 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1638 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1765 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1766 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1765 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b7b4 ctsFifoLkey=0x19b7b4 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1765 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1766 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b7b4 ctsFifoLkey=0x19b7b4 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1766 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1765 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1766 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4832 [0] NCCL INFO Connected all trees +host-a:4832:4832 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:4832:4832 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:4832:4832 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:4832:4832 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea488881d000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea4888862000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea48888aa000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea48888f2000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6667 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6668 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6669 pkey=0 pd=0xea488809bf70 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6699 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6700 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6701 pkey=0 pd=0xea4888053eb0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6670 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6671 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6672 pkey=0 pd=0xea488809bf70 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6702 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6703 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6704 pkey=0 pd=0xea4888053eb0 +host-a:4832:4832 [0] NCCL INFO Connected binomial trees +host-a:4832:4832 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:4832:4832 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:4832:4832 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:4832:4832 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:4832:4832 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-a:4832:4832 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:4832:4832 [0] NCCL INFO ncclCommInitRank comm 0x402bb110 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x938dfd5667d23ca7 - Init COMPLETE +host-a:4832:4832 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 4.66 (kernels 0.25, alloc 0.04, bootstrap 4.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.34, rest 0.00) +host-a:4832:4832 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:4832:4832 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:4832:4832 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:4832:4832 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:4832:4832 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:4832:4832 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:4832:4832 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:4832:4832 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea4888b17000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea4888b58000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6673 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6674 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6673 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1afaf7 ctsFifoLkey=0x1afaf7 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6673 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6674 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1afaf7 ctsFifoLkey=0x1afaf7 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6674 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1639 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1640 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1641 pkey=0 pd=0xea488831a220 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea4888c74000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea4888cb8000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea4888d00000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea4888d44000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea4888d8c000) +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6705 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6706 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6705 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ccccc ctsFifoLkey=0x1ccccc +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6705 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6706 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ccccc ctsFifoLkey=0x1ccccc +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6706 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea4888e29000) +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6673 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6674 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6705 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6706 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1767 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1768 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1769 pkey=0 pd=0xea488840cf60 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1770 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1771 pkey=0 pd=0xea488840cf60 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1770 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19ece9 ctsFifoLkey=0x19ece9 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1770 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1771 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19ece9 ctsFifoLkey=0x19ece9 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1771 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1642 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1643 pkey=0 pd=0xea488831a220 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1642 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x1813d0 ctsFifoLkey=0x1813d0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1642 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1643 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x1813d0 ctsFifoLkey=0x1813d0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1643 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6707 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6708 pkey=0 pd=0xea4888053eb0 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6709 pkey=0 pd=0xea4888053eb0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6675 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6676 pkey=0 pd=0xea488809bf70 oooRq=0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6677 pkey=0 pd=0xea488809bf70 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1770 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1771 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4832:4849 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1642 set_ece={supportedinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 78.03 GiB; 8 streams at once, the engine's own at a 1048576-token window (12.88 GiB, in the model's), 7 more taking 1326 pages of 512 rows (8.26 GiB) as they need them, 131 MiB fixed each, of 25.2 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 16.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 1048576; loaded in 247.6s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-15ac50155a325a7600c76f728f203459 prompt=14 cached=0 thinking=False effort=none tokens=16 sha=e195338a9a9d finish=length rounds=6 accepted=9/13 ttft=0.113 decode=0.355 elapsed=0.468 queued=0.000 tok/s=42.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-aa066c44272f4324d2358d650afba02f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.111 decode=5.254 elapsed=5.365 queued=0.000 tok/s=48.5 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-0af039d52282ae7727cda035ef3e4cfc prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=108 accepted=147/202 ttft=0.109 decode=6.164 elapsed=6.273 queued=0.000 tok/s=41.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 4, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-6dd6af45a68efa4c5528bf792a1fa419 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=106 accepted=150/212 ttft=0.109 decode=6.240 elapsed=6.349 queued=0.000 tok/s=40.9 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 5, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-0a5192266b5fc008dec466f145bbada0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=105 accepted=150/218 ttft=0.109 decode=6.222 elapsed=6.330 queued=0.000 tok/s=41.0 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-a7f1fa49e57cb30c1d2d29b548719dbb prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=77 accepted=179/227 ttft=0.110 decode=4.937 elapsed=5.047 queued=0.000 tok/s=51.6 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-9f5f8893150620a8b2ce9ad49b583fe4 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=104 accepted=151/218 ttft=0.109 decode=6.232 elapsed=6.341 queued=0.000 tok/s=40.9 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 8, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-3989bbafe99e1c4f10f56d7e98782f7c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=106 accepted=149/206 ttft=0.109 decode=6.239 elapsed=6.348 queued=0.000 tok/s=40.9 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 9, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-62588fc2ab71a1f138a7f3d0e0697578 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=119 accepted=136/196 ttft=0.110 decode=6.704 elapsed=6.815 queued=0.000 tok/s=38.0 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 10, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-abaa13f6e1b545fa251539576af2e3cf prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.240 elapsed=11.343 queued=0.000 tok/s=22.7 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 11, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-4e3a71d504551a217bf2f9c90427c504 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.329 elapsed=11.431 queued=0.000 tok/s=22.5 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-da444c9a8f22ad5a993c55508ed28792 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.245 elapsed=11.347 queued=0.000 tok/s=22.7 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 13, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-18e6adeff494d4491339f66d7a2afff5 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.218 elapsed=11.320 queued=0.000 tok/s=22.7 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 14, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-fa99b48079b18e3a606d518e89d0202c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=255 accepted=0/0 ttft=0.104 decode=11.338 elapsed=11.442 queued=0.000 tok/s=22.5 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 15, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-5af59f9f4d9c235b698537d66e1bc7da prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=255 accepted=0/0 ttft=0.104 decode=11.220 elapsed=11.324 queued=0.000 tok/s=22.7 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-644fc24f0472970c3deb74a3cd4e8438 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=255 accepted=0/0 ttft=0.104 decode=11.323 elapsed=11.427 queued=0.000 tok/s=22.5 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 17, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-a9cfd9891cfc352d9df72fce29f07cae prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.240 elapsed=11.341 queued=0.000 tok/s=22.7 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 18, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-a5f5928124bb243446d10d2df26ba436 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.109 decode=5.284 elapsed=5.393 queued=0.000 tok/s=48.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 19, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-d79bb9a56bb9242c2a6485e0705b0272 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.276 decode=7.359 elapsed=7.634 queued=0.167 tok/s=34.7 seed=1234 +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-b42b2061c6e54c7ea067d22def7169d1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=96 accepted=159/273 ttft=0.111 decode=8.610 elapsed=8.721 queued=0.000 tok/s=29.6 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 21, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-bc924d5b16fa38b5858e72202bcc2b80 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/227 ttft=0.109 decode=11.219 elapsed=11.328 queued=0.000 tok/s=22.7 seed=1234 +[tensorfold] done cmpl-abbbafa8dc529caaeccbca543bcbdc4e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=89 accepted=167/264 ttft=0.277 decode=12.591 elapsed=12.868 queued=0.168 tok/s=20.3 seed=1237 +[tensorfold] done cmpl-f3da1480873b882744669f5e10f33f4a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=89 accepted=166/264 ttft=0.458 decode=12.494 elapsed=12.952 queued=0.168 tok/s=20.4 seed=1236 +info: kv pool idle: promised 0, free 1326 + store 0 == total 1326 +[tensorfold] done cmpl-ee74d7a94212540d7a27d443b05e8554 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=96 accepted=160/287 ttft=0.674 decode=12.790 elapsed=13.464 queued=0.168 tok/s=19.9 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 25, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/server-rank1.log new file mode 100644 index 0000000000..9b7327084a --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/server-rank1.log @@ -0,0 +1,5489 @@ +[kvp-serve] 2026-10-22T10:35:51+00:00 rank 1 of 3 context 1048576 parallel 8 binary tensorfold-native 00552dccd11bea13 env TF_KV_PAGES=all TF_GLM_MM=unset args +info: rank 1: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-b:4058:4058 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:4058:4058 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:4058:4058 [0] NCCL INFO cudaDriverVersion 13030 +host-b:4058:4058 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:4058:4058 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:4058:4058 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:4058:4058 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:4058:4058 [0] NCCL INFO Count not open libibverbs.so +host-b:4058:4058 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:4058:4058 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:4058:4058 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:4058:4058 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x19782420 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:4058:4058 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x197c3b20 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:4058:4058 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x19805220 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:4058:4058 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x19846920 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:4058:4058 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:4058:4058 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:4058:4058 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:4058:4058 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:4058:4058 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:4058:4058 [0] NCCL INFO Initialized NET plugin IB +host-b:4058:4058 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:4058:4058 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:4058:4058 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:4058:4058 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:4058:4058 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:4058:4058 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:4058:4058 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:4058:4058 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:4058:4058 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:4058:4058 [0] NCCL INFO Using network IB +host-b:4058:4058 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:4058:4058 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x16b4ad90 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x938dfd5667d23ca7 - Init START +host-b:4058:4058 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:4058:4058 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:4058:4058 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:4058:4058 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:4058:4058 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:4058:4058 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:4058:4058 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:4058:4058 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:4058:4058 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:4058:4058 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:4058:4058 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:4058:4058 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:4058:4058 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:4058:4058 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:4058:4058 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:4058:4058 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:4058:4058 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:4058:4058 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:4058:4058 [0] NCCL INFO Rank 1: 6 Net devices +host-b:4058:4058 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:4058:4058 [0] NCCL INFO comm 0x16b4ad90 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:4058:4058 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:4058:4058 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:4058:4058 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:4058:4058 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:4058:4058 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:4058:4058 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:4058:4074 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-b:4058:4073 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-b:4058:4073 [0] NCCL INFO proxy listening socket at <41949> +host-b:4058:4075 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-b:4058:4058 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:4058:4058 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:4058:4058 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:4058:4058 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:4058:4058 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:4058:4058 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:4058:4058 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:4058:4058 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60c010000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60c054000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60c09c000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60c0e0000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60c128000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1885 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1886 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1885 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1885 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1886 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1886 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60c1d1000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60c219000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60c25d000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6469 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6470 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6471 pkey=0 pd=0xe3a60c1afde0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6437 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6438 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6439 pkey=0 pd=0xe3a60c2bfa80 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6472 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6473 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6474 pkey=0 pd=0xe3a60c1afde0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1885 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1886 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6440 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6441 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6442 pkey=0 pd=0xe3a60c2bfa80 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2013 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2014 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2013 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2013 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2014 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2014 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1887 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1888 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1887 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b4f4c ctsFifoLkey=0x1b4f4c +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1887 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1888 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b4f4c ctsFifoLkey=0x1b4f4c +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1888 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2015 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2016 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2015 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1deee4 ctsFifoLkey=0x1deee4 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2015 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2016 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1deee4 ctsFifoLkey=0x1deee4 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2016 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2013 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2014 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1887 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1888 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2015 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2016 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4058 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:4058:4058 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:4058:4058 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:4058:4058 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:4058:4058 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60c5b7000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60c5fc000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60c641000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60c686000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1889 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1890 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1891 pkey=0 pd=0xe3a60c053c70 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2017 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2018 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2019 pkey=0 pd=0xe3a60c317d50 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1892 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1893 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1894 pkey=0 pd=0xe3a60c053c70 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2020 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2021 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2022 pkey=0 pd=0xe3a60c317d50 +host-b:4058:4058 [0] NCCL INFO Connected all trees +host-b:4058:4058 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:4058:4058 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:4058:4058 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:4058:4058 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60c8a7000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60c8e8000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6443 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6444 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6443 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x16bd77 ctsFifoLkey=0x16bd77 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6443 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6444 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x16bd77 ctsFifoLkey=0x16bd77 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6444 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6475 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6476 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6475 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x18c0be ctsFifoLkey=0x18c0be +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6475 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6476 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x18c0be ctsFifoLkey=0x18c0be +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6476 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60c9df000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60ca23000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6443 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6444 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6475 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6476 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6445 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6446 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6445 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x164401 ctsFifoLkey=0x164401 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6445 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6446 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x164401 ctsFifoLkey=0x164401 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6446 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6477 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6478 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6477 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19e6e5 ctsFifoLkey=0x19e6e5 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6477 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6478 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19e6e5 ctsFifoLkey=0x19e6e5 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6478 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6445 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6446 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6477 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6478 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4058 [0] NCCL INFO Connected binomial trees +host-b:4058:4058 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:4058:4058 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:4058:4058 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:4058:4058 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:4058:4058 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-b:4058:4058 [0] NCCL INFO ncclCommInitRank comm 0x16b4ad90 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x938dfd5667d23ca7 - Init COMPLETE +host-b:4058:4058 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 0.66 (kernels 0.24, alloc 0.05, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.34, rest 0.00) +host-b:4058:4058 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:4058:4058 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:4058:4058 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:4058:4058 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:4058:4058 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:4058:4058 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:4058:4058 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:4058:4058 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60cb19000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60cb5a000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60cb9f000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1895 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1896 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1895 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1895 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1896 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1896 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6447 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6448 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6449 pkey=0 pd=0xe3a60c2bfa80 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2023 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2024 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2023 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de8e0 ctsFifoLkey=0x1de8e0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2023 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2024 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de8e0 ctsFifoLkey=0x1de8e0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2024 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60cd13000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1895 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1896 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6479 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6480 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6481 pkey=0 pd=0xe3a60c1afde0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60cdd2000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60ce16000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2023 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2024 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3a60ce5b000) +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3a60ce9c000) +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2025 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2026 pkey=0 pd=0xe3a60c317d50 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2027 pkey=0 pd=0xe3a60c317d50 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1897 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1898 pkey=0 pd=0xe3a60c053c70 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1899 pkey=0 pd=0xe3a60c053c70 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6482 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6483 pkey=0 pd=0xe3a60c1afde0 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6482 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19160b ctsFifoLkey=0x19160b +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6482 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6483 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19160b ctsFifoLkey=0x19160b +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6483 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6450 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6451 pkey=0 pd=0xe3a60c2bfa80 oooRq=0 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6450 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x178d4b ctsFifoLkey=0x178d4b +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6450 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6451 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x178d4b ctsFifoLkey=0x178d4b +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6451 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6482 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6483 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4058:4073 [0] NCCL INFO info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +error: rank 1: another rank's process ended (its line closed); this rank stops instead of waiting on it diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/server-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/server-rank2.log new file mode 100644 index 0000000000..d4ec516a1d --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/found-b7fecb3d/r6all/server-rank2.log @@ -0,0 +1,3016 @@ +[kvp-serve] 2026-10-22T10:35:50+00:00 rank 2 of 3 context 1048576 parallel 8 binary tensorfold-native 00552dccd11bea13 env TF_KV_PAGES=all TF_GLM_MM=unset args +info: rank 2: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-c:4022:4022 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:4022:4022 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:4022:4022 [0] NCCL INFO cudaDriverVersion 13030 +host-c:4022:4022 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:4022:4022 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:4022:4022 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:4022:4022 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:4022:4022 [0] NCCL INFO Count not open libibverbs.so +host-c:4022:4022 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:4022:4022 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:4022:4022 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:4022:4022 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x36ade450 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:4022:4022 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x36b1fb50 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:4022:4022 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x36b61250 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:4022:4022 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x36ba2950 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:4022:4022 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:4022:4022 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:4022:4022 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:4022:4022 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:4022:4022 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:4022:4022 [0] NCCL INFO Initialized NET plugin IB +host-c:4022:4022 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:4022:4022 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:4022:4022 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:4022:4022 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:4022:4022 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:4022:4022 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:4022:4022 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:4022:4022 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:4022:4022 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:4022:4022 [0] NCCL INFO Using network IB +host-c:4022:4022 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:4022:4022 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x33ea6dd0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x938dfd5667d23ca7 - Init START +host-c:4022:4022 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:4022:4022 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:4022:4022 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:4022:4022 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:4022:4022 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:4022:4022 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:4022:4022 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:4022:4022 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:4022:4022 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:4022:4022 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:4022:4022 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:4022:4022 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:4022:4022 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:4022:4022 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:4022:4022 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:4022:4022 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:4022:4022 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:4022:4022 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:4022:4022 [0] NCCL INFO Rank 2: 6 Net devices +host-c:4022:4022 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:4022:4022 [0] NCCL INFO comm 0x33ea6dd0 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:4022:4022 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:4022:4022 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:4022:4022 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:4022:4022 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:4022:4022 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:4022:4022 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:4022:4038 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-c:4022:4037 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-c:4022:4037 [0] NCCL INFO proxy listening socket at <48003> +host-c:4022:4022 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:4022:4039 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-c:4022:4022 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:4022:4022 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:4022:4022 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:4022:4022 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:4022:4022 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:4022:4022 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:4022:4022 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe35838010000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe35838054000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3583809c000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1885 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1886 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1885 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1885 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1886 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1886 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1629 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1630 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1631 pkey=0 pd=0xe358380f2420 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2013 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2014 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2013 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2013 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2014 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2014 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe3583822e000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe35838276000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe358382ba000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe35838302000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe35838346000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1885 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1886 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2013 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2014 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1757 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1758 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1759 pkey=0 pd=0xe358380df4c0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1887 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1888 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1887 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bb0ad ctsFifoLkey=0x1bb0ad +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1887 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1888 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bb0ad ctsFifoLkey=0x1bb0ad +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1888 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1632 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1633 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1634 pkey=0 pd=0xe358380f2420 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2015 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2016 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2015 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1dcecf ctsFifoLkey=0x1dcecf +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2015 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2016 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1dcecf ctsFifoLkey=0x1dcecf +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2016 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1760 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1761 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1762 pkey=0 pd=0xe358380df4c0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1887 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1888 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2015 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2016 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4022 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:4022:4022 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:4022:4022 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:4022:4022 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:4022:4022 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:4022:4022 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:4022:4022 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:4022:4022 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:4022:4022 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe358385b5000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe358385f6000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe3583863b000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1635 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1636 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1635 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1635 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1636 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1636 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1889 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1890 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1891 pkey=0 pd=0xe35838053590 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1763 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1764 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1763 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1763 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1764 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1764 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe358387a6000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1635 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1636 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1763 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1764 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2017 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2018 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2019 pkey=0 pd=0xe358380f2f90 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe35838860000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe358388a4000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe358388ec000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1637 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1638 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1637 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1808c6 ctsFifoLkey=0x1808c6 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1637 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1638 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1808c6 ctsFifoLkey=0x1808c6 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1638 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1892 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1893 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1894 pkey=0 pd=0xe35838053590 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1765 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1766 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1765 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197a79 ctsFifoLkey=0x197a79 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1765 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1766 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197a79 ctsFifoLkey=0x197a79 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1766 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe35838a5c000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1637 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1638 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1765 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1766 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2020 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2021 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2022 pkey=0 pd=0xe358380f2f90 +host-c:4022:4022 [0] NCCL INFO Connected all trees +host-c:4022:4022 [0] NCCL INFO Connected binomial trees +host-c:4022:4022 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:4022:4022 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:4022:4022 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:4022:4022 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:4022:4022 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-c:4022:4022 [0] NCCL INFO ncclCommInitRank comm 0x33ea6dd0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x938dfd5667d23ca7 - Init COMPLETE +host-c:4022:4022 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.62 (kernels 0.25, alloc 0.04, bootstrap 0.02, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.29, rest 0.00) +host-c:4022:4022 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:4022:4022 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:4022:4022 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:4022:4022 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:4022:4022 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:4022:4022 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:4022:4022 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:4022:4022 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe35838b16000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe35838b57000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe35838ba0000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe35838be4000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe35838c2c000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe35838c70000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1895 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=1896 pkey=0 pd=0xe35838053590 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1895 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1b9795 ctsFifoLkey=0x1b9795 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1895 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 1896 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1b9795 ctsFifoLkey=0x1b9795 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 1896 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe35838d13000) +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe35838d57000) +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1639 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1640 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1641 pkey=0 pd=0xe358380f2420 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1895 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 1896 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2023 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2024 pkey=0 pd=0xe358380f2f90 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2023 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1da99f ctsFifoLkey=0x1da99f +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2023 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2024 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1da99f ctsFifoLkey=0x1da99f +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2024 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1767 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1768 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1769 pkey=0 pd=0xe358380df4c0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1770 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1771 pkey=0 pd=0xe358380df4c0 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1770 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19928f ctsFifoLkey=0x19928f +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1770 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1771 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19928f ctsFifoLkey=0x19928f +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1771 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4022:4037 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1642 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1643 pkey=0 pd=0xe358380f2420 oooRq=0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1642 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1813d0 ctsFifoLkey=0x1813d0 +host-c:4022:4037 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1642 query_ece={supporinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +error: rank 2: another rank's process ended (its line closed); this rank stops instead of waiting on it diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/ring.patch b/docs/receipts/glm-kv-pages-0b094b74/tp3/ring.patch new file mode 100644 index 0000000000..4215f54329 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/ring.patch @@ -0,0 +1,521 @@ +From 0c3560e979834faf0da4ea49bd5a526053944aab Mon Sep 17 00:00:00 2001 +From: +Date: Fri, 16 Oct 2026 15:35:38 +0900 +Subject: [PATCH 1/2] core/heat: with TF_HEAT_CEILING a chunk also waits while + its expected rise would pass the ceiling; GLM reads it and marks each prompt + pass's start + +--- + RUNBOOK.md | 2 + + zig/glm_prompt_host.zig | 2 +- + zig/src/core/heat.zig | 143 +++++++++++++++++++++----- + zig/src/families/glm/cuda_engine.zig | 8 +- + zig/src/families/glm/cuda_prompts.zig | 2 +- + zig/src/families/glm/cuda_ranks.zig | 29 +++++- + 6 files changed, 154 insertions(+), 32 deletions(-) + +diff --git a/RUNBOOK.md b/RUNBOOK.md +index 101faf4c3..44f6494b2 100644 +--- a/RUNBOOK.md ++++ b/RUNBOOK.md +@@ -168,6 +168,8 @@ The server logs `heat_wait_s`, a cancel ends the wait, and the reply's tokens do + On three GB10s serving GLM-5.3-Flash NVFP4 at TP=3, a 1,035,295-token prompt without the bands kept one host's hottest + zone at 94 C or more for most of the prompt (97.7 C at the top); `TF_HEAT_HIGH=92` and `TF_HEAT_LOW=88` held all three + at 93.0 C or under, and the first token came 10% later. ++For GLM, `TF_HEAT_CEILING` (above the high band, with both) also makes a chunk wait while the hottest zone plus the ++last rise of a chunk run straight after another would pass it, until that rise fits under it and the low band holds. + Archives whose names contain `host-only` are CPU verification artifacts and cannot serve CUDA inference. + + ### GB10 CPU pinning for long CUDA runs +diff --git a/zig/glm_prompt_host.zig b/zig/glm_prompt_host.zig +index 1c094d175..3d0709f8f 100644 +--- a/zig/glm_prompt_host.zig ++++ b/zig/glm_prompt_host.zig +@@ -85,7 +85,7 @@ const engine = struct { + if (slot >= 32) return error.BadSlot; + e.cur = &e.seqs[slot]; + } +- pub fn gate(_: *Engine, _: ?u8) !void {} ++ pub fn gate(_: *Engine, _: ?u8, _: bool) !void {} + pub fn exchange(_: *Engine) u8 { + return 0; + } +diff --git a/zig/src/core/heat.zig b/zig/src/core/heat.zig +index eec72a634..19dc487c8 100644 +--- a/zig/src/core/heat.zig ++++ b/zig/src/core/heat.zig +@@ -1,10 +1,11 @@ +-//! Opt-in wait before a prompt chunk when the hottest thermal zone is above a band. ++//! Opt-in wait before a prompt chunk when the hottest thermal zone is above a band or would rise past a ceiling. + const std = @import("std"); + const platform = @import("sys"); + + pub const default_root = "/sys/class/thermal"; + +-pub const Bands = struct { high_c: f64, low_c: f64 }; ++/// Degrees. `ceiling_c` also waits for a chunk's rise; a family that sets it calls `Gate.startPrompt` at each prompt. ++pub const Bands = struct { high_c: f64, low_c: f64, ceiling_c: ?f64 = null }; + + /// Asked after each pause: true ends the wait with error.Cancelled (the request was cancelled meanwhile). + pub const Stop = struct { ptr: *anyopaque, check: *const fn (*anyopaque) bool }; +@@ -15,6 +16,12 @@ pub const Gate = struct { + /// Set after the first unreadable zone, so later chunks skip the read and the log. + no_zone: bool = false, + root: []const u8 = default_root, ++ /// The gathered maximum the last chunk began at; null: the next chunk is a prompt's first. ++ began: ?f64 = null, ++ /// The same when that chunk began straight after another; null: after a rest (a wait, or the gap before a prompt). ++ after: ?f64 = null, ++ /// The last rise of a chunk begun straight after another, in degrees: what the ceiling expects of the next. ++ loaded: f64 = 0, + + /// TF_HEAT_HIGH and TF_HEAT_LOW in degrees, both or neither (neither: no read); TF_HEAT_ROOT moves the zones. + pub fn fromEnv() !Gate { +@@ -22,26 +29,55 @@ pub const Gate = struct { + return .{ .bands = try bandsFrom(envSpan("TF_HEAT_HIGH"), envSpan("TF_HEAT_LOW")), .root = root }; + } + ++ /// A prompt begins: its first chunk has no rise, and it begins after the gap before the prompt. ++ pub fn startPrompt(g: *Gate) void { ++ g.began = null; ++ g.after = null; ++ } ++ + /// `read` returns the gathered maximum in celsius. `sleep_fn` stands in for the 2 s pause. + pub fn waitUntil(g: *Gate, read: *const fn (*anyopaque) anyerror!f64, sleep_fn: *const fn (*anyopaque) void, ctx: *anyopaque, stop: ?Stop) !void { + const bands = g.bands orelse return; + if (g.no_zone) return; +- var waiting = false; +- while (true) { +- const max_c = read(ctx) catch |err| { +- if (err == error.NoZone) { +- g.no_zone = true; +- noteNoZone(g.root); +- return; +- } +- return err; +- }; +- if (allow(bands, max_c, waiting)) return; +- waiting = true; ++ var max_c = (try g.reading(read, ctx)) orelse return; ++ const floor = g.floorFor(bands, max_c) orelse return; ++ while (max_c > floor) { + sleep_fn(ctx); + g.waited_s += 2; + if (stop) |s| if (s.check(s.ptr)) return error.Cancelled; ++ max_c = (try g.reading(read, ctx)) orelse return; + } ++ g.began = max_c; // the next chunk begins after a rest ++ g.after = null; ++ } ++ ++ /// One gathered reading; null once no zone is readable (logged once, and the chunk runs). ++ fn reading(g: *Gate, read: *const fn (*anyopaque) anyerror!f64, ctx: *anyopaque) !?f64 { ++ return read(ctx) catch |err| { ++ if (err != error.NoZone) return err; ++ g.no_zone = true; ++ noteNoZone(g.root); ++ return null; ++ }; ++ } ++ ++ /// Before a chunk at `top`: null to run now, else the reading its wait lasts until (low, or lower to fit a rise). ++ fn floorFor(g: *Gate, b: Bands, top: f64) ?f64 { ++ const inf = std.math.inf(f64); ++ const rise: f64 = if (g.began) |at| (if (at == -inf) 0 else @max(0, top - at)) else 0; ++ if (g.after) |at| { ++ if (at != -inf and top != -inf) g.loaded = rise; // the last chunk began straight after another ++ } ++ const ahead: f64 = if (g.began == null) 0 else g.loaded; ++ const ceiling = b.ceiling_c orelse inf; ++ if (top <= b.high_c and top + ahead <= ceiling) { ++ g.after = if (g.began == null) null else top; ++ g.began = top; ++ return null; ++ } ++ const floor = @min(b.low_c, ceiling - @max(rise, ahead)); ++ noteWait(b, top, ahead, floor); ++ return floor; + } + + /// The real pause, on this process's own hottest zone (`gatheredMax` takes several readings), until `stop` says so. +@@ -76,6 +112,16 @@ pub fn bandsFrom(high: ?[]const u8, low: ?[]const u8) !?Bands { + return bands; + } + ++/// TF_HEAT_CEILING's text on `bands`: in degrees, above the high band, and only with the bands. ++pub fn ceilingFrom(bands: ?Bands, text: ?[]const u8) !?Bands { ++ const t = text orelse return bands; ++ var b = bands orelse return error.HeatBands; ++ const c = try degrees(t); ++ if (c <= b.high_c) return error.HeatBands; ++ b.ceiling_c = c; ++ return b; ++} ++ + pub fn degrees(text: []const u8) !f64 { + return std.fmt.parseFloat(f64, std.mem.trim(u8, text, " \t\r\n")); + } +@@ -86,12 +132,6 @@ pub fn zoneCelsius(text: []const u8) !f64 { + return @as(f64, @floatFromInt(milli)) / 1000.0; + } + +-/// Run when idle and at or under high, or when already waiting and at or under low. +-pub fn allow(bands: Bands, max_c: f64, waiting: bool) bool { +- if (waiting) return max_c <= bands.low_c; +- return max_c <= bands.high_c; +-} +- + pub fn gatheredMax(xs: []const f64) f64 { + var m = xs[0]; + for (xs[1..]) |x| m = @max(m, x); +@@ -120,6 +160,16 @@ pub fn note(seconds_: f64) void { + std.log.info("heat_wait_s {d:.1}", .{seconds_}); + } + ++/// Why a chunk waits, at the wait's start: above the high band, or its expected rise past the ceiling. ++fn noteWait(b: Bands, top: f64, ahead: f64, floor: f64) void { ++ if (@import("builtin").is_test) return; ++ if (top > b.high_c) { ++ std.log.info("heat: hottest zone {d:.1} C above {d} C; the prefill waits for {d:.1} C", .{ top, b.high_c, floor }); ++ } else { ++ std.log.info("heat: hottest zone {d:.1} C plus a chunk's {d:.1} C rise, one run straight after another, above {d} C; the prefill waits for {d:.1} C", .{ top, ahead, b.ceiling_c orelse 0, floor }); ++ } ++} ++ + /// Test builds count the warning. A real process logs it. + var no_zone_notes: u32 = 0; + +@@ -149,10 +199,9 @@ test "bands parse as degrees and a lone value is refused" { + + test "the chunk runs under the high band, and a hot reading waits until the low band" { + const bands = Bands{ .high_c = 92, .low_c = 88 }; +- try std.testing.expect(allow(bands, 92, false)); +- try std.testing.expect(!allow(bands, 92.5, false)); +- try std.testing.expect(!allow(bands, 90, true)); +- try std.testing.expect(allow(bands, 88, true)); ++ var at = Gate{ .bands = bands }; ++ try std.testing.expectEqual(@as(?f64, null), at.floorFor(bands, 92)); // at the high band the chunk runs ++ try std.testing.expectEqual(@as(?f64, 88), at.floorFor(bands, 92.5)); // above it, a wait until the low band + try std.testing.expectEqual(@as(f64, 94), gatheredMax(&.{ 80, 94, 91 })); + const Script = struct { + temps: []const f64, +@@ -195,6 +244,52 @@ test "the chunk runs under the high band, and a hot reading waits until the low + try std.testing.expectEqual(@as(u32, 1), stuck.sleeps); // a cancel ends the wait after the pause it came in + } + ++/// A gate's gathered maxima, one a read (the last repeats), and the pauses taken. ++const Readings = struct { ++ temps: []const f64, ++ i: usize = 0, ++ sleeps: u32 = 0, ++ fn read(ptr: *anyopaque) !f64 { ++ const r: *Readings = @ptrCast(@alignCast(ptr)); ++ const t = r.temps[r.i]; ++ if (r.i + 1 < r.temps.len) r.i += 1; ++ return t; ++ } ++ fn sleep(ptr: *anyopaque) void { ++ @as(*Readings, @ptrCast(@alignCast(ptr))).sleeps += 1; ++ } ++ /// One chunk's gate over `temps`: the pauses it took. ++ fn chunk(g: *Gate, temps: []const f64) !u32 { ++ var r = Readings{ .temps = temps }; ++ try g.waitUntil(read, sleep, &r, null); ++ return r.sleeps; ++ } ++}; ++ ++test "a ceiling comes with both bands and above the high one" { ++ const b = Bands{ .high_c = 92, .low_c = 88 }; ++ try std.testing.expect((try ceilingFrom(null, null)) == null); ++ try std.testing.expectEqual(b, (try ceilingFrom(b, null)).?); ++ try std.testing.expectEqual(@as(?f64, 93), (try ceilingFrom(b, " 93\n")).?.ceiling_c); ++ try std.testing.expectError(error.HeatBands, ceilingFrom(null, "93")); ++ try std.testing.expectError(error.HeatBands, ceilingFrom(b, "92")); ++} ++ ++test "with a ceiling, a chunk begun straight after another waits until the last such rise fits under it" { ++ var g = Gate{ .bands = .{ .high_c = 92, .low_c = 88, .ceiling_c = 93 } }; ++ g.startPrompt(); ++ try std.testing.expectEqual(@as(u32, 0), try Readings.chunk(&g, &.{70})); // a prompt's first chunk: no rise ++ try std.testing.expectEqual(@as(u32, 0), try Readings.chunk(&g, &.{80})); // chunk 1 began after the gap: not kept ++ try std.testing.expectEqual(@as(u32, 2), try Readings.chunk(&g, &.{ 90, 86, 83 })); // 90 + 10 > 93: wait to 83 ++ try std.testing.expectEqual(@as(u32, 2), try Readings.chunk(&g, &.{ 91, 84, 83 })); // its own 8 C, but 10 C counts ++ try std.testing.expectEqual(@as(f64, 8), g.waited_s); ++ g.startPrompt(); ++ try std.testing.expectEqual(@as(u32, 0), try Readings.chunk(&g, &.{ 90, 70 })); // a new prompt's first takes none ++ var bands = Gate{ .bands = .{ .high_c = 92, .low_c = 88 } }; // the same readings with the bands alone ++ bands.startPrompt(); ++ for ([_]f64{ 70, 80, 90 }) |t| try std.testing.expectEqual(@as(u32, 0), try Readings.chunk(&bands, &.{t})); ++} ++ + const env = if (@import("builtin").os.tag == .windows) @import("sys").c else struct { + extern "c" fn setenv(name: [*:0]const u8, value: [*:0]const u8, overwrite: i32) i32; + extern "c" fn unsetenv(name: [*:0]const u8) i32; +diff --git a/zig/src/families/glm/cuda_engine.zig b/zig/src/families/glm/cuda_engine.zig +index e56b02469..a615c9b6b 100644 +--- a/zig/src/families/glm/cuda_engine.zig ++++ b/zig/src/families/glm/cuda_engine.zig +@@ -79,7 +79,7 @@ pub const Engine = struct { + slot: u32 = 0, + capacity: usize, + mtp: bool, +- heat_gate: heat.Gate, // upstream's bands (off unless set), read before each prompt chunk on the gathered maximum ++ heat_gate: heat.Gate, // upstream's bands and ceiling (off unless set), read before each prompt chunk on the gathered maximum + shared_buf: cuda.DeviceBuffer, // one f64 every rank all-reduces to its maximum + snaps: std.AutoHashMapUnmanaged(u64, Snap) = .empty, // kept prompt states by id (the same ids on every rank) + snap_pages: ?snapshots.pages.Pool = null, +@@ -304,6 +304,7 @@ pub const Engine = struct { + /// decode.prefill: `ids` in 2,048-row chunks, then the first token from the last row; `from` restores a kept state. + pub fn prefill(e: *Engine, ids: []const u32, from: ?Resume, cancel: ?Cancel, marks: ?Marks) !u32 { + var start = try e.prefillStart(ids.len, from); ++ e.heat_gate.startPrompt(); // its chunks get no `from`: a resumed pass's first is marked here + while (true) : (start += fw.prefill_rows) { + if (try e.prefillChunk(ids[start..@min(ids.len, start + fw.prefill_rows + 1)], start, ids.len, null, cancel, marks)) |first| return first; + } +@@ -341,7 +342,7 @@ pub const Engine = struct { + p.prompt.?.invalid = true; + }; + var stamp = prof_mod.span(f.prof, e.stream, .gate); +- try e.gate(cancel); ++ try e.gate(cancel, start == 0 or from != null); + stamp.end(); + if (f.prof) |p| p.mark(e.stream, .gate); + try f.run(as_i32[0..rows]); +@@ -384,9 +385,10 @@ pub const Engine = struct { + } + + /// The chunk gate on every rank: a kept state at the mark, the heat wait, rank 0's cancel; the id exchanged first. +- pub fn gate(e: *Engine, cancel: ?Cancel) !void { ++ pub fn gate(e: *Engine, cancel: ?Cancel, first: bool) !void { + try snapshots.gate(e); + if (e.heat_gate.bands != null) { ++ if (first) e.heat_gate.startPrompt(); // a pass's first chunk carries no rise in + var w: Wait = .{ .e = e, .cancel = cancel }; + const before = e.heat_gate.waited_s; + e.heat_gate.waitUntil(Wait.read, Wait.sleep, &w, .{ .ptr = &w, .check = Wait.stop }) catch |err| return w.failed orelse err; +diff --git a/zig/src/families/glm/cuda_prompts.zig b/zig/src/families/glm/cuda_prompts.zig +index b66392f44..da12cc079 100644 +--- a/zig/src/families/glm/cuda_prompts.zig ++++ b/zig/src/families/glm/cuda_prompts.zig +@@ -36,7 +36,7 @@ pub fn pass(e: anytype, parts: []const engine.Prompt, firsts: []u32, comptime Fo + @memcpy(ids[rows..][0..p.ids.len], p.ids); + rows += p.ids.len; + } +- try e.gate(null); ++ try e.gate(null, true); // fresh prompts at position 0: the pass carries no rise in + var one: [1]Rank = .{.{ .w = &e.w, .st = segs[0].st, .b = &e.bp, .segs = segs[0..parts.len] }}; + var f: Forward = .{ .d = e.d, .m = &e.m, .stream = e.stream, .ranks = &one, .exchange = e.exchange(), .pieces = &e.pieces, .split = e.settings.split, .ring = e.settings.ring, .kda_wide = e.settings.kda_wide, .taps = if (e.drafter) |dr| dr.taps(true) else null }; + try f.run(@ptrCast(ids[0..rows])); +diff --git a/zig/src/families/glm/cuda_ranks.zig b/zig/src/families/glm/cuda_ranks.zig +index 04bd50ad5..ead8f4572 100644 +--- a/zig/src/families/glm/cuda_ranks.zig ++++ b/zig/src/families/glm/cuda_ranks.zig +@@ -14,7 +14,7 @@ pub const Settings = struct { + ring: bool = false, // TF_GLM_PREFILL_REDUCE=ring: split, its exchanges passed along the ring (cuda_relay) + kda_wide: bool = true, // TF_GLM_KDA_DECODE_WIDE: 1 (decode windows on KDA's three-kernel chain) or 0 + pieces: u32 = 3, // TF_GLM_OVERLAP_PIECES: a prompt chunk's exchanges in this many row pieces (1: whole). +- heat: ?heat.Bands = null, // TF_HEAT_HIGH and TF_HEAT_LOW (upstream's names): a prompt chunk waits on every rank's hottest zone ++ heat: ?heat.Bands = null, // TF_HEAT_HIGH, TF_HEAT_LOW (and TF_HEAT_CEILING): a prompt chunk waits on every rank's hottest zone + layers: u32 = 0, // TF_GLM_LAYERS: the first this many layers only, a thin model for one-GPU checks (0: all) + dense_bits: u8 = 16, // TF_GLM_DENSE_BITS: the dense linears packed at load to 4, 6 or 8 bits (16: bf16) + head_bits: u8 = 16, // TF_GLM_HEAD_BITS: the head, likewise +@@ -77,10 +77,14 @@ pub const Settings = struct { + }, + }; + }; +- s.heat = heat.bandsFrom(get("TF_HEAT_HIGH"), get("TF_HEAT_LOW")) catch { ++ const bands = heat.bandsFrom(get("TF_HEAT_HIGH"), get("TF_HEAT_LOW")) catch { + problem.* = "TF_HEAT_HIGH and TF_HEAT_LOW: both or neither, in degrees, with low at or under high"; + return error.BadSetting; + }; ++ s.heat = heat.ceilingFrom(bands, get("TF_HEAT_CEILING")) catch { ++ problem.* = "TF_HEAT_CEILING: degrees above TF_HEAT_HIGH, with both bands"; ++ return error.BadSetting; ++ }; + if (s.prompt_fixed_k and !s.tiled) { + problem.* = "TF_GLM_PROMPT_K=shape requires TF_GLM_MM=1"; + return error.BadSetting; +@@ -119,6 +123,7 @@ pub const Fingerprint = extern struct { + capacity: u64, // tokens a sequence holds + heat_high: f64, // the heat bands in degrees (0 and 0: off): every rank must wait alike + heat_low: f64, ++ heat_ceiling: f64, // TF_HEAT_CEILING (0: none) + config: [32]u8, // sha256 of config.json + program: [32]u8, // sha256 of this executable + checkpoint: [64]u8, // hex digest of the checkpoint's metadata and every shard byte, through the digest cache +@@ -144,6 +149,7 @@ pub const Fingerprint = extern struct { + capacity, + heat_high, + heat_low, ++ heat_ceiling, + config, + program, + checkpoint, +@@ -174,6 +180,7 @@ pub const Fingerprint = extern struct { + .capacity = capacity, + .heat_high = if (s.heat) |b| b.high_c else 0, + .heat_low = if (s.heat) |b| b.low_c else 0, ++ .heat_ceiling = if (s.heat) |b| b.ceiling_c orelse 0 else 0, + .config = config, + .program = program, + .checkpoint = std.fmt.bytesToHex(checkpoint_digest, .lower), +@@ -208,6 +215,7 @@ pub fn mismatch(all: []const Fingerprint) ?Mismatch { + "capacity", + "heat_high", + "heat_low", ++ "heat_ceiling", + "config", + "program", + "checkpoint", +@@ -315,6 +323,7 @@ fn explain(f: Fingerprint.Field) []const u8 { + .capacity => "the sequence capacity (the context)", + .heat_high => "TF_HEAT_HIGH", + .heat_low => "TF_HEAT_LOW", ++ .heat_ceiling => "TF_HEAT_CEILING", + .config => "config.json", + .program => "the executable", + .checkpoint => "checkpoint metadata or shard bytes", +@@ -326,6 +335,7 @@ fn fake(name: [:0]const u8) ?[]const u8 { + if (std.mem.eql(u8, name, "TF_GLM_OVERLAP_PIECES")) return "2"; + if (std.mem.eql(u8, name, "TF_HEAT_HIGH")) return "92"; + if (std.mem.eql(u8, name, "TF_HEAT_LOW")) return "88"; ++ if (std.mem.eql(u8, name, "TF_HEAT_CEILING")) return "93"; + if (std.mem.eql(u8, name, "TF_GLM_DENSE_BITS")) return "4"; + if (std.mem.eql(u8, name, "TF_GLM_HEAD_BITS")) return "8"; + return null; +@@ -341,6 +351,11 @@ fn bad(name: [:0]const u8) ?[]const u8 { + return null; + } + ++fn lowCeiling(name: [:0]const u8) ?[]const u8 { ++ if (std.mem.eql(u8, name, "TF_HEAT_CEILING")) return "92"; ++ return fake(name); ++} ++ + fn typo(name: [:0]const u8) ?[]const u8 { + if (std.mem.eql(u8, name, "TF_GLM_DENSE_BITS")) return "abc"; + return null; +@@ -351,8 +366,11 @@ test "settings from the environment: the defaults, the choices, and a refusal na + const s = try Settings.read(&fake, &problem); + try std.testing.expect(!s.split and s.kda_wide and s.pieces == 2 and s.heat.?.high_c == 92 and s.heat.?.low_c == 88); + try std.testing.expect(s.dense_bits == 4 and s.head_bits == 8); ++ try std.testing.expectEqual(@as(?f64, 93), s.heat.?.ceiling_c); + try std.testing.expectError(error.BadSetting, Settings.read(&bad, &problem)); + try std.testing.expectEqualStrings("TF_GLM_PREFILL_REDUCE: split, ring or gather", problem); ++ try std.testing.expectError(error.BadSetting, Settings.read(&lowCeiling, &problem)); ++ try std.testing.expectEqualStrings("TF_HEAT_CEILING: degrees above TF_HEAT_HIGH, with both bands", problem); + const r = try Settings.read(&ringed, &problem); + try std.testing.expect(r.split and r.ring); + try std.testing.expectEqual(@as(u32, 2), Fingerprint.of(2, 0, r, 8192, true, @splat(0), @splat(0), hashBytes("/c")).split); +@@ -378,6 +396,11 @@ test "fingerprints: agreeing ranks pass; the first differing field and rank are + const heat_on: Settings = .{ .heat = .{ .high_c = 92, .low_c = 88 } }; + all[1] = Fingerprint.of(2, 1, heat_on, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); + try std.testing.expectEqual(Fingerprint.Field.heat_high, mismatch(&all).?.field); ++ const ceiling_on: Settings = .{ .heat = .{ .high_c = 92, .low_c = 88, .ceiling_c = 93 } }; ++ all[0] = Fingerprint.of(2, 0, heat_on, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); ++ all[1] = Fingerprint.of(2, 1, ceiling_on, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); ++ try std.testing.expectEqual(Fingerprint.Field.heat_ceiling, mismatch(&all).?.field); ++ all[0] = Fingerprint.of(2, 0, .{}, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); + all[1] = Fingerprint.of(2, 1, .{}, 8192, true, h0, h0, hashBytes("/hub/snapshots/other")); + try std.testing.expectEqual(Fingerprint.Field.checkpoint, mismatch(&all).?.field); + all[1] = Fingerprint.of(2, 1, .{}, 8192, false, h0, h0, hashBytes("/hub/snapshots/423acf37")); // --no-drafts on rank 1 only +@@ -501,7 +524,7 @@ test "every media limit is part of the ranks' agreement" { + test "every scheduling and identity field that differs stops the ranks" { + const zero = Fingerprint.of(2, 0, .{}, 8192, true, @splat(0), @splat(0), hashBytes("/same")); + const one = Fingerprint.of(2, 1, .{}, 8192, true, @splat(0), @splat(0), hashBytes("/same")); +- inline for (.{ "layout", "world", "rank", "split", "kda_wide", "pieces", "layers", "mtp", "vision", "video_tokens", "max_images", "max_videos", "max_frames", "capacity", "heat_high", "heat_low" }) |name| { ++ inline for (.{ "layout", "world", "rank", "split", "kda_wide", "pieces", "layers", "mtp", "vision", "video_tokens", "max_images", "max_videos", "max_frames", "capacity", "heat_high", "heat_low", "heat_ceiling" }) |name| { + var all = [_]Fingerprint{ zero, one }; + @field(all[1], name) += 1; + try std.testing.expectEqual(@field(Fingerprint.Field, name), mismatch(&all).?.field); +-- +2.56.0.windows.2 + + +From 4c8cd52b05546e2ecca3558848323130e10d2620 Mon Sep 17 00:00:00 2001 +From: +Date: Mon, 19 Oct 2026 15:19:38 +0900 +Subject: [PATCH 2/2] glm: TF_GLM_NCCL_ID_FILE hands rank 0's NCCL id to the + others in a file, and NCCL_COMM_ID then names only the ranks' lines (local, + not for upstream: TP=3 on the ring's /32 until #636) + +1.1.0's tp_watch reads NCCL_COMM_ID for the ranks' lines, and NCCL's bootstrap needs an interface in that address's +subnet, which a /32 ring has only on rank 0. With TF_GLM_NCCL_ID_FILE set, each rank logs the NCCL_COMM_ID it takes +out of its environment after the lines are up and before NCCL's first call; rank 0 writes its unique id to the file +as hex and the others read it there (5b6aa233's form), waiting up to 10 minutes. +--- + zig/src/families/glm/cuda_native.zig | 37 +++++++++++++++++++++++++--- + 1 file changed, 34 insertions(+), 3 deletions(-) + +diff --git a/zig/src/families/glm/cuda_native.zig b/zig/src/families/glm/cuda_native.zig +index e840131d5..72bb047e2 100644 +--- a/zig/src/families/glm/cuda_native.zig ++++ b/zig/src/families/glm/cuda_native.zig +@@ -124,6 +124,36 @@ const Owned = struct { + }; + + extern "c" fn setenv(name: [*:0]const u8, value: [*:0]const u8, overwrite: c_int) c_int; ++extern "c" fn unsetenv(name: [*:0]const u8) c_int; ++ ++/// Local, not for upstream (TP=3 on the ring's /32 until #636): with TF_GLM_NCCL_ID_FILE, NCCL_COMM_ID names only the ++/// ranks' lines; it leaves NCCL's environment, whose bootstrap looks for an interface in that address's subnet. ++fn commIdByFile(rank: usize) ?[]const u8 { ++ const f = std.mem.span(std.c.getenv("TF_GLM_NCCL_ID_FILE") orelse return null); ++ const was = std.mem.span(std.c.getenv("NCCL_COMM_ID") orelse ""); ++ std.log.info("rank {d}: TF_GLM_NCCL_ID_FILE {s}; NCCL_COMM_ID {s} leaves NCCL's environment", .{ rank, f, was }); ++ _ = unsetenv("NCCL_COMM_ID"); ++ return f; ++} ++ ++/// Local: rank 0 writes its unique id to `path` as hex; the others read it there, waiting up to 10 minutes. ++fn idByFile(io: std.Io, path: []const u8, rank: usize, lib: *const cuda.nccl.Library, id: *cuda.nccl.UniqueId) !void { ++ if (rank == 0) { ++ try lib.check(lib.api.ncclGetUniqueId(id), "ncclGetUniqueId"); ++ const hex = std.fmt.bytesToHex(id.internal, .lower); ++ return std.Io.Dir.cwd().writeFile(io, .{ .sub_path = path, .data = &hex }); ++ } ++ var hex: [256]u8 = undefined; ++ for (0..600) |_| { ++ const got = std.Io.Dir.cwd().readFile(io, path, &hex) catch hex[0..0]; ++ if (got.len == hex.len) { ++ _ = try std.fmt.hexToBytes(&id.internal, &hex); ++ return; ++ } ++ std.Io.sleep(io, .fromMilliseconds(1000), .awake) catch {}; ++ } ++ return error.NoCommId; ++} + + /// Every peer's send/recv connected now, before the weights fill memory (connected on first use, ibv_reg_mr can fail). + fn warmPeers(d: *const cuda.Driver, g: *const cuda.tp_group.Group) !void { +@@ -178,14 +208,15 @@ pub fn open(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: [ + set.kv8 = o.kv8; + // the ranks' lines before NCCL (cuda/tp_watch.zig): a rank that fails, refuses or dies stops every rank + if (p.world > 1) try cuda.tp_watch.start(io, p.rank, p.world); +- return openRank(gpa, io, ctx, dir, o, p, set, roce_bytes) catch |err| { ++ const id_file = commIdByFile(p.rank); // local: after the lines read NCCL_COMM_ID, before NCCL's first call ++ return openRank(gpa, io, ctx, dir, o, p, set, roce_bytes, id_file) catch |err| { + cuda.tp_watch.fail(@errorName(err)); // the other ranks stop with this rank's error + return err; + }; + } + + /// `open` once the ranks' lines are up: NCCL, the engine and the link, then a follower's loop or rank 0's lanes. +-fn openRank(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: []const u8, o: Options, p: Place, set: ranks.Settings, roce_bytes: usize) !Loaded { ++fn openRank(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: []const u8, o: Options, p: Place, set: ranks.Settings, roce_bytes: usize, id_file: ?[]const u8) !Loaded { + const own = try gpa.create(Owned); + errdefer gpa.destroy(own); + own.gpa = gpa; +@@ -197,7 +228,7 @@ fn openRank(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: [ + var lib = try cuda.nccl.Library.open(); + errdefer lib.close(); + var id: cuda.nccl.UniqueId = undefined; +- try lib.check(lib.api.ncclGetUniqueId(&id), "ncclGetUniqueId"); // NCCL_COMM_ID: the same id on every rank ++ if (id_file) |f| try idByFile(io, f, p.rank, &lib, &id) else try lib.check(lib.api.ncclGetUniqueId(&id), "ncclGetUniqueId"); // NCCL_COMM_ID: the same id on every rank + var comm: cuda.nccl.Comm = null; + try lib.check(lib.api.ncclCommInitRank(&comm, @intCast(p.world), id, @intCast(p.rank)), "ncclCommInitRank"); + own.group = .{ .lib = lib, .comm = comm, .rank = p.rank, .world = p.world }; +-- +2.56.0.windows.2 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/admit.json b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/admit.json new file mode 100644 index 0000000000..893204125b --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/admit.json @@ -0,0 +1 @@ +{"alone": [{"prompt": "p65536-0", "seed": 1000, "sent": 725525.873895511, "first": 725569.111682234, "last": 725572.880225269, "tokens": 128, "pieces": [[725569.111682234, 3], [725569.18736717, 9], [725569.240071063, 14], [725569.293657356, 10], [725569.348828321, 17], [725569.411392141, 10], [725569.471272078, 4], [725569.525866058, 5], [725569.582748617, 4], [725569.64462383, 5], [725569.705861727, 10], [725569.765902398, 9], [725569.828153888, 5], [725569.890956375, 5], [725569.961390878, 11], [725570.030619194, 7], [725570.096454266, 11], [725570.164598544, 35], [725570.23007355, 8], [725570.287987496, 7], [725570.341193783, 10], [725570.395342033, 4], [725570.449326749, 17], [725570.502734096, 16], [725570.557804978, 9], [725570.608542862, 3], [725570.661940719, 10], [725570.714742323, 13], [725570.767860145, 7], [725570.820871948, 14], [725570.875887604, 9], [725570.929286053, 11], [725570.995237689, 5], [725571.055313149, 5], [725571.107705319, 9], [725571.160643076, 16], [725571.212698864, 17], [725571.264796388, 23], [725571.318666954, 4], [725571.374849126, 3], [725571.436429771, 4], [725571.497533209, 5], [725571.558027144, 6], [725571.620711063, 9], [725571.690668676, 11], [725571.757104208, 16], [725571.82409496, 5], [725571.880865367, 6], [725571.942033041, 11], [725572.002584299, 12], [725572.063825618, 8], [725572.127585045, 5], [725572.196576498, 3], [725572.266767018, 8], [725572.331739257, 3], [725572.385332523, 2], [725572.437267711, 5], [725572.491141637, 8], [725572.544816807, 6], [725572.598616447, 9], [725572.651949538, 13], [725572.708659549, 14], [725572.770571982, 9], [725572.829233037, 9], [725572.880225269, 5]], "completed": 725572.880303779, "ttft_s": 43.23778672295157, "decode_tps": 33.70002646106575, "prompt_tokens": 65536, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "d8a8421597ee", "token_sha256": "d8a8421597ee17ad5b7801dacb29f07eb6692a742fb9fc92104dd0e7527a4a3d", "output_sha256": "0dfe13857bf300b4a865cec4f36e215954dd3fa63c9e610b0d86d3613508c25e", "elapsed_s": 47.006408268003725, "chunk_offsets_s": [43.23778672295157, 43.31347165897023, 43.36617555201519, 43.41976184491068, 43.4749328099424, 43.53749662998598, 43.59737656696234, 43.65197054692544, 43.70885310601443, 43.770728318952024, 43.831966215977445, 43.89200688700657, 43.9542583769653, 44.017060864018276, 44.087495366926305, 44.15672368300147, 44.22255875496194, 44.29070303298067, 44.3561780389864, 44.41409198497422, 44.4672982719494, 44.52144652197603, 44.57543123792857, 44.62883858499117, 44.68390946695581, 44.73464735096786, 44.788045207969844, 44.84084681200329, 44.89396463392768, 44.94697643700056, 45.001992092933506, 45.055390542023815, 45.12134217796847, 45.1814176379703, 45.2338098079199, 45.28674756491091, 45.33880335302092, 45.39090087695513, 45.44477144291159, 45.50095361494459, 45.562534259981476, 45.62363769800868, 45.68413163302466, 45.74681555200368, 45.816773165017366, 45.883208696963266, 45.950199448969215, 46.00696985598188, 46.068137530004606, 46.12868878792506, 46.18993010697886, 46.25368953391444, 46.32268098695204, 46.392871506977826, 46.45784374594223, 46.51143701199908, 46.56337220000569, 46.617246125941165, 46.67092129599769, 46.72472093591932, 46.778054027003236, 46.83476403797977, 46.896676470991224, 46.95533752592746, 47.006329757976346], "chunk_gaps_ms": {"count": 64, "p50": 57.39825253840536, "p95": 69.95761301368475, "max": 75.68493601866066}}, {"prompt": "p65536-1", "seed": 1001, "sent": 725572.881364773, "first": 725616.127008745, "last": 725619.243815066, "tokens": 128, "pieces": [[725616.127008745, 2], [725616.208367923, 11], [725616.267771226, 14], [725616.328376346, 6], [725616.397662082, 20], [725616.466583561, 18], [725616.534478677, 18], [725616.600899765, 28], [725616.666583311, 2], [725616.726251349, 14], [725616.785118457, 16], [725616.844324107, 7], [725616.897702138, 17], [725616.963776795, 10], [725617.019369997, 4], [725617.080362899, 9], [725617.141177897, 8], [725617.204986271, 9], [725617.266928978, 11], [725617.331288394, 6], [725617.401252141, 12], [725617.470245519, 12], [725617.540937942, 11], [725617.609424972, 12], [725617.67971425, 21], [725617.7501167, 7], [725617.819319028, 10], [725617.888875293, 11], [725617.95757945, 18], [725618.025988563, 15], [725618.092546651, 11], [725618.153811121, 10], [725618.21344664, 6], [725618.267198882, 6], [725618.322108966, 10], [725618.384110385, 20], [725618.441812432, 20], [725618.503415507, 7], [725618.562346192, 6], [725618.614145135, 18], [725618.668367206, 7], [725618.728474857, 16], [725618.789397841, 15], [725618.849677987, 12], [725618.911532468, 13], [725618.974381609, 2], [725619.029451642, 7], [725619.088228268, 10], [725619.142724132, 9], [725619.193990194, 1], [725619.243815066, 6]], "completed": 725619.243987994, "ttft_s": 43.24564397195354, "decode_tps": 40.746837282074466, "prompt_tokens": 65536, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "bf47482e16cb", "token_sha256": "bf47482e16cb91a3294931e7a597aa8938db10afebffe6941e0abe5a20c9d02d", "output_sha256": "506ecff1415d0489be9cdf8709f2dace158282371918e04ed030e81d69827fa7", "elapsed_s": 46.36262322100811, "chunk_offsets_s": [43.24564397195354, 43.32700315001421, 43.3864064529771, 43.44701157300733, 43.51629730896093, 43.585218788008206, 43.65311390394345, 43.71953499200754, 43.78521853801794, 43.84488657594193, 43.90375368401874, 43.962959334021434, 44.01633736502845, 44.08241202193312, 44.13800522394013, 44.1989981259685, 44.259813123964705, 44.32362149795517, 44.3855642050039, 44.449923621024936, 44.51988736796193, 44.58888074592687, 44.65957316895947, 44.72806019894779, 44.798349477001466, 44.86875192692969, 44.93795425503049, 45.00751051993575, 45.076214676955715, 45.144623789936304, 45.21118187799584, 45.27244634798262, 45.33208186691627, 45.3858341089217, 45.44074419292156, 45.50274561194237, 45.560447658994235, 45.622050734004006, 45.68098141893279, 45.73278036201373, 45.78700243297499, 45.84711008402519, 45.9080330679426, 45.96831321401987, 46.03016769501846, 46.09301683597732, 46.14808686892502, 46.20686349493917, 46.2613593590213, 46.31262542097829, 46.36245029291604], "chunk_gaps_ms": {"count": 50, "p50": 61.433772498276085, "p95": 70.40244992822409, "max": 81.35917806066573}}, {"prompt": "p65536-2", "seed": 1002, "sent": 725619.244652002, "first": 725662.55173984, "last": 725665.80738046, "tokens": 128, "pieces": [[725662.55173984, 4], [725662.632778853, 7], [725662.687031036, 7], [725662.739266006, 14], [725662.791995897, 9], [725662.844449834, 16], [725662.898839482, 10], [725662.95206055, 8], [725663.005920038, 4], [725663.059491089, 2], [725663.11291485, 19], [725663.16611161, 10], [725663.219653136, 11], [725663.273251472, 9], [725663.327733872, 4], [725663.383286801, 9], [725663.44577364, 5], [725663.506861507, 12], [725663.569503302, 12], [725663.629665513, 15], [725663.689945842, 16], [725663.7538635, 3], [725663.808168636, 13], [725663.869979417, 14], [725663.92876085, 5], [725663.982345983, 5], [725664.034892161, 14], [725664.090356498, 16], [725664.149396906, 25], [725664.212888935, 10], [725664.271301662, 17], [725664.335016803, 5], [725664.405612484, 9], [725664.470306682, 14], [725664.538768649, 14], [725664.602970342, 4], [725664.657797683, 9], [725664.719696351, 9], [725664.778411052, 17], [725664.839815343, 8], [725664.903289176, 18], [725664.972472954, 23], [725665.034533107, 5], [725665.08797439, 9], [725665.140271545, 8], [725665.194250445, 10], [725665.246922244, 15], [725665.301411364, 17], [725665.361979886, 17], [725665.423066423, 9], [725665.486468866, 14], [725665.547267495, 9], [725665.63181181, 28], [725665.698298875, 19], [725665.756916805, 14], [725665.80738046, 1]], "completed": 725665.807453993, "ttft_s": 43.30708783795126, "decode_tps": 39.00921963535294, "prompt_tokens": 65536, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "30f96d96d0b7", "token_sha256": "30f96d96d0b71ddd72f2ccfbcb05bce868c47cc84c22cbf27d7b26c6a33cc946", "output_sha256": "cb508766a2f4dc5642532a8d981a8d3b8e900dc060b072caa7c1c3e94292cc37", "elapsed_s": 46.56280199100729, "chunk_offsets_s": [43.30708783795126, 43.38812685094308, 43.4423790340079, 43.49461400404107, 43.5473438949557, 43.59979783196468, 43.65418747998774, 43.707408547983505, 43.76126803597435, 43.814839086961, 43.868262848001905, 43.92145960801281, 43.97500113397837, 44.0285994700389, 44.083081869990565, 44.138634799048305, 44.201121637946926, 44.26220950495917, 44.32485129998531, 44.38501351105515, 44.445293840020895, 44.509211497963406, 44.56351663405076, 44.625327414949425, 44.684108847985044, 44.73769398103468, 44.790240159025416, 44.845704496023245, 44.904744904022664, 44.96823693299666, 45.02664965996519, 45.09036480099894, 45.16096048196778, 45.22565468004905, 45.29411664698273, 45.35831834003329, 45.413145681028254, 45.47504434897564, 45.533759049954824, 45.59516334102955, 45.65863717405591, 45.727820951957256, 45.789881104952656, 45.84332238801289, 45.895619543036446, 45.94959844299592, 46.00227024196647, 46.05675936199259, 46.11732788395602, 46.17841442103963, 46.2418168639997, 46.30261549295392, 46.38715980795678, 46.45364687300753, 46.51226480305195, 46.56272845796775], "chunk_gaps_ms": {"count": 55, "p50": 58.714700979180634, "p95": 70.59568096883595, "max": 84.54431500285864}}], "together": [{"prompt": "p65536-0", "seed": 1000, "sent": 725665.809405914, "first": 725667.252070173, "last": 725671.024119124, "tokens": 128, "pieces": [[725667.252070173, 3], [725667.326736444, 9], [725667.379314144, 14], [725667.43397995, 10], [725667.490032726, 17], [725667.551049328, 10], [725667.611186475, 4], [725667.664895516, 5], [725667.72086114, 4], [725667.783262761, 5], [725667.844150364, 10], [725667.905463916, 9], [725667.9669129, 5], [725668.02874717, 5], [725668.099796178, 11], [725668.168407466, 7], [725668.236230851, 11], [725668.304705473, 35], [725668.37025763, 8], [725668.428165893, 7], [725668.481175815, 10], [725668.535928099, 4], [725668.589812867, 17], [725668.642969407, 16], [725668.698276624, 9], [725668.748563717, 3], [725668.801587513, 10], [725668.855382024, 13], [725668.909533128, 7], [725668.962765458, 14], [725669.016806222, 9], [725669.069997264, 11], [725669.136115027, 5], [725669.194583952, 5], [725669.249412522, 9], [725669.301813821, 16], [725669.354507269, 17], [725669.406990374, 23], [725669.460690019, 4], [725669.515217735, 3], [725669.576866705, 4], [725669.638048215, 5], [725669.699170227, 6], [725669.76245855, 9], [725669.831591898, 11], [725669.899509986, 16], [725669.966172023, 5], [725670.022752814, 6], [725670.084194803, 11], [725670.145072498, 12], [725670.20644998, 8], [725670.270718464, 5], [725670.338753357, 3], [725670.409745023, 8], [725670.47396177, 3], [725670.527089551, 2], [725670.580627342, 5], [725670.634476822, 8], [725670.688382712, 6], [725670.743353514, 9], [725670.79655795, 13], [725670.852467018, 14], [725670.914829173, 9], [725670.973886874, 9], [725671.024119124, 5]], "completed": 725671.024230648, "ttft_s": 1.442664258996956, "decode_tps": 33.668704104698115, "prompt_tokens": 65536, "cached_tokens": 63488, "cache_state": "reused", "complete": true, "token_sha": "d8a8421597ee", "token_sha256": "d8a8421597ee17ad5b7801dacb29f07eb6692a742fb9fc92104dd0e7527a4a3d", "output_sha256": "0dfe13857bf300b4a865cec4f36e215954dd3fa63c9e610b0d86d3613508c25e", "elapsed_s": 5.2148247339064255, "chunk_offsets_s": [1.442664258996956, 1.5173305299831554, 1.5699082299834117, 1.6245740359881893, 1.6806268119253218, 1.741643413901329, 1.8017805609852076, 1.8554896019631997, 1.9114552260143682, 1.973856846918352, 2.0347444500075653, 2.0960580019745976, 2.157506985939108, 2.2193412559572607, 2.290390264010057, 2.3590015519876033, 2.426824936992489, 2.495299558970146, 2.5608517159707844, 2.6187599789118394, 2.6717699009459466, 2.72652218490839, 2.7804069529520348, 2.8335634929826483, 2.8888707099249586, 2.9391578029608354, 2.9921815989073366, 3.0459761099191383, 3.1001272139837965, 3.1533595439977944, 3.2074003079906106, 3.2605913499137387, 3.3267091129673645, 3.385178037919104, 3.440006607910618, 3.492407906916924, 3.5451013549463823, 3.5975844599306583, 3.6512841049116105, 3.705811820924282, 3.767460790928453, 3.828642300912179, 3.8897643130039796, 3.9530526360031217, 4.0221859839512035, 4.090104072005488, 4.156766108935699, 4.2133468999527395, 4.2747888889862224, 4.335666583967395, 4.397044065990485, 4.46131255000364, 4.529347442905419, 4.600339108961634, 4.664555855910294, 4.717683636932634, 4.77122142794542, 4.825070907943882, 4.878976797917858, 4.933947599958628, 4.987152035930194, 5.04306110390462, 5.105423258966766, 5.164480959996581, 5.214713210007176], "chunk_gaps_ms": {"count": 64, "p50": 57.244526979047805, "p95": 69.13334794808179, "max": 74.66627098619938}}, {"prompt": "p65536-1", "seed": 1001, "sent": 725665.811314324, "first": 725672.47396915, "last": 725675.595184399, "tokens": 128, "pieces": [[725672.47396915, 2], [725672.554393545, 11], [725672.616020301, 14], [725672.676875669, 6], [725672.745778941, 20], [725672.815277072, 18], [725672.882764039, 18], [725672.950284037, 28], [725673.016416312, 2], [725673.07644001, 14], [725673.135509744, 16], [725673.193616824, 7], [725673.247191532, 17], [725673.314126719, 10], [725673.369645587, 4], [725673.431386506, 9], [725673.492355815, 8], [725673.555566488, 9], [725673.618084061, 11], [725673.682348877, 6], [725673.752818367, 12], [725673.822393054, 12], [725673.892529172, 11], [725673.961928372, 12], [725674.030724855, 21], [725674.101354502, 7], [725674.170314727, 10], [725674.239190232, 11], [725674.30852486, 18], [725674.375969562, 15], [725674.441676542, 11], [725674.503116271, 10], [725674.56272258, 6], [725674.616332749, 6], [725674.671868003, 10], [725674.733721886, 20], [725674.791882174, 20], [725674.853581502, 7], [725674.912669221, 6], [725674.964722133, 18], [725675.019866996, 7], [725675.080188091, 16], [725675.140814136, 15], [725675.201384332, 12], [725675.263061751, 13], [725675.325904053, 2], [725675.380992025, 7], [725675.439419464, 10], [725675.494236843, 9], [725675.544865097, 1], [725675.595184399, 6]], "completed": 725675.595335226, "ttft_s": 6.662654825951904, "decode_tps": 40.6892796125309, "prompt_tokens": 65536, "cached_tokens": 63488, "cache_state": "reused", "complete": true, "token_sha": "bf47482e16cb", "token_sha256": "bf47482e16cb91a3294931e7a597aa8938db10afebffe6941e0abe5a20c9d02d", "output_sha256": "506ecff1415d0489be9cdf8709f2dace158282371918e04ed030e81d69827fa7", "elapsed_s": 9.784020902006887, "chunk_offsets_s": [6.662654825951904, 6.743079220992513, 6.804705976974219, 6.8655613450100645, 6.9344646169338375, 7.003962747985497, 7.071449714945629, 7.1389697130071, 7.205101987929083, 7.265125685953535, 7.32419542002026, 7.382302499958314, 7.4358772080158815, 7.502812394988723, 7.558331263018772, 7.620072181918658, 7.681041490985081, 7.744252163916826, 7.806769736926071, 7.871034552925266, 7.941504042944871, 8.011078729992732, 8.08121484797448, 8.150614048005082, 8.21941053098999, 8.290040177991614, 8.359000402968377, 8.427875907975249, 8.497210535919294, 8.56465523794759, 8.630362218013033, 8.691801946959458, 8.75140825600829, 8.805018424987793, 8.86055367894005, 8.922407561913133, 8.980567849939689, 9.042267177952453, 9.101354897022247, 9.153407808975317, 9.208552671945654, 9.268873766995966, 9.329499811981805, 9.390070007997565, 9.451747426996008, 9.514589729020372, 9.569677700987086, 9.628105139941908, 9.682922518928535, 9.733550773002207, 9.78387007501442], "chunk_gaps_ms": {"count": 50, "p50": 61.68837350560352, "p95": 70.46949001960456, "max": 80.42439504060894}}, {"prompt": "p65536-2", "seed": 1002, "sent": 725665.812635683, "first": 725677.041819082, "last": 725680.304344195, "tokens": 128, "pieces": [[725677.041819082, 4], [725677.120300592, 7], [725677.173386954, 7], [725677.227866543, 14], [725677.2809976, 9], [725677.333603618, 16], [725677.387328526, 10], [725677.44067327, 8], [725677.495059177, 4], [725677.548117885, 2], [725677.602412592, 19], [725677.655862474, 10], [725677.70997206, 11], [725677.764831061, 9], [725677.81925968, 4], [725677.875050465, 9], [725677.93754467, 5], [725677.999314184, 12], [725678.062651885, 12], [725678.122764091, 15], [725678.183069708, 16], [725678.246855474, 3], [725678.301933026, 13], [725678.362978344, 14], [725678.421900447, 5], [725678.47611469, 5], [725678.529388012, 14], [725678.583919753, 16], [725678.642165093, 25], [725678.704572897, 10], [725678.763511931, 17], [725678.827788633, 5], [725678.898044207, 9], [725678.963098649, 14], [725679.032294224, 14], [725679.096109931, 4], [725679.149838247, 9], [725679.211793505, 9], [725679.270693379, 17], [725679.332983129, 8], [725679.395382499, 18], [725679.466163864, 23], [725679.528476434, 5], [725679.581777584, 9], [725679.634725886, 8], [725679.688598259, 10], [725679.742101366, 15], [725679.796822565, 17], [725679.857864073, 17], [725679.918591413, 9], [725679.98264665, 14], [725680.043856301, 9], [725680.128030533, 28], [725680.193986177, 19], [725680.25291636, 14], [725680.304344195, 1]], "completed": 725680.304434515, "ttft_s": 11.229183399002068, "decode_tps": 38.92690342561823, "prompt_tokens": 65536, "cached_tokens": 63488, "cache_state": "reused", "complete": true, "token_sha": "30f96d96d0b7", "token_sha256": "30f96d96d0b71ddd72f2ccfbcb05bce868c47cc84c22cbf27d7b26c6a33cc946", "output_sha256": "cb508766a2f4dc5642532a8d981a8d3b8e900dc060b072caa7c1c3e94292cc37", "elapsed_s": 14.491798831964843, "chunk_offsets_s": [11.229183399002068, 11.3076649090508, 11.360751271015033, 11.415230860002339, 11.46836191706825, 11.520967934979126, 11.574692843016237, 11.628037587041035, 11.68242349405773, 11.735482202027924, 11.789776909048669, 11.843226791010238, 11.897336377063766, 11.952195378020406, 12.006623997003771, 12.062414782005362, 12.124908986967057, 12.186678500962444, 12.250016201986, 12.3101284080185, 12.370434024953283, 12.434219790971838, 12.489297343068756, 12.550342660979368, 12.609264764003456, 12.663479006965645, 12.716752329026349, 12.771284069982357, 12.829529409995303, 12.891937214066274, 12.950876248069108, 13.015152949956246, 13.085408524028026, 13.15046296606306, 13.219658541027457, 13.283474248019047, 13.33720256399829, 13.399157822015695, 13.458057695999742, 13.520347445970401, 13.582746815984137, 13.653528181021102, 13.715840750955977, 13.769141901051626, 13.822090203058906, 13.87596257601399, 13.929465683060698, 13.984186882036738, 14.045228390023112, 14.105955729959533, 14.170010967063718, 14.231220618006773, 14.315394849982113, 14.381350493989885, 14.440280677052215, 14.491708512068726], "chunk_gaps_ms": {"count": 55, "p50": 58.922103024087846, "p95": 70.78136503696442, "max": 84.17423197533935}}], "rows": [{"name": "p65536-0", "prompt": 65536, "sent": 0.0017439790535718203, "first": 1.4444082380505279, "last": 5.216457189060748, "error": null, "equal_alone": true, "started_after": []}, {"name": "p65536-1", "prompt": 65536, "sent": 0.0036523890448734164, "first": 6.6663072149967775, "last": 9.787522464059293, "error": null, "equal_alone": true, "started_after": ["p65536-0"]}, {"name": "p65536-2", "prompt": 65536, "sent": 0.004973748000338674, "first": 11.234157147002406, "last": 14.496682260069065, "error": null, "equal_alone": true, "started_after": ["p65536-0", "p65536-1"]}]} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/client.log new file mode 100644 index 0000000000..e50cd293f8 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/client.log @@ -0,0 +1,10 @@ +server up 2026-10-23 03:00:11 +fit: 206 pages; 65536-token prompts take 129 each +alone p65536-0: prompt 65536 ttft 43.23778672295157 sha d8a8421597ee17ad5b7801dacb29f07eb6692a742fb9fc92104dd0e7527a4a3d +alone p65536-1: prompt 65536 ttft 43.24564397195354 sha bf47482e16cb91a3294931e7a597aa8938db10afebffe6941e0abe5a20c9d02d +alone p65536-2: prompt 65536 ttft 43.30708783795126 sha 30f96d96d0b71ddd72f2ccfbcb05bce868c47cc84c22cbf27d7b26c6a33cc946 + p65536-0: prompt 65536 first 1.44 s last 5.22 s equal_alone True started after - + p65536-1: prompt 65536 first 6.67 s last 9.79 s equal_alone True started after ['p65536-0'] + p65536-2: prompt 65536 first 11.23 s last 14.50 s equal_alone True started after ['p65536-0', 'p65536-1'] +together == alone: 3/3; 2 request(s) started only after another ended +=== kvp-client r6admit rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/hostwatch-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/hostwatch-rank0.log new file mode 100644 index 0000000000..d85e89b141 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/hostwatch-rank0.log @@ -0,0 +1,203 @@ +02:56:07 114 +02:56:09 114 +02:56:11 111 +02:56:13 111 +02:56:15 110 +02:56:17 110 +02:56:19 110 +02:56:21 108 +02:56:23 108 +02:56:26 108 +02:56:28 107 +02:56:30 107 +02:56:32 105 +02:56:34 105 +02:56:36 104 +02:56:38 104 +02:56:40 104 +02:56:42 104 +02:56:44 103 +02:56:46 103 +02:56:48 101 +02:56:50 101 +02:56:52 100 +02:56:54 100 +02:56:56 100 +02:56:58 100 +02:57:00 98 +02:57:03 98 +02:57:05 97 +02:57:07 97 +02:57:09 97 +02:57:11 95 +02:57:13 95 +02:57:15 94 +02:57:17 94 +02:57:19 93 +02:57:21 92 +02:57:23 93 +02:57:25 91 +02:57:27 91 +02:57:29 89 +02:57:31 89 +02:57:33 89 +02:57:35 88 +02:57:37 88 +02:57:39 87 +02:57:42 87 +02:57:44 87 +02:57:46 85 +02:57:48 85 +02:57:50 84 +02:57:52 84 +02:57:54 84 +02:57:56 84 +02:57:58 82 +02:58:00 82 +02:58:02 81 +02:58:04 81 +02:58:06 81 +02:58:08 80 +02:58:10 80 +02:58:12 79 +02:58:14 78 +02:58:16 78 +02:58:19 77 +02:58:21 77 +02:58:23 76 +02:58:25 75 +02:58:27 75 +02:58:29 74 +02:58:31 74 +02:58:33 72 +02:58:35 72 +02:58:37 72 +02:58:39 72 +02:58:41 71 +02:58:43 71 +02:58:45 70 +02:58:47 70 +02:58:49 69 +02:58:51 68 +02:58:53 68 +02:58:55 67 +02:58:58 67 +02:59:00 65 +02:59:02 65 +02:59:04 65 +02:59:06 64 +02:59:08 64 +02:59:10 64 +02:59:12 63 +02:59:14 63 +02:59:16 63 +02:59:18 61 +02:59:20 61 +02:59:22 60 +02:59:24 60 +02:59:26 60 +02:59:28 59 +02:59:30 58 +02:59:32 58 +02:59:35 58 +02:59:37 57 +02:59:39 57 +02:59:41 56 +02:59:43 55 +02:59:45 55 +02:59:47 54 +02:59:49 54 +02:59:51 53 +02:59:53 54 +02:59:55 50 +02:59:57 50 +02:59:59 50 +03:00:01 50 +03:00:03 50 +03:00:05 50 +03:00:07 39 +03:00:09 11 +03:00:12 11 +03:00:14 11 +03:00:16 11 +03:00:18 12 +03:00:20 12 +03:00:22 12 +03:00:24 12 +03:00:26 12 +03:00:28 12 +03:00:30 12 +03:00:32 12 +03:00:34 12 +03:00:36 12 +03:00:38 12 +03:00:40 12 +03:00:42 12 +03:00:44 12 +03:00:46 12 +03:00:49 12 +03:00:51 12 +03:00:53 12 +03:00:55 12 +03:00:57 12 +03:00:59 12 +03:01:01 12 +03:01:03 12 +03:01:05 12 +03:01:07 12 +03:01:09 12 +03:01:11 12 +03:01:13 12 +03:01:15 12 +03:01:17 12 +03:01:19 12 +03:01:22 12 +03:01:24 12 +03:01:26 12 +03:01:28 12 +03:01:30 12 +03:01:32 12 +03:01:34 12 +03:01:36 12 +03:01:38 12 +03:01:40 12 +03:01:42 12 +03:01:44 12 +03:01:46 12 +03:01:48 12 +03:01:50 12 +03:01:53 12 +03:01:55 12 +03:01:57 12 +03:01:59 12 +03:02:01 12 +03:02:03 12 +03:02:05 12 +03:02:07 12 +03:02:09 12 +03:02:11 12 +03:02:13 12 +03:02:15 12 +03:02:17 12 +03:02:19 12 +03:02:21 12 +03:02:23 12 +03:02:25 12 +03:02:27 12 +03:02:30 12 +03:02:32 12 +03:02:34 12 +03:02:36 12 +03:02:38 12 +03:02:40 12 +03:02:42 12 +03:02:44 12 +03:02:46 12 +03:02:48 12 +03:02:50 12 +03:02:52 12 +03:02:54 12 +03:02:56 12 +03:02:58 12 +03:03:00 72 +03:03:02 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/hostwatch-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/hostwatch-rank1.log new file mode 100644 index 0000000000..33002051a4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/hostwatch-rank1.log @@ -0,0 +1,202 @@ +02:56:07 114 +02:56:09 114 +02:56:11 112 +02:56:13 112 +02:56:16 111 +02:56:18 110 +02:56:20 110 +02:56:22 108 +02:56:24 108 +02:56:26 108 +02:56:28 107 +02:56:30 107 +02:56:32 106 +02:56:34 105 +02:56:36 105 +02:56:38 103 +02:56:40 103 +02:56:42 103 +02:56:44 102 +02:56:46 102 +02:56:48 101 +02:56:50 101 +02:56:53 101 +02:56:55 99 +02:56:57 99 +02:56:59 99 +02:57:01 98 +02:57:03 98 +02:57:05 97 +02:57:07 97 +02:57:09 97 +02:57:11 97 +02:57:13 95 +02:57:15 95 +02:57:17 94 +02:57:19 94 +02:57:21 93 +02:57:23 93 +02:57:25 93 +02:57:27 91 +02:57:30 91 +02:57:32 90 +02:57:34 89 +02:57:36 89 +02:57:38 89 +02:57:40 88 +02:57:42 87 +02:57:44 86 +02:57:46 86 +02:57:48 86 +02:57:50 85 +02:57:52 85 +02:57:54 84 +02:57:56 84 +02:57:58 83 +02:58:00 82 +02:58:02 82 +02:58:04 81 +02:58:07 81 +02:58:09 80 +02:58:11 79 +02:58:13 79 +02:58:15 80 +02:58:17 78 +02:58:19 78 +02:58:21 77 +02:58:23 77 +02:58:25 75 +02:58:27 75 +02:58:29 75 +02:58:31 74 +02:58:33 74 +02:58:35 73 +02:58:37 72 +02:58:39 72 +02:58:41 71 +02:58:44 71 +02:58:46 71 +02:58:48 69 +02:58:50 69 +02:58:52 69 +02:58:54 68 +02:58:56 68 +02:58:58 67 +02:59:00 67 +02:59:02 67 +02:59:04 65 +02:59:06 65 +02:59:08 66 +02:59:10 64 +02:59:12 64 +02:59:14 63 +02:59:16 63 +02:59:18 63 +02:59:20 61 +02:59:23 61 +02:59:25 61 +02:59:27 60 +02:59:29 60 +02:59:31 58 +02:59:33 58 +02:59:35 57 +02:59:37 57 +02:59:39 57 +02:59:41 56 +02:59:43 55 +02:59:45 55 +02:59:47 54 +02:59:49 54 +02:59:51 54 +02:59:53 52 +02:59:55 50 +02:59:57 50 +03:00:00 50 +03:00:02 50 +03:00:04 50 +03:00:06 50 +03:00:08 23 +03:00:10 12 +03:00:12 12 +03:00:14 12 +03:00:16 12 +03:00:18 12 +03:00:20 12 +03:00:22 12 +03:00:24 12 +03:00:26 12 +03:00:28 12 +03:00:30 12 +03:00:32 12 +03:00:35 12 +03:00:37 12 +03:00:39 12 +03:00:41 12 +03:00:43 12 +03:00:45 12 +03:00:47 12 +03:00:49 12 +03:00:51 12 +03:00:53 12 +03:00:55 12 +03:00:57 12 +03:00:59 12 +03:01:01 12 +03:01:03 12 +03:01:06 12 +03:01:08 12 +03:01:10 12 +03:01:12 12 +03:01:14 12 +03:01:16 12 +03:01:18 12 +03:01:20 12 +03:01:22 12 +03:01:24 12 +03:01:26 12 +03:01:28 12 +03:01:30 12 +03:01:32 12 +03:01:34 12 +03:01:36 12 +03:01:38 12 +03:01:41 12 +03:01:43 12 +03:01:45 12 +03:01:47 12 +03:01:49 12 +03:01:51 12 +03:01:53 12 +03:01:55 12 +03:01:57 12 +03:01:59 12 +03:02:01 12 +03:02:03 12 +03:02:05 12 +03:02:07 12 +03:02:09 12 +03:02:11 12 +03:02:14 12 +03:02:16 12 +03:02:18 12 +03:02:20 12 +03:02:22 12 +03:02:24 12 +03:02:26 12 +03:02:28 12 +03:02:30 12 +03:02:32 12 +03:02:34 12 +03:02:36 12 +03:02:38 12 +03:02:40 12 +03:02:42 12 +03:02:45 12 +03:02:47 12 +03:02:49 12 +03:02:51 12 +03:02:53 12 +03:02:55 12 +03:02:57 12 +03:02:59 12 +03:03:01 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/hostwatch-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/hostwatch-rank2.log new file mode 100644 index 0000000000..6dd6126922 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/hostwatch-rank2.log @@ -0,0 +1,202 @@ +02:56:08 114 +02:56:10 111 +02:56:12 111 +02:56:14 110 +02:56:16 110 +02:56:18 110 +02:56:20 108 +02:56:22 108 +02:56:24 108 +02:56:26 107 +02:56:28 106 +02:56:30 106 +02:56:32 106 +02:56:35 105 +02:56:37 105 +02:56:39 105 +02:56:41 104 +02:56:43 104 +02:56:45 103 +02:56:47 103 +02:56:49 101 +02:56:51 101 +02:56:53 101 +02:56:55 100 +02:56:57 100 +02:56:59 99 +02:57:01 98 +02:57:03 98 +02:57:05 97 +02:57:07 97 +02:57:09 97 +02:57:12 96 +02:57:14 96 +02:57:16 94 +02:57:18 94 +02:57:20 94 +02:57:22 93 +02:57:24 93 +02:57:26 92 +02:57:28 92 +02:57:30 92 +02:57:32 91 +02:57:34 89 +02:57:36 89 +02:57:38 89 +02:57:40 88 +02:57:42 88 +02:57:44 87 +02:57:46 86 +02:57:49 86 +02:57:51 85 +02:57:53 85 +02:57:55 84 +02:57:57 84 +02:57:59 84 +02:58:01 82 +02:58:03 83 +02:58:05 81 +02:58:07 81 +02:58:09 80 +02:58:11 80 +02:58:13 80 +02:58:15 79 +02:58:17 79 +02:58:19 77 +02:58:21 77 +02:58:23 76 +02:58:26 76 +02:58:28 75 +02:58:30 75 +02:58:32 75 +02:58:34 73 +02:58:36 73 +02:58:38 73 +02:58:40 72 +02:58:42 72 +02:58:44 72 +02:58:46 71 +02:58:48 71 +02:58:50 69 +02:58:52 69 +02:58:54 69 +02:58:56 68 +02:58:58 68 +02:59:00 67 +02:59:02 67 +02:59:05 67 +02:59:07 66 +02:59:09 66 +02:59:11 64 +02:59:13 64 +02:59:15 63 +02:59:17 63 +02:59:19 62 +02:59:21 62 +02:59:23 62 +02:59:25 60 +02:59:27 60 +02:59:29 59 +02:59:31 59 +02:59:33 58 +02:59:35 57 +02:59:37 55 +02:59:39 55 +02:59:42 55 +02:59:44 55 +02:59:46 55 +02:59:48 56 +02:59:50 42 +02:59:52 42 +02:59:54 42 +02:59:56 42 +02:59:58 42 +03:00:00 42 +03:00:02 42 +03:00:04 42 +03:00:06 42 +03:00:08 17 +03:00:10 17 +03:00:12 17 +03:00:14 17 +03:00:16 17 +03:00:19 17 +03:00:21 17 +03:00:23 17 +03:00:25 17 +03:00:27 17 +03:00:29 17 +03:00:31 17 +03:00:33 17 +03:00:35 17 +03:00:37 17 +03:00:39 17 +03:00:41 17 +03:00:43 17 +03:00:45 17 +03:00:47 17 +03:00:49 17 +03:00:52 17 +03:00:54 17 +03:00:56 17 +03:00:58 17 +03:01:00 17 +03:01:02 17 +03:01:04 17 +03:01:06 17 +03:01:08 17 +03:01:10 17 +03:01:12 17 +03:01:14 17 +03:01:16 17 +03:01:18 17 +03:01:20 17 +03:01:22 17 +03:01:24 17 +03:01:27 17 +03:01:29 17 +03:01:31 17 +03:01:33 17 +03:01:35 17 +03:01:37 17 +03:01:39 17 +03:01:41 17 +03:01:43 17 +03:01:45 17 +03:01:47 17 +03:01:49 17 +03:01:51 17 +03:01:53 17 +03:01:55 17 +03:01:57 17 +03:01:59 17 +03:02:02 17 +03:02:04 17 +03:02:06 17 +03:02:08 17 +03:02:10 17 +03:02:12 17 +03:02:14 17 +03:02:16 17 +03:02:18 17 +03:02:20 17 +03:02:22 17 +03:02:24 17 +03:02:26 17 +03:02:28 17 +03:02:30 17 +03:02:33 17 +03:02:35 17 +03:02:37 17 +03:02:39 17 +03:02:41 17 +03:02:43 17 +03:02:45 17 +03:02:47 17 +03:02:49 17 +03:02:51 17 +03:02:53 17 +03:02:55 17 +03:02:57 17 +03:02:59 17 +03:03:01 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/server-rank0.log new file mode 100644 index 0000000000..1a15585522 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/server-rank0.log @@ -0,0 +1,9572 @@ +[kvp-serve] 2026-10-22T17:55:58+00:00 rank 0 of 3 context 1048576 parallel 8 binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 22.8 +info: rank 0: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-a:10241:10241 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:10241:10241 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:10241:10241 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:10241:10241 [0] NCCL INFO cudaDriverVersion 13030 +host-a:10241:10241 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:10241:10241 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:10241:10241 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:10241:10241 [0] NCCL INFO Count not open libibverbs.so +host-a:10241:10241 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:10241:10241 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:10241:10241 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:10241:10241 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0xb985910 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:10241:10241 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0xb9c7010 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:10241:10241 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0xba08710 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:10241:10241 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0xba49e10 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:10241:10241 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:10241:10241 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:10241:10241 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:10241:10241 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:10241:10241 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:10241:10241 [0] NCCL INFO Initialized NET plugin IB +host-a:10241:10241 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:10241:10241 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:10241:10241 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:10241:10241 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:10241:10241 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:10241:10241 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:10241:10241 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:10241:10241 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:10241:10241 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:10241:10241 [0] NCCL INFO Using network IB +host-a:10241:10241 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:10241:10241 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x8d4e250 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xb32498ea1f573cb7 - Init START +host-a:10241:10241 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:10241:10241 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:10241:10241 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:10241:10241 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:10241:10241 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:10241:10241 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:10241:10241 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:10241:10241 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:10241:10241 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:10241:10241 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:10241:10241 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:10241:10241 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:10241:10241 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:10241:10241 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:10241:10241 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:10241:10241 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:10241:10241 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:10241:10241 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:10241:10241 [0] NCCL INFO Rank 0: 6 Net devices +host-a:10241:10241 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:10241:10241 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:10241:10241 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:10241:10241 [0] NCCL INFO comm 0x8d4e250 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:10241:10241 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:10241:10241 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:10241:10241 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:10241:10241 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:10241:10241 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:10241:10241 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:10241:10241 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:10241:10241 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:10241:10241 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:10241:10241 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:10241:10259 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 10 +host-a:10241:10258 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-a:10241:10258 [0] NCCL INFO proxy listening socket at <58275> +host-a:10241:10241 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:10241:10241 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:10241:10260 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-a:10241:10241 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:10241:10241 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:10241:10241 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:10241:10241 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:10241:10241 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:10241:10241 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b0010000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b0054000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b009c000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6837 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6838 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6837 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6837 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6838 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6838 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b0145000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b018d000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b01d1000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b0219000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b025d000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6869 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6870 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6869 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6869 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6870 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6870 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1937 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1938 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1939 pkey=0 pd=0xe919b02b7470 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1809 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1810 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1811 pkey=0 pd=0xe919b02b7f80 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1940 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1941 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1942 pkey=0 pd=0xe919b02b7470 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6837 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6838 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1812 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1813 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1814 pkey=0 pd=0xe919b02b7f80 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6869 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6870 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6839 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6840 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6839 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x19605d ctsFifoLkey=0x19605d +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6839 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6840 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x19605d ctsFifoLkey=0x19605d +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6840 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6871 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6872 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6871 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1cd2d6 ctsFifoLkey=0x1cd2d6 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6871 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6872 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1cd2d6 ctsFifoLkey=0x1cd2d6 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6872 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6839 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6840 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6871 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6872 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10241 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:10241:10241 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:10241:10241 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:10241:10241 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:10241:10241 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b05b5000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b05f6000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b0637000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1815 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1816 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1815 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1815 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1816 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1816 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b06d1000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1943 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1944 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1943 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c2c1 ctsFifoLkey=0x19c2c1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1943 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1944 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c2c1 ctsFifoLkey=0x19c2c1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1944 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1815 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1816 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1943 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1944 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1817 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1818 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1817 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1817 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1818 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1818 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1945 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1946 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1945 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a6a8 ctsFifoLkey=0x19a6a8 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1945 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1946 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a6a8 ctsFifoLkey=0x19a6a8 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1946 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1817 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1818 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1945 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1946 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10241 [0] NCCL INFO Connected all trees +host-a:10241:10241 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:10241:10241 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:10241:10241 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:10241:10241 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b0822000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b0867000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b08ac000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b08f4000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6841 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6842 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6843 pkey=0 pd=0xe919b0053590 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6873 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6874 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6875 pkey=0 pd=0xe919b00f2d40 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6844 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6845 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6846 pkey=0 pd=0xe919b0053590 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6876 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6877 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6878 pkey=0 pd=0xe919b00f2d40 +host-a:10241:10241 [0] NCCL INFO Connected binomial trees +host-a:10241:10241 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:10241:10241 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:10241:10241 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:10241:10241 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:10241:10241 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-a:10241:10241 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:10241:10241 [0] NCCL INFO ncclCommInitRank comm 0x8d4e250 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xb32498ea1f573cb7 - Init COMPLETE +host-a:10241:10241 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 4.59 (kernels 0.24, alloc 0.04, bootstrap 3.99, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.31, rest 0.00) +host-a:10241:10241 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:10241:10241 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:10241:10241 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:10241:10241 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:10241:10241 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:10241:10241 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:10241:10241 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:10241:10241 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b0b18000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b0b59000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b0b9e000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b0be2000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b0c2a000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b0c6e000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe919b0cb6000) +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe919b0cfa000) +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6847 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6848 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6847 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x190b04 ctsFifoLkey=0x190b04 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6847 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6848 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x190b04 ctsFifoLkey=0x190b04 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6848 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1819 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1820 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1821 pkey=0 pd=0xe919b02b7f80 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6879 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6880 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6879 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d8e88 ctsFifoLkey=0x1d8e88 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6879 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6880 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d8e88 ctsFifoLkey=0x1d8e88 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6880 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1947 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1948 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1949 pkey=0 pd=0xe919b02b7470 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1950 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1951 pkey=0 pd=0xe919b02b7470 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1950 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b4ad ctsFifoLkey=0x19b4ad +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1950 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1951 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b4ad ctsFifoLkey=0x19b4ad +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1951 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1822 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1823 pkey=0 pd=0xe919b02b7f80 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1822 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17924b ctsFifoLkey=0x17924b +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1822 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1823 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17924b ctsFifoLkey=0x17924b +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1823 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6847 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6848 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-awareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 78.03 GiB; 8 streams at once, the engine's own at a 1048576-token window (12.88 GiB, in the model's), 7 more taking 206 pages of 512 rows (1.28 GiB) as they need them, 131 MiB fixed each, of 25.0 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 22.8 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 1048576; loaded in 250.8s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 1 states, 922 MiB and 22424 MiB spare of 23347 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 206 + store 0 == total 206 +[tensorfold] done chatcmpl-412e2d91069a008aae9c2aba7a734ea9 prompt=65536 cached=0 thinking=False effort=none tokens=128 sha=d8a8421597ee finish=length rounds=64 accepted=63/108 ttft=43.222 decode=3.769 elapsed=46.991 queued=0.000 tok/s=33.7 seed=1000 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 2 states, 1844 MiB and 21502 MiB spare of 23347 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 206 + store 0 == total 206 +[tensorfold] done chatcmpl-f2bae3ebadf872deb200a039d63f13e1 prompt=65536 cached=0 thinking=False effort=none tokens=128 sha=bf47482e16cb finish=length rounds=50 accepted=77/108 ttft=43.243 decode=3.117 elapsed=46.360 queued=0.000 tok/s=40.7 seed=1001 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 3 states, 2766 MiB and 20580 MiB spare of 23347 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 206 + store 0 == total 206 +[tensorfold] done chatcmpl-f1131cec916c19b2a961dfaafdac96ba prompt=65536 cached=0 thinking=False effort=none tokens=128 sha=30f96d96d0b7 finish=length rounds=55 accepted=73/98 ttft=43.303 decode=3.256 elapsed=46.558 queued=0.000 tok/s=39.0 seed=1002 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 63488, kept 0; 3 states, 2766 MiB and 20580 MiB spare of 23347 MiB (hits 1, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 206 + store 0 == total 206 +[tensorfold] done chatcmpl-f2a08b7d79c0bf90b4e0f905897a4911 prompt=65536 cached=63488 thinking=False effort=none tokens=128 sha=d8a8421597ee finish=length rounds=64 accepted=63/108 ttft=1.437 decode=3.772 elapsed=5.210 queued=0.000 tok/s=33.7 seed=1000 +info: prompt cache: 65536 tokens, resumed at 63488, kept 0; 3 states, 2766 MiB and 20580 MiB spare of 23347 MiB (hits 2, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 206 + store 0 == total 206 +[tensorfold] done chatcmpl-8c9f179fd545918885f36d6b513c9b5d prompt=65536 cached=63488 thinking=False effort=none tokens=128 sha=bf47482e16cb finish=length rounds=50 accepted=77/108 ttft=6.659 decode=3.121 elapsed=9.781 queued=5.212 tok/s=40.7 seed=1001 +info: prompt cache: 65536 tokens, resumed at 63488, kept 0; 3 states, 2766 MiB and 20580 MiB spare of 23347 MiB (hits 3, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 206 + store 0 == total 206 +[tensorfold] done chatcmpl-b31aeeb91cc2b80593b2a00aaf517335 prompt=65536 cached=63488 thinking=False effort=none tokens=128 sha=30f96d96d0b7 finish=length rounds=55 accepted=73/98 ttft=11.227 decode=3.263 elapsed=14.489 queued=5.212 tok/s=38.9 seed=1002 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped + routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6879 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6880 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1950 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1951 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1822 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1823 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6849 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6850 pkey=0 pd=0xe919b0053590 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6851 pkey=0 pd=0xe919b0053590 +host-a:10241:10258 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6881 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6882 pkey=0 pd=0xe919b00f2d40 oooRq=0 +host-a:10241:10258 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6883 pkey=0 pd=0xe919b00f2d40 +host-a:10241:10241 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/server-rank1.log new file mode 100644 index 0000000000..5e17ef0fd6 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/server-rank1.log @@ -0,0 +1,8030 @@ +[kvp-serve] 2026-10-22T17:55:57+00:00 rank 1 of 3 context 1048576 parallel 8 binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 22.8 +info: rank 1: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-b:8982:8982 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:8982:8982 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:8982:8982 [0] NCCL INFO cudaDriverVersion 13030 +host-b:8982:8982 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:8982:8982 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:8982:8982 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:8982:8982 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:8982:8982 [0] NCCL INFO Count not open libibverbs.so +host-b:8982:8982 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:8982:8982 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:8982:8982 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:8982:8982 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x1e594450 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:8982:8982 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x1e5d5b50 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:8982:8982 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x1e617250 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:8982:8982 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x1e658950 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:8982:8982 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:8982:8982 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:8982:8982 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:8982:8982 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:8982:8982 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:8982:8982 [0] NCCL INFO Initialized NET plugin IB +host-b:8982:8982 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:8982:8982 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:8982:8982 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:8982:8982 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:8982:8982 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:8982:8982 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:8982:8982 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:8982:8982 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:8982:8982 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:8982:8982 [0] NCCL INFO Using network IB +host-b:8982:8982 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:8982:8982 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x1b95cdd0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xb32498ea1f573cb7 - Init START +host-b:8982:8982 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:8982:8982 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:8982:8982 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:8982:8982 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:8982:8982 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:8982:8982 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:8982:8982 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:8982:8982 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:8982:8982 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:8982:8982 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:8982:8982 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:8982:8982 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:8982:8982 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:8982:8982 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:8982:8982 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:8982:8982 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:8982:8982 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:8982:8982 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:8982:8982 [0] NCCL INFO Rank 1: 6 Net devices +host-b:8982:8982 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:8982:8982 [0] NCCL INFO comm 0x1b95cdd0 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:8982:8982 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:8982:8982 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:8982:8982 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:8982:8982 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:8982:8982 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:8982:8982 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:8982:8997 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-b:8982:8997 [0] NCCL INFO proxy listening socket at <49529> +host-b:8982:8998 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-b:8982:8999 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-b:8982:8982 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:8982:8982 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:8982:8982 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:8982:8982 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:8982:8982 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:8982:8982 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:8982:8982 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:8982:8982 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce8010000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce8054000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce809c000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2065 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2066 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2065 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2065 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2066 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2066 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2193 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2194 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2193 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2193 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2194 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2194 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce81aa000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6613 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6614 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6615 pkey=0 pd=0xf0bce8123a60 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6645 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6646 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6647 pkey=0 pd=0xf0bce820cd90 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce82fa000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2065 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2066 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2193 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2194 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce833e000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce8383000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce83c4000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2067 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2068 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2067 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8481 ctsFifoLkey=0x1b8481 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2067 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2068 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8481 ctsFifoLkey=0x1b8481 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2068 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2195 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2196 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2195 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de8e0 ctsFifoLkey=0x1de8e0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2195 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2196 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de8e0 ctsFifoLkey=0x1de8e0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2196 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2067 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2068 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2195 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2196 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6616 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6617 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6618 pkey=0 pd=0xf0bce8123a60 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6648 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6649 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6650 pkey=0 pd=0xf0bce820cd90 +host-b:8982:8982 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:8982:8982 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:8982:8982 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:8982:8982 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:8982:8982 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce85ab000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce85f0000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce8635000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce867d000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2069 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2070 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2071 pkey=0 pd=0xf0bce8053590 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2197 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2198 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2199 pkey=0 pd=0xf0bce80f2420 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2072 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2073 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2074 pkey=0 pd=0xf0bce8053590 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2200 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2201 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2202 pkey=0 pd=0xf0bce80f2420 +host-b:8982:8982 [0] NCCL INFO Connected all trees +host-b:8982:8982 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:8982:8982 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:8982:8982 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:8982:8982 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce88a7000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce88e8000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6619 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6620 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6619 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6619 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6620 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6620 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6651 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6652 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6651 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x15f7f4 ctsFifoLkey=0x15f7f4 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6651 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6652 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x15f7f4 ctsFifoLkey=0x15f7f4 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6652 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce89df000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce8a23000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6619 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6620 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6651 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6652 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6621 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6622 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6621 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x164401 ctsFifoLkey=0x164401 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6621 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6622 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x164401 ctsFifoLkey=0x164401 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6622 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6653 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6654 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6653 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c7c4 ctsFifoLkey=0x19c7c4 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6653 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6654 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c7c4 ctsFifoLkey=0x19c7c4 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6654 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6621 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6622 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6653 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6654 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8982 [0] NCCL INFO Connected binomial trees +host-b:8982:8982 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:8982:8982 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:8982:8982 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:8982:8982 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:8982:8982 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-b:8982:8982 [0] NCCL INFO ncclCommInitRank comm 0x1b95cdd0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xb32498ea1f573cb7 - Init COMPLETE +host-b:8982:8982 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 1.59 (kernels 0.24, alloc 0.05, bootstrap 0.98, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.31, rest 0.00) +host-b:8982:8982 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:8982:8982 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:8982:8982 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:8982:8982 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:8982:8982 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:8982:8982 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:8982:8982 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:8982:8982 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce8b19000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce8b5a000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2075 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2076 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2075 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2075 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2076 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2076 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce8bfa000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce8c3e000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce8c86000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce8cca000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf0bce8d12000) +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf0bce8d56000) +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6623 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6624 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6625 pkey=0 pd=0xf0bce8123a60 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2203 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2204 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2203 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1ddcd3 ctsFifoLkey=0x1ddcd3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2203 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2204 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1ddcd3 ctsFifoLkey=0x1ddcd3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2204 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6655 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6656 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6657 pkey=0 pd=0xf0bce820cd90 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2205 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2206 pkey=0 pd=0xf0bce80f2420 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2207 pkey=0 pd=0xf0bce80f2420 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6626 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6627 pkey=0 pd=0xf0bce8123a60 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6626 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x177532 ctsFifoLkey=0x177532 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6626 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6627 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x177532 ctsFifoLkey=0x177532 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6627 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2075 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2076 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2203 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2204 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6658 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6659 pkey=0 pd=0xf0bce820cd90 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6658 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19cdca ctsFifoLkey=0x19cdca +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6658 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6659 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19cdca ctsFifoLkey=0x19cdca +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6659 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2077 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2078 pkey=0 pd=0xf0bce8053590 oooRq=0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2079 pkey=0 pd=0xf0bce8053590 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6626 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6627 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 1: rank 0 ended; leaving +overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6658 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8997 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6659 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8982:8982 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/server-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/server-rank2.log new file mode 100644 index 0000000000..17191e9b15 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6admit/server-rank2.log @@ -0,0 +1,7450 @@ +[kvp-serve] 2026-10-22T17:55:56+00:00 rank 2 of 3 context 1048576 parallel 8 binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 22.8 +info: rank 2: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-c:8938:8938 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:8938:8938 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:8938:8938 [0] NCCL INFO cudaDriverVersion 13030 +host-c:8938:8938 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:8938:8938 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:8938:8938 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:8938:8938 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:8938:8938 [0] NCCL INFO Count not open libibverbs.so +host-c:8938:8938 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:8938:8938 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:8938:8938 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:8938:8938 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x3fbd65f0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:8938:8938 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x3fc17cf0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:8938:8938 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x3fc593f0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:8938:8938 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x3fc9aaf0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:8938:8938 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:8938:8938 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:8938:8938 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:8938:8938 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:8938:8938 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:8938:8938 [0] NCCL INFO Initialized NET plugin IB +host-c:8938:8938 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:8938:8938 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:8938:8938 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:8938:8938 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:8938:8938 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:8938:8938 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:8938:8938 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:8938:8938 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:8938:8938 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:8938:8938 [0] NCCL INFO Using network IB +host-c:8938:8938 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:8938:8938 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x3cf9ef80 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xb32498ea1f573cb7 - Init START +host-c:8938:8938 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:8938:8938 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:8938:8938 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:8938:8938 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:8938:8938 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:8938:8938 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:8938:8938 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:8938:8938 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:8938:8938 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:8938:8938 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:8938:8938 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:8938:8938 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:8938:8938 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:8938:8938 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:8938:8938 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:8938:8938 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:8938:8938 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:8938:8938 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:8938:8938 [0] NCCL INFO Rank 2: 6 Net devices +host-c:8938:8938 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:8938:8938 [0] NCCL INFO comm 0x3cf9ef80 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:8938:8938 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:8938:8938 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:8938:8938 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:8938:8938 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:8938:8938 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:8938:8938 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:8938:8953 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-c:8938:8954 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 4 +host-c:8938:8953 [0] NCCL INFO proxy listening socket at <43207> +host-c:8938:8955 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-c:8938:8938 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:8938:8938 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:8938:8938 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:8938:8938 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:8938:8938 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:8938:8938 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:8938:8938 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:8938:8938 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396c010000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396c054000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396c09c000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396c0e0000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396c128000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396c16c000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396c1b4000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396c1f8000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1809 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1810 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1811 pkey=0 pd=0xfc396c053eb0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2193 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2194 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2193 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2193 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2194 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2194 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1937 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1938 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1939 pkey=0 pd=0xfc396c28cad0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2065 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2066 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2065 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bd4d1 ctsFifoLkey=0x1bd4d1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2065 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2066 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bd4d1 ctsFifoLkey=0x1bd4d1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2066 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2195 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2196 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2195 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d1c1b ctsFifoLkey=0x1d1c1b +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2195 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2196 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d1c1b ctsFifoLkey=0x1d1c1b +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2196 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2067 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2068 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2067 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbcba ctsFifoLkey=0x1bbcba +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2067 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2068 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbcba ctsFifoLkey=0x1bbcba +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2068 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2193 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2194 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2065 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2066 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2195 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2196 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1812 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1813 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1814 pkey=0 pd=0xfc396c053eb0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1940 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1941 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1942 pkey=0 pd=0xfc396c28cad0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2067 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2068 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8938 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:8938:8938 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:8938:8938 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:8938:8938 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:8938:8938 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:8938:8938 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:8938:8938 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:8938:8938 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:8938:8938 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396c5af000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396c5f0000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396c635000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1815 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1816 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1815 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1807c5 ctsFifoLkey=0x1807c5 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1815 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1816 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1807c5 ctsFifoLkey=0x1807c5 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1816 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1943 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1944 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1943 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19ebe8 ctsFifoLkey=0x19ebe8 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1943 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1944 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19ebe8 ctsFifoLkey=0x19ebe8 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1944 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396c72d000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2069 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2070 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2071 pkey=0 pd=0xfc396c2d7de0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2197 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2198 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2199 pkey=0 pd=0xfc396c25aa40 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396c868000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1815 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1816 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1943 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1944 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1817 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1818 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1817 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x179655 ctsFifoLkey=0x179655 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1817 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1818 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x179655 ctsFifoLkey=0x179655 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1818 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396c8ff000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396c947000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396c98b000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2072 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2073 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2074 pkey=0 pd=0xfc396c2d7de0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1945 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1946 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1945 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1945 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1946 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1946 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2200 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2201 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2202 pkey=0 pd=0xfc396c25aa40 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1817 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1818 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1945 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1946 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8938 [0] NCCL INFO Connected all trees +host-c:8938:8938 [0] NCCL INFO Connected binomial trees +host-c:8938:8938 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:8938:8938 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:8938:8938 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:8938:8938 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:8938:8938 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-c:8938:8938 [0] NCCL INFO ncclCommInitRank comm 0x3cf9ef80 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xb32498ea1f573cb7 - Init COMPLETE +host-c:8938:8938 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.53 (kernels 0.24, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.23, rest 0.00) +host-c:8938:8938 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:8938:8938 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:8938:8938 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:8938:8938 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:8938:8938 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:8938:8938 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:8938:8938 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:8938:8938 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396cb1e000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396cb5f000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396cba4000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396cbe5000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396cc2d000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2075 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2076 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2075 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1b9795 ctsFifoLkey=0x1b9795 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2075 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2076 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1b9795 ctsFifoLkey=0x1b9795 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2076 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1819 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1820 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1821 pkey=0 pd=0xfc396c053eb0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2203 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2204 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2203 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d4d4d ctsFifoLkey=0x1d4d4d +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2203 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2204 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d4d4d ctsFifoLkey=0x1d4d4d +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2204 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1947 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1948 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1947 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1947 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1948 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1948 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396cdfc000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc396ce44000) +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc396ce88000) +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2075 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2076 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1949 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1950 pkey=0 pd=0xfc396c28cad0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1951 pkey=0 pd=0xfc396c28cad0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2203 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2204 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2205 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2206 pkey=0 pd=0xfc396c25aa40 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2207 pkey=0 pd=0xfc396c25aa40 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2077 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2078 pkey=0 pd=0xfc396c2d7de0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2079 pkey=0 pd=0xfc396c2d7de0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1947 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1948 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1822 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1823 pkey=0 pd=0xfc396c053eb0 oooRq=0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1822 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17bb78 ctsFifoLkey=0x17bb78 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1822 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1823 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17bb78 ctsFifoLkey=0x17bb78 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1823 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1822 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8953 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1823 set_ece={supported=1, vendor_info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 2: rank 0 ended; leaving +id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8938:8938 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/client.log new file mode 100644 index 0000000000..3a92f11529 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/client.log @@ -0,0 +1,19 @@ +server up 2026-10-23 02:24:38 +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 48.4, "per_stream_tps": 48.4, "ttft_s_max": 0.11, "timing_valid": true, "end_to_end_tps": 47.575070687275236, "makespan_s": 5.380969409015961, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 59.2, "per_stream_tps": 32.2, "ttft_s_max": 0.28, "timing_valid": true, "end_to_end_tps": 58.71271323264427, "makespan_s": 8.720428197062574, "steady_tps": 61.5, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 77.2, "per_stream_tps": 20.4, "ttft_s_max": 0.69, "timing_valid": true, "end_to_end_tps": 76.81033516659433, "makespan_s": 13.331539275008254, "steady_tps": 80.0, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 82.1, "per_stream_tps": 13.8, "ttft_s_max": 17.29, "timing_valid": true, "end_to_end_tps": 82.07306424255029, "makespan_s": 24.953375615994446, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 35.1, "per_stream_tps": 35.1, "ttft_s_max": 0.16, "timing_valid": true, "end_to_end_tps": 34.45920140316111, "makespan_s": 7.429075241903774, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 53.3, "per_stream_tps": 27.8, "ttft_s_max": 0.38, "timing_valid": true, "end_to_end_tps": 52.575569935379995, "makespan_s": 9.738363286014646, "steady_tps": 55.8, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 69.3, "per_stream_tps": 18.3, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 68.78482603081082, "makespan_s": 14.887004286982119, "steady_tps": 73.2, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 72.2, "per_stream_tps": 12.3, "ttft_s_max": 21.23, "timing_valid": true, "end_to_end_tps": 72.0420621546337, "makespan_s": 28.427836998947896, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 70.6, "per_stream_tps": 70.6, "ttft_s_max": 0.11, "timing_valid": true, "end_to_end_tps": 68.81210534106044, "makespan_s": 3.720275651081465, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 81.7, "per_stream_tps": 41.6, "ttft_s_max": 0.27, "timing_valid": true, "end_to_end_tps": 80.66172781763856, "makespan_s": 6.347496066009626, "steady_tps": 83.5, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 115.4, "per_stream_tps": 30.1, "ttft_s_max": 0.67, "timing_valid": true, "end_to_end_tps": 114.4309649062791, "makespan_s": 8.948626806028187, "steady_tps": 121.9, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 128.8, "per_stream_tps": 21.7, "ttft_s_max": 12.01, "timing_valid": true, "end_to_end_tps": 128.3746663734432, "makespan_s": 15.953303388087079, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 34.9, "per_stream_tps": 34.9, "ttft_s_max": 0.16, "timing_valid": true, "end_to_end_tps": 34.25549711861656, "makespan_s": 7.473253099014983, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 49.9, "per_stream_tps": 25.3, "ttft_s_max": 0.38, "timing_valid": true, "end_to_end_tps": 49.33394520505358, "makespan_s": 10.378249658970162, "steady_tps": 50.7, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 69.0, "per_stream_tps": 17.8, "ttft_s_max": 0.9, "timing_valid": true, "end_to_end_tps": 68.55453390507776, "makespan_s": 14.937013522954658, "steady_tps": 71.9, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 71.6, "per_stream_tps": 12.4, "ttft_s_max": 20.97, "timing_valid": true, "end_to_end_tps": 71.49847345532, "makespan_s": 28.64396819996182, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +logs/r6all-conc.json: 16 cells; alone 60/60 equal (0 unequal, 0 failed); serial 72/72 equal (0 unequal, 0 failed) +=== kvp-client r6all rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/conc.json b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/conc.json new file mode 100644 index 0000000000..b237a3269c --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/conc.json @@ -0,0 +1,9533 @@ +{ + "label": "", + "tokens": 256, + "mixed": false, + "cells": [ + { + "prompt": "code", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.4, + "per_stream_tps": 48.4, + "ttft_s_max": 0.11, + "timing_valid": true, + "end_to_end_tps": 47.575070687275236, + "makespan_s": 5.380969409015961, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.37998950619049, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 5.380969409015961, + "end_to_end_tps": 47.575070687275236, + "timing_valid": true, + "per_stream_tps": [ + 48.4 + ], + "ttft_s": [ + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + } + ], + "client_intervals": [ + { + "start": 723533.216236708, + "first": 723533.326280068, + "last": 723538.597054258, + "end": 723538.597206117, + "elapsed_s": 5.380969409015961, + "chunk_offsets_s": [ + 0.11004336003679782, + 0.16907837695907801, + 0.22692002495750785, + 0.2872834970476106, + 0.35263796406798065, + 0.4199859469663352, + 0.4836040230002254, + 0.5472227570135146, + 0.6122368490323424, + 0.6793567619752139, + 0.7463953329715878, + 0.8139776600291952, + 0.881683852057904, + 0.9481735579902306, + 1.0136931770248339, + 1.0759575490374118, + 1.1334249490173534, + 1.1885008660610765, + 1.2398376159835607, + 1.2934483729768544, + 1.3534384040394798, + 1.4152369439834729, + 1.4746545830275863, + 1.533962543006055, + 1.5952286489773542, + 1.6637162730330601, + 1.7337235040031374, + 1.8007084419950843, + 1.8649082150077447, + 1.9323764380533248, + 1.9971444889670238, + 2.0629224840085953, + 2.129006296978332, + 2.1959838300244883, + 2.265246203984134, + 2.3328966450644657, + 2.401063904981129, + 2.468477827962488, + 2.5385775710456073, + 2.6075927020283416, + 2.676408280036412, + 2.7432908470509574, + 2.810783870983869, + 2.880994973005727, + 2.9492497709579766, + 3.0186127780470997, + 3.0883115750038996, + 3.157591272960417, + 3.2273167229723185, + 3.2960981120122597, + 3.3639946880284697, + 3.433260843041353, + 3.5005000460660085, + 3.5665482020704076, + 3.6336885190103203, + 3.7017179210670292, + 3.7666468310635537, + 3.825121687958017, + 3.8854958730516955, + 3.948885511024855, + 4.014224185026251, + 4.078948643058538, + 4.16824362706393, + 4.238800017046742, + 4.3056677100248635, + 4.373591150972061, + 4.438135200063698, + 4.502265262068249, + 4.590834871050902, + 4.6565782990073785, + 4.722264606971294, + 4.788893081014976, + 4.856358638033271, + 4.922972319996916, + 4.993187003070489, + 5.0601338089909405, + 5.129873031983152, + 5.199862231966108, + 5.268217968055978, + 5.332464306033216, + 5.380817550001666 + ], + "chunk_gaps_ms": { + "count": 80, + "p50": 66.91468646749854, + "p95": 70.2111020218581, + "max": 89.2949840053916 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.2, + "per_stream_tps": 32.2, + "ttft_s_max": 0.28, + "timing_valid": true, + "end_to_end_tps": 58.71271323264427, + "makespan_s": 8.720428197062574, + "steady_tps": 61.5, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.23650281976831, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 8.720428197062574, + "end_to_end_tps": 58.71271323264427, + "timing_valid": true, + "steady_tps": 61.50178209986573, + "per_stream_tps": [ + 34.7, + 29.6 + ], + "ttft_s": [ + 0.28, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + } + ], + "client_intervals": [ + { + "start": 723538.599413891, + "first": 723538.875066608, + "last": 723546.232637206, + "end": 723546.232748303, + "elapsed_s": 7.633334412006661, + "chunk_offsets_s": [ + 0.27565271698404104, + 0.3571963970316574, + 0.44925564306322485, + 0.5436132090399042, + 0.63703641097527, + 0.7289123890222982, + 0.8181203249841928, + 0.9105206499807537, + 1.0013880700571463, + 1.0967062989948317, + 1.1919021420180798, + 1.2874035090208054, + 1.3833639550721273, + 1.4777306800242513, + 1.5730651260819286, + 1.6643392570549622, + 1.7572483440162614, + 1.848584401072003, + 1.9407486750278622, + 2.0360061330720782, + 2.132191126001999, + 2.2276959960581735, + 2.3225714419968426, + 2.418190740980208, + 2.516451528063044, + 2.612555565079674, + 2.7066851240815595, + 2.8028775149723515, + 2.8970703850500286, + 2.9897045750403777, + 3.084521102020517, + 3.1811582080554217, + 3.2821648720419034, + 3.3800232160137966, + 3.4754739350173622, + 3.5700876030605286, + 3.6659377720206976, + 3.7652917630039155, + 3.8623791720019653, + 3.9571194930467755, + 4.055986604071222, + 4.1511669480241835, + 4.247629372985102, + 4.3466868720715865, + 4.445869653020054, + 4.5461072120815516, + 4.645555941038765, + 4.745914182974957, + 4.841759340022691, + 4.937155832070857, + 5.038792315986939, + 5.136662577977404, + 5.231659085024148, + 5.327431740006432, + 5.424146134988405, + 5.521276395069435, + 5.619224313995801, + 5.712450421066023, + 5.809830365004018, + 5.9073554740753025, + 6.008573379018344, + 6.109107105061412, + 6.206951098982245, + 6.301185923046432, + 6.396294753067195, + 6.495645257062279, + 6.592706955969334, + 6.6871147570200264, + 6.783194178016856, + 6.880611233995296, + 6.975697870017029, + 7.07246146898251, + 7.172537883045152, + 7.273525568074547, + 7.368208443047479, + 7.462845167028718, + 7.55328538606409, + 7.633223315002397 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 95.5048700561747, + "p95": 100.98768502939492, + "max": 101.63648391608149 + } + }, + { + "start": 723538.599315326, + "first": 723538.710035519, + "last": 723547.319591717, + "end": 723547.319743523, + "elapsed_s": 8.720428197062574, + "chunk_offsets_s": [ + 0.11072019301354885, + 0.16812284698244184, + 0.35700381197966635, + 0.44902047503273934, + 0.543268125038594, + 0.6367634470807388, + 0.7285779369994998, + 0.8178035150049254, + 0.9102872440125793, + 1.0010920549975708, + 1.0963514200411737, + 1.1915153100853786, + 1.2871552800061181, + 1.3832485970342532, + 1.4774391000391915, + 1.5734129290794954, + 1.6642845240421593, + 1.7568449940299615, + 1.848296056035906, + 1.9404217650881037, + 2.0354718490270898, + 2.1318330310750753, + 2.2274645610013977, + 2.3221852210117504, + 2.4180816920707002, + 2.5162987750954926, + 2.6123492480255663, + 2.7063749770168215, + 2.8026801639935, + 2.896763744065538, + 2.989478343981318, + 3.0841779940528795, + 3.1811430170200765, + 3.281850003055297, + 3.3795984480530024, + 3.4751325550023466, + 3.5699722630670294, + 3.666161571047269, + 3.7651474170852453, + 3.8621021800208837, + 3.957026021089405, + 4.0556626750621945, + 4.150818062014878, + 4.247362034046091, + 4.346451166085899, + 4.44554774207063, + 4.5458026380511, + 4.645321883028373, + 4.745819575036876, + 4.841515951091424, + 4.936879241024144, + 5.038665403029881, + 5.136365849059075, + 5.231632320093922, + 5.327785494038835, + 5.424070465029217, + 5.521186462021433, + 5.61913466907572, + 5.712649933993816, + 5.809500180068426, + 5.9071564599871635, + 6.008544902084395, + 6.10897689498961, + 6.2065636509796605, + 6.300824632984586, + 6.3958611860871315, + 6.495261283009313, + 6.592296760994941, + 6.6867081969976425, + 6.782757234061137, + 6.880260783014819, + 6.9752238830551505, + 7.0723320920951664, + 7.172297712997533, + 7.273190853069536, + 7.368116062018089, + 7.462455368018709, + 7.553001753985882, + 7.632890073000453, + 7.698570834007114, + 7.756604377995245, + 7.808277560048737, + 7.862616338068619, + 7.9209206129889935, + 7.977986010024324, + 8.037353391991928, + 8.088776236982085, + 8.14207708905451, + 8.20019081805367, + 8.259137215092778, + 8.320862962980755, + 8.381644656066783, + 8.43718611705117, + 8.502076021046378, + 8.587835030979477, + 8.662390436045825, + 8.72027639101725 + ], + "chunk_gaps_ms": { + "count": 96, + "p50": 95.1027354458347, + "p95": 100.70698603522032, + "max": 188.8809649972245 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 77.2, + "per_stream_tps": 20.4, + "ttft_s_max": 0.69, + "timing_valid": true, + "end_to_end_tps": 76.81033516659433, + "makespan_s": 13.331539275008254, + "steady_tps": 80.0, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 77.15621407695932, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 13.331539275008254, + "end_to_end_tps": 76.81033516659433, + "timing_valid": true, + "steady_tps": 80.02556346917368, + "per_stream_tps": [ + 23.1, + 20.2, + 20.4, + 20.3 + ], + "ttft_s": [ + 0.47, + 0.69, + 0.28, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0b4d014c8872", + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b82191a9a6ec", + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a" + } + ], + "client_intervals": [ + { + "start": 723547.323012559, + "first": 723547.791389559, + "last": 723558.829954432, + "end": 723558.830220582, + "elapsed_s": 11.50720802298747, + "chunk_offsets_s": [ + 0.468377000070177, + 0.5770319760777056, + 0.809416295029223, + 0.9391964530805126, + 1.0767510819714516, + 1.2103628440527245, + 1.3369429640006274, + 1.4661544139962643, + 1.5964352580485865, + 1.735561294015497, + 1.8715011429740116, + 2.0149086529854685, + 2.160736709018238, + 2.3019341320032254, + 2.439325647079386, + 2.5705480810720474, + 2.704935297020711, + 2.8422109769890085, + 2.9812381400261074, + 3.1237007670570165, + 3.26623056398239, + 3.403633321984671, + 3.5479120240779594, + 3.6931754190009087, + 3.839887439040467, + 3.985001669032499, + 4.130413893028162, + 4.271094260970131, + 4.409541159053333, + 4.548009189078584, + 4.690649318974465, + 4.834572067018598, + 4.979864594060928, + 5.123016832047142, + 5.27270197798498, + 5.4183444359805435, + 5.560505900066346, + 5.704520735074766, + 5.847296750056557, + 5.993501177057624, + 6.13730075804051, + 6.282202513073571, + 6.434231007006019, + 6.577250179019757, + 6.724643400986679, + 6.873580908984877, + 7.021872190060094, + 7.173046366078779, + 7.3142675660783425, + 7.458535087062046, + 7.599040852976032, + 7.736192558077164, + 7.883146506967023, + 8.029263161006384, + 8.177049453021027, + 8.321717484039254, + 8.467159586027265, + 8.6066228330601, + 8.749216121970676, + 8.896260278997943, + 9.044685123022646, + 9.194406385999173, + 9.344384083058685, + 9.485141744022258, + 9.626670225989074, + 9.770861872006208, + 9.917439441080205, + 10.065643879002891, + 10.213316358975135, + 10.357307609985583, + 10.503786048968323, + 10.65080626506824, + 10.800536051974632, + 10.94711011997424, + 11.091621347004548, + 11.237025839975104, + 11.374663532013074, + 11.50694187299814 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 143.99125101044774, + "p95": 149.97769705951214, + "max": 232.38431895151734 + } + }, + { + "start": 723547.322045455, + "first": 723548.008325222, + "last": 723560.653372557, + "end": 723560.653462443, + "elapsed_s": 13.33141698804684, + "chunk_offsets_s": [ + 0.6862797669600695, + 0.8098963809898123, + 0.9403684620046988, + 1.0779259400442243, + 1.2117504989728332, + 1.3382063639583066, + 1.4673038669861853, + 1.5979886769782752, + 1.736780463019386, + 1.8728760549565777, + 2.0162296179914847, + 2.1613527860026807, + 2.303087329957634, + 2.4400886789662763, + 2.5716608279617503, + 2.7060169560136274, + 2.843394831987098, + 2.982459646067582, + 3.1248769690282643, + 3.2672740110429004, + 3.404681266983971, + 3.548979211016558, + 3.69424161396455, + 3.8410059160087258, + 3.986116528045386, + 4.131641705986112, + 4.272403217968531, + 4.410743743064813, + 4.549191458034329, + 4.691751883015968, + 4.83575795404613, + 4.981148145976476, + 5.124170494033024, + 5.273562147980556, + 5.419416522025131, + 5.561832673964091, + 5.705633961013518, + 5.848364240955561, + 5.994617699994706, + 6.138369610067457, + 6.283426707028411, + 6.435278280056082, + 6.57842407503631, + 6.725754386046901, + 6.874917719047517, + 7.023030622978695, + 7.173883532988839, + 7.314984800061211, + 7.45956455799751, + 7.600294725969434, + 7.737219676026143, + 7.884251937037334, + 8.030518379062414, + 8.178111933055334, + 8.322873851051554, + 8.46856159705203, + 8.60800043202471, + 8.750297924038023, + 8.897444613045081, + 9.045849157962948, + 9.195840494008735, + 9.345402925042436, + 9.486277524963953, + 9.627745975973085, + 9.77192158298567, + 9.918568979017437, + 10.06671905296389, + 10.214175328030251, + 10.358363206963986, + 10.504849425982684, + 10.651908972999081, + 10.802053992985748, + 10.948331704013981, + 11.092751716962084, + 11.238134391023777, + 11.375735072069801, + 11.507998910034075, + 11.622338633053005, + 11.743439403013326, + 11.85917488404084, + 11.97643095604144, + 12.095495218061842, + 12.211773919989355, + 12.333341664052568, + 12.458472938043997, + 12.57854972104542, + 12.686381075996906, + 12.766596361994743, + 12.830411156988703, + 12.886657619965263, + 12.946701726061292, + 13.001912771025673, + 13.067470101988874, + 13.153040579985827, + 13.22621249803342, + 13.281687575043179, + 13.331327102030627 + ], + "chunk_gaps_ms": { + "count": 96, + "p50": 142.3472670139745, + "p95": 149.5624310337007, + "max": 151.85157302767038 + } + }, + { + "start": 723547.32260198, + "first": 723547.599532836, + "last": 723560.088459841, + "end": 723560.088792023, + "elapsed_s": 12.766190043068491, + "chunk_offsets_s": [ + 0.2769308560527861, + 0.3601024131057784, + 0.577527686022222, + 0.8089551629964262, + 0.9392074350034818, + 1.0775432320078835, + 1.2104198019951582, + 1.3372076130472124, + 1.4666590890847147, + 1.5971695380285382, + 1.7363399340538308, + 1.8721376130124554, + 2.015395751106553, + 2.1609403180191293, + 2.302390238037333, + 2.4396709440043196, + 2.5710034189978614, + 2.705392492003739, + 2.842687718104571, + 2.981739646056667, + 3.124063818017021, + 3.266546391067095, + 3.4039641011040658, + 3.5482805180363357, + 3.693535861093551, + 3.8402474640170112, + 3.985454637091607, + 4.130866221035831, + 4.271460914053023, + 4.4099023850867525, + 4.548461389029399, + 4.690976398997009, + 4.834909521043301, + 4.9803535650717095, + 5.123519681044854, + 5.272904371027835, + 5.418679665075615, + 5.561139760073274, + 5.70476294006221, + 5.84754780807998, + 5.993752459064126, + 6.137439824058674, + 6.2827320960350335, + 6.434588696109131, + 6.5777298010652885, + 6.725141801056452, + 6.874195927055553, + 7.022239564103074, + 7.17339064006228, + 7.314623717102222, + 7.458905054023489, + 7.599416502052918, + 7.736561436089687, + 7.883594994083978, + 8.029707805020735, + 8.177426831098273, + 8.322165842051618, + 8.467604694073088, + 8.607068405020982, + 8.749591130996123, + 8.896633527008817, + 9.045129047008231, + 9.194872673018835, + 9.344924631994218, + 9.485492453095503, + 9.627041761064902, + 9.77123305608984, + 9.917888103052974, + 10.066103363060392, + 10.213560726027936, + 10.357680090004578, + 10.50415981002152, + 10.651257952093147, + 10.800993837998249, + 10.947562688030303, + 11.092067528050393, + 11.237395774107426, + 11.375031641102396, + 11.507304284023121, + 11.621673493995331, + 11.742760961060412, + 11.858778904075734, + 11.97591826110147, + 12.094758490100503, + 12.211147168069147, + 12.33273010305129, + 12.457962412037887, + 12.578149196109734, + 12.68513536406681, + 12.765857861028053 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 143.1411049561575, + "p95": 150.05195897538215, + "max": 231.42747697420418 + } + }, + { + "start": 723547.321923168, + "first": 723547.433439306, + "last": 723560.007849762, + "end": 723560.008036411, + "elapsed_s": 12.686113242991269, + "chunk_offsets_s": [ + 0.11151613795664161, + 0.1696286229416728, + 0.3609302929835394, + 0.5780366510152817, + 0.8103100809967145, + 0.9399588130181655, + 1.0779059779597446, + 1.2112567429430783, + 1.338215120951645, + 1.4675850669154897, + 1.5977626739768311, + 1.7368580549955368, + 1.872953807003796, + 2.0165885529713705, + 2.1615496550220996, + 2.303169180988334, + 2.4402634719153866, + 2.571741605992429, + 2.7061782230157405, + 2.843443016987294, + 2.9827475369675085, + 3.125168142025359, + 3.267601346014999, + 3.4049065669532865, + 3.549282389925793, + 3.694405985996127, + 3.8410702369874343, + 3.9864036979852244, + 4.131950711016543, + 4.272430305019952, + 4.410968289943412, + 4.5494988629361615, + 4.69181791797746, + 4.836076149018481, + 4.981517181964591, + 5.124457889003679, + 5.27373879798688, + 5.41950391093269, + 5.561999287921935, + 5.705700075021014, + 5.84844982100185, + 5.9946993589401245, + 6.1385359349660575, + 6.283593929023482, + 6.435484737972729, + 6.5786035109777, + 6.725920710945502, + 6.875095009920187, + 7.023200261988677, + 7.17391327698715, + 7.315168494009413, + 7.459731603972614, + 7.600330456974916, + 7.737389778951183, + 7.884416228975169, + 8.03058396501001, + 8.17827822593972, + 8.323108242941089, + 8.468452390981838, + 8.607924249954522, + 8.750463256030343, + 8.897670071921311, + 9.046012409962714, + 9.195755747961812, + 9.345367708941922, + 9.48644513101317, + 9.62783093296457, + 9.772100714966655, + 9.918628945946693, + 10.066887379973195, + 10.214343381929211, + 10.358529628021643, + 10.505014357971959, + 10.652072015916929, + 10.802227264968678, + 10.948518406949006, + 11.092915753019042, + 11.238302252953872, + 11.375904071028344, + 11.50818138802424, + 11.622510240995325, + 11.743610034929588, + 11.859366582008079, + 11.976428732974455, + 12.095538744935766, + 12.211970677017234, + 12.333526846021414, + 12.458874046918936, + 12.579208523966372, + 12.685926593956538 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 143.11877300497144, + "p95": 150.71301499847323, + "max": 232.27342998143286 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 82.1, + "per_stream_tps": 13.8, + "ttft_s_max": 17.29, + "timing_valid": true, + "end_to_end_tps": 82.07306424255029, + "makespan_s": 24.953375615994446, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 82.11660373947991, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 24.953375615994446, + "end_to_end_tps": 82.07306424255029, + "timing_valid": true, + "per_stream_tps": [ + 15.9, + 13.1, + 13.7, + 13.8, + 15.5, + 33.3, + 13.6, + 12.7 + ], + "ttft_s": [ + 1.16, + 0.11, + 0.46, + 0.68, + 1.43, + 17.29, + 0.91, + 0.27 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0b4d014c8872", + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b82191a9a6ec", + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "3f4413b23169", + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "51fe79c071ab", + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e9b7a497dd30", + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "097c9fe5896b", + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05" + } + ], + "client_intervals": [ + { + "start": 723560.659959985, + "first": 723561.822105965, + "last": 723577.842451743, + "end": 723577.842842664, + "elapsed_s": 17.18288267904427, + "chunk_offsets_s": [ + 1.1621459800517187, + 1.3225410000886768, + 1.6108766740653664, + 1.799073604051955, + 1.990711924037896, + 2.1852893120376393, + 2.384478764026426, + 2.5797582590021193, + 2.7684129220433533, + 2.9599927610252053, + 3.161830657045357, + 3.3648621370084584, + 3.5655982800526544, + 3.770536682102829, + 3.9774167680880055, + 4.173775163013488, + 4.3713070480152965, + 4.570980710093863, + 4.777776075061411, + 4.977977593080141, + 5.184065571054816, + 5.387730052112602, + 5.59318599710241, + 5.800503370002843, + 6.0148477060720325, + 6.222002912079915, + 6.4437916880706325, + 6.646167169092223, + 6.848664307035506, + 7.052589874016121, + 7.258000421104953, + 7.474954198114574, + 7.677634823019616, + 7.882548706023954, + 8.088466128101572, + 8.300536908092909, + 8.521882169065066, + 8.729983127093874, + 8.939961926080287, + 9.141458236030303, + 9.351685355999507, + 9.573249353095889, + 9.784213817096315, + 9.996955688111484, + 10.20475015300326, + 10.408294865046628, + 10.636589450063184, + 10.844975970103405, + 11.04742638010066, + 11.263087660074234, + 11.467131450073794, + 11.673018333036453, + 11.882525862078182, + 12.099317636108026, + 12.318447480094619, + 12.527373942080885, + 12.733060604077764, + 12.938750502071343, + 13.148346644011326, + 13.368802193086594, + 13.583843777072616, + 13.797942122095264, + 14.012527345097624, + 14.223869121051393, + 14.445372710004449, + 14.659315268043429, + 14.870755655108951, + 15.081566019100137, + 15.29793625511229, + 15.527898997068405, + 15.741033207043074, + 15.953966449014843, + 16.16199385910295, + 16.36909865809139, + 16.58446508506313, + 16.785014843102545, + 16.98728626302909, + 17.18249175802339 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 207.10479898843914, + "p95": 221.7887759907171, + "max": 288.3356739766896 + } + }, + { + "start": 723560.658517336, + "first": 723560.768412431, + "last": 723580.256658941, + "end": 723580.256762573, + "elapsed_s": 19.59824523702264, + "chunk_offsets_s": [ + 0.10989509499631822, + 0.1661431749816984, + 0.3527311560465023, + 0.5699642079416662, + 0.8041414689505473, + 1.0560482590226457, + 1.3235500450246036, + 1.611713337013498, + 1.7997184409759939, + 1.992038490017876, + 2.1860051619587466, + 2.3856362169608474, + 2.5805820720270276, + 2.769166095997207, + 2.9612751529784873, + 3.162590799969621, + 3.365677680936642, + 3.566945343045518, + 3.771481800940819, + 3.9781605020398274, + 4.17472790600732, + 4.372883975040168, + 4.57169504603371, + 4.778902204008773, + 4.978966508992016, + 5.185020947013982, + 5.3886421519564465, + 5.594057267997414, + 5.8013940720120445, + 6.016017955029383, + 6.222975488984957, + 6.445102390018292, + 6.64702438598033, + 6.849546578945592, + 7.053721523960121, + 7.258916493039578, + 7.47569615195971, + 7.678281944943592, + 7.883307388983667, + 8.089616226963699, + 8.301367496955208, + 8.522966388962232, + 8.730882424977608, + 8.941539008985274, + 9.142770624021068, + 9.352689889026806, + 9.574200029019266, + 9.785265868995339, + 9.99801204202231, + 10.205765601014718, + 10.409173646941781, + 10.63749233703129, + 10.846082904026844, + 11.04878103802912, + 11.263967468985356, + 11.468095330987126, + 11.67394579097163, + 11.883191189961508, + 12.100181423942558, + 12.31921740504913, + 12.52815291902516, + 12.734429216943681, + 12.939652779023163, + 13.14942098397296, + 13.369823673041537, + 13.584904632996768, + 13.798727773944847, + 14.013745988951996, + 14.224724638042971, + 14.446397145977244, + 14.66124252602458, + 14.871639988035895, + 15.082849930040538, + 15.29899935098365, + 15.529206758015789, + 15.742706084973179, + 15.954956765985116, + 16.162750603980385, + 16.369987832964398, + 16.58533117803745, + 16.785860536037944, + 16.988799807964824, + 17.1835110289976, + 17.4919276740402, + 17.711870020022616, + 17.914420022978447, + 18.09912309993524, + 18.285569192958064, + 18.4652081199456, + 18.656518177012913, + 18.858293427038006, + 19.032402254990302, + 19.18737942504231, + 19.330748089938425, + 19.46805578994099, + 19.59814160503447 + ], + "chunk_gaps_ms": { + "count": 95, + "p50": 206.30883798003197, + "p95": 234.17726100888103, + "max": 308.41664504259825 + } + }, + { + "start": 723560.658246161, + "first": 723561.117841124, + "last": 723579.691569248, + "end": 723579.692059491, + "elapsed_s": 19.03381332999561, + "chunk_offsets_s": [ + 0.45959496300201863, + 0.5703587470343336, + 0.8042785139987245, + 1.0559609580086544, + 1.3235639050835744, + 1.6114911400945857, + 1.7997915740124881, + 1.9919423970859498, + 2.1864870850695297, + 2.3858863110654056, + 2.580831383005716, + 2.7694664100417867, + 2.9616160340374336, + 3.1629044130677357, + 3.3661352950148284, + 3.5677387040341273, + 3.7718167731072754, + 3.9785468840273097, + 4.174960812088102, + 4.372810280066915, + 4.572144411038607, + 4.779222937999293, + 4.979329096036963, + 5.18541581009049, + 5.389024627045728, + 5.5944374080281705, + 5.801621297025122, + 6.016886866069399, + 6.223361680051312, + 6.44515358901117, + 6.647383228060789, + 6.8501655120635405, + 7.054161173058674, + 7.259484629030339, + 7.476148225017823, + 7.678728993050754, + 7.884094029082917, + 8.0899956850335, + 8.301740096067078, + 8.523333327029832, + 8.731197159038857, + 8.941888299072161, + 9.143131171003915, + 9.353070745011792, + 9.574620590079576, + 9.786172313033603, + 9.998383597005159, + 10.20616039109882, + 10.409604211105034, + 10.637830376042984, + 10.846520401071757, + 11.049425174016505, + 11.26439401099924, + 11.468416771036573, + 11.674346336047165, + 11.883614142076112, + 12.100411049090326, + 12.319539037998766, + 12.528480965993367, + 12.735094740055501, + 12.939966130070388, + 13.149623545003124, + 13.370038005057722, + 13.585248181014322, + 13.799056082032621, + 14.0140577100683, + 14.225302414037287, + 14.446745895082131, + 14.661268094088882, + 14.872005311073735, + 15.082580671994947, + 15.29935479001142, + 15.52937641099561, + 15.742938774055801, + 15.955167156062089, + 16.16345238906797, + 16.370689554023556, + 16.58602149307262, + 16.786044434993528, + 16.989463905105367, + 17.18416247109417, + 17.492637120070867, + 17.71257544902619, + 17.915827192016877, + 18.09981896309182, + 18.286171393003315, + 18.465925739030354, + 18.6572862660978, + 18.858679672004655, + 19.03332308703102 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 206.73011092003435, + "p95": 233.91976696439087, + "max": 308.4746489766985 + } + }, + { + "start": 723560.659322426, + "first": 723561.335211577, + "last": 723579.845970318, + "end": 723579.846298538, + "elapsed_s": 19.18697611207608, + "chunk_offsets_s": [ + 0.6758891510544345, + 0.8035516630625352, + 1.054972285986878, + 1.3231032070470974, + 1.610726059996523, + 1.7990041649900377, + 1.9911230549914762, + 2.185859296005219, + 2.385171672096476, + 2.5804329779930413, + 2.7690831690561026, + 2.9609635090455413, + 3.1624328270554543, + 3.3652465880149975, + 3.5664751530857757, + 3.7712152920430526, + 3.9778384590754285, + 4.174379056086764, + 4.3719931551022455, + 4.571455697063357, + 4.778484135051258, + 4.978580525028519, + 5.184574693092145, + 5.388267791015096, + 5.593727621017024, + 5.801192217040807, + 6.015423744102009, + 6.222858113003895, + 6.444488994078711, + 6.646668950095773, + 6.849205709993839, + 7.053145011072047, + 7.258531414088793, + 7.475276026059873, + 7.678380058030598, + 7.882820945000276, + 8.089025864028372, + 8.301377330091782, + 8.522677516099066, + 8.730772870010696, + 8.940447780070826, + 9.14244021999184, + 9.352276989025995, + 9.573799489997327, + 9.785002944990993, + 9.997811692999676, + 10.205746453022584, + 10.408837369061075, + 10.637393475044519, + 10.8460109900916, + 11.048445168067701, + 11.263912779046223, + 11.467730179079808, + 11.673983642016537, + 11.88296296109911, + 12.099491150001995, + 12.318638555007055, + 12.527909048018046, + 12.733429464045912, + 12.939319127006456, + 13.14902825106401, + 13.369476650026627, + 13.584669905016199, + 13.798471732996404, + 14.013120983028784, + 14.223959055030718, + 14.445504741044715, + 14.659896203083917, + 14.870877717039548, + 15.081440346082672, + 15.298097090097144, + 15.52825265598949, + 15.74192270101048, + 15.954303493024781, + 16.16199175803922, + 16.369347613072023, + 16.584708857000805, + 16.78572944109328, + 16.98757729108911, + 17.182788752019405, + 17.49123350402806, + 17.711231795023195, + 17.91407188703306, + 18.09840876108501, + 18.284838601015508, + 18.46445319603663, + 18.65577045409009, + 18.857530891080387, + 19.031840902054682, + 19.18664789199829 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 206.62316703237593, + "p95": 230.1555658923462, + "max": 308.4447520086542 + } + }, + { + "start": 723560.658759397, + "first": 723562.088199545, + "last": 723578.573658855, + "end": 723578.573769762, + "elapsed_s": 17.915010364959016, + "chunk_offsets_s": [ + 1.4294401480583474, + 1.6118491239612922, + 1.7999411909841, + 1.9919941669795662, + 2.186252939980477, + 2.3856369180139154, + 2.5806570290587842, + 2.7695497820386663, + 2.961436051991768, + 3.162800131016411, + 3.3661191010614857, + 3.567102950066328, + 3.771663513034582, + 3.978659973014146, + 4.1748888220172375, + 4.372861362062395, + 4.571874122018926, + 4.77887684199959, + 4.979092588997446, + 5.18522392201703, + 5.389010893064551, + 5.594418350956403, + 5.801661366014741, + 6.016277175047435, + 6.223158783046529, + 6.444767183973454, + 6.647509258007631, + 6.849910032004118, + 7.053393427049741, + 7.259328968008049, + 7.475903409998864, + 7.67860936303623, + 7.883661270025186, + 8.089633809053339, + 8.301573983975686, + 8.522631554980762, + 8.731003897963092, + 8.941211860976182, + 9.14303806098178, + 9.35291469504591, + 9.574404702987522, + 9.785599460010417, + 9.998195008025505, + 10.206094490014948, + 10.40935397404246, + 10.637460961006582, + 10.846500821062364, + 11.048971654032357, + 11.264511598972604, + 11.46818514703773, + 11.674413847038522, + 11.883645556052215, + 12.099852008046582, + 12.319084960967302, + 12.5278136030538, + 12.73382722795941, + 12.939356098067947, + 13.14905748097226, + 13.369472819962539, + 13.584556051995605, + 13.798446574015543, + 14.013446522993036, + 14.224958578008227, + 14.446193951996975, + 14.660821110010147, + 14.871547227026895, + 15.082309685065411, + 15.298712456016801, + 15.529023815994151, + 15.742375134956092, + 15.955421819002368, + 16.162614947999828, + 16.36985201702919, + 16.585200160043314, + 16.786072727059945, + 16.9892356779892, + 17.183445984032005, + 17.49185207206756, + 17.711735907010734, + 17.914899458060972 + ], + "chunk_gaps_ms": { + "count": 79, + "p50": 207.00271998066455, + "p95": 221.60840092692524, + "max": 308.4060880355537 + } + }, + { + "start": 723560.66129632, + "first": 723577.952278666, + "last": 723585.611134939, + "end": 723585.611477352, + "elapsed_s": 24.950181031948887, + "chunk_offsets_s": [ + 17.290982345934026, + 17.490200408967212, + 17.709624263923615, + 17.91228805296123, + 18.097030386910774, + 18.2834197540069, + 18.4631178380223, + 18.654433883959427, + 18.855060850968584, + 19.030314521980472, + 19.1847827329766, + 19.328329176991247, + 19.465825423947535, + 19.595860781962983, + 19.690210733911954, + 19.78849565796554, + 19.88682729692664, + 19.97964273393154, + 20.071045361924917, + 20.163427110994235, + 20.257397135021165, + 20.343965968932025, + 20.410894757020287, + 20.47673328500241, + 20.54555150901433, + 20.609144478919916, + 20.666309060994536, + 20.72613463702146, + 20.787037653964944, + 20.84894136991352, + 20.9150528809987, + 20.98058806802146, + 21.040634564007632, + 21.091573587968014, + 21.143227563006803, + 21.194900767994113, + 21.245749593013898, + 21.29938439000398, + 21.35806939401664, + 21.415821276023053, + 21.475403468008153, + 21.536526239011437, + 21.601524167926982, + 21.665751157910563, + 21.719389009987935, + 21.778766711009666, + 21.839146208018064, + 21.89721493702382, + 21.956140144960955, + 22.021505019976757, + 22.086145726963878, + 22.15083375899121, + 22.21611523302272, + 22.282531835953705, + 22.346189338015392, + 22.411836336017586, + 22.47901027300395, + 22.544835145003162, + 22.609835378010757, + 22.675561498967, + 22.73815924092196, + 22.803829269018024, + 22.867070912965573, + 22.93049317097757, + 22.99572664592415, + 23.06021496001631, + 23.1248238759581, + 23.189619438024238, + 23.253050816012546, + 23.320127025945112, + 23.38614262000192, + 23.45192680193577, + 23.51998503995128, + 23.583259609993547, + 23.647230324917473, + 23.70850646100007, + 23.765312604955398, + 23.8176846599672, + 23.86979467398487, + 23.92191473697312, + 23.97241963201668, + 24.023898225976154, + 24.0760525589576, + 24.127333062933758, + 24.17875652294606, + 24.231777405948378, + 24.283530839020386, + 24.33559442393016, + 24.387765523977578, + 24.438181334990077, + 24.48951230093371, + 24.542209321982227, + 24.593913157004863, + 24.644544719019905, + 24.69618124992121, + 24.74760558991693, + 24.798535184003413, + 24.848988062003627, + 24.90067320095841, + 24.94983861898072 + ], + "chunk_gaps_ms": { + "count": 99, + "p50": 63.59296990558505, + "p95": 191.31604593712837, + "max": 219.42385495640337 + } + }, + { + "start": 723560.658887459, + "first": 723561.570319179, + "last": 723580.257322772, + "end": 723580.257453312, + "elapsed_s": 19.598565853084438, + "chunk_offsets_s": [ + 0.9114317200146616, + 1.0555752160726115, + 1.3237803070805967, + 1.611666625016369, + 1.799878626014106, + 1.9918218010570854, + 2.186328770010732, + 2.3857798280660063, + 2.580901477020234, + 2.7692423299886286, + 2.961363533977419, + 3.162799222045578, + 3.365809076000005, + 3.5671588809927925, + 3.771579522988759, + 3.978390472009778, + 4.1748867890564725, + 4.372775437077507, + 4.571970607037656, + 4.7790559630375355, + 4.978926383075304, + 5.185058771050535, + 5.389013346051797, + 5.59422488801647, + 5.8014753960305825, + 6.016344900010154, + 6.2234139640349895, + 6.444812174071558, + 6.647338170092553, + 6.849841783987358, + 7.0537314399844036, + 7.259448861004785, + 7.47612729400862, + 7.6784206830197945, + 7.88358587899711, + 8.0895798120182, + 8.301658315001987, + 8.52280148002319, + 8.731245895032771, + 8.94129339104984, + 9.142777739092708, + 9.35282214905601, + 9.574357267003506, + 9.785615755012259, + 9.997992153046653, + 10.205924657988362, + 10.409312491072342, + 10.637425031978637, + 10.846481875982136, + 11.048918692977168, + 11.264263081015088, + 11.468105094041675, + 11.674132106010802, + 11.88346794701647, + 12.099971906049177, + 12.319190961075947, + 12.528390828054398, + 12.734168781084009, + 12.939866196014918, + 13.149498149054125, + 13.369690068997443, + 13.585295120021328, + 13.798815579037182, + 14.013684099074453, + 14.224974308046512, + 14.446187897003256, + 14.660436931066215, + 14.871670435066335, + 15.082320574088953, + 15.298888575984165, + 15.529054459067993, + 15.742044580052607, + 15.955109404982068, + 16.16286438901443, + 16.37037252506707, + 16.585502324043773, + 16.78621294198092, + 16.98822973005008, + 17.183760496089235, + 17.49246525403578, + 17.71237621200271, + 17.914423846057616, + 18.099220313015394, + 18.285864960053004, + 18.465457144076936, + 18.65675019903574, + 18.85758692503441, + 19.032751602004282, + 19.187585484003648, + 19.33066233806312, + 19.468477223068476, + 19.598435313091613 + ], + "chunk_gaps_ms": { + "count": 91, + "p50": 206.02701196912676, + "p95": 228.11254090629518, + "max": 308.7047579465434 + } + }, + { + "start": 723560.658101736, + "first": 723560.931846869, + "last": 723581.005129442, + "end": 723581.005275328, + "elapsed_s": 20.34717359195929, + "chunk_offsets_s": [ + 0.2737451329594478, + 0.35324157995637506, + 0.5704618640011176, + 0.8043866129592061, + 1.0559651789953932, + 1.3236756169935688, + 1.6115481239976361, + 1.7998839750653133, + 1.9921516160247847, + 2.1864751670509577, + 2.3859732940327376, + 2.5809325110167265, + 2.7695477610686794, + 2.962396163959056, + 3.1630699529778212, + 3.366966638015583, + 3.567841234966181, + 3.7722298819571733, + 3.9787309679668397, + 4.175203131977469, + 4.373450973071158, + 4.572248856071383, + 4.7797943180194125, + 4.979534126003273, + 5.185487512033433, + 5.389115032972768, + 5.594638937036507, + 5.8018806100590155, + 6.016319610062055, + 6.223558177007362, + 6.445478852023371, + 6.6475851649884135, + 6.850058719981462, + 7.054603743017651, + 7.259399374015629, + 7.4762250010389835, + 7.678782899049111, + 7.883819664013572, + 8.090087822987698, + 8.302146419067867, + 8.523184402962215, + 8.731243015965447, + 8.941760673071258, + 9.143238145974465, + 9.35316390299704, + 9.57472673302982, + 9.785952048026957, + 9.998480687034316, + 10.206372044980526, + 10.409686955972575, + 10.638573240023106, + 10.847330896998756, + 11.049932676018216, + 11.2648112840252, + 11.468772793072276, + 11.674793849000707, + 11.884055785019882, + 12.100978175993077, + 12.320089804008603, + 12.528927727020346, + 12.73514523205813, + 12.940449323039502, + 13.150117038050666, + 13.370512994006276, + 13.58565298805479, + 13.79948858707212, + 14.014297847053967, + 14.225686119985767, + 14.447297479957342, + 14.661009997013025, + 14.8723999990616, + 15.083330727065913, + 15.299624506034888, + 15.529795282986015, + 15.742939459043555, + 15.95599350205157, + 16.163983579026535, + 16.37132780405227, + 16.586255655041896, + 16.786649046000093, + 16.98868744506035, + 17.184656450990587, + 17.492946101003326, + 17.713332319050096, + 17.91493226098828, + 18.100340084987693, + 18.286688180058263, + 18.4661719360156, + 18.657690569991246, + 18.85864767699968, + 19.033249338972382, + 19.188531180960126, + 19.331329504027963, + 19.46887052699458, + 19.598962037009187, + 19.693624873063527, + 19.791859645047225, + 19.89006123598665, + 19.982964230002835, + 20.074198623071425, + 20.166584532009438, + 20.26063264196273, + 20.347027705982327 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 205.7386450469494, + "p95": 230.17077695112675, + "max": 308.289650012739 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 35.1, + "per_stream_tps": 35.1, + "ttft_s_max": 0.16, + "timing_valid": true, + "end_to_end_tps": 34.45920140316111, + "makespan_s": 7.429075241903774, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 35.08758626234495, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 7.429075241903774, + "end_to_end_tps": 34.45920140316111, + "timing_valid": true, + "per_stream_tps": [ + 35.1 + ], + "ttft_s": [ + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + } + ], + "client_intervals": [ + { + "start": 723738.180132396, + "first": 723738.341484645, + "last": 723745.6090122, + "end": 723745.609207638, + "elapsed_s": 7.429075241903774, + "chunk_offsets_s": [ + 0.16135224897880107, + 0.22485386894550174, + 0.28459466889034957, + 0.3368964719120413, + 0.39141539100091904, + 0.4483682638965547, + 0.4992641039425507, + 0.5514859169488773, + 0.6030751049984246, + 0.6545769319636747, + 0.7047991699073464, + 0.7565872099949047, + 0.8080379529856145, + 0.8607670209603384, + 0.9124988449038938, + 0.9628241459140554, + 1.0146765079116449, + 1.0653487798990682, + 1.1166516139637679, + 1.1678392729954794, + 1.2195843909867108, + 1.2698997809784487, + 1.3225080859847367, + 1.3767461410025135, + 1.4357027169317007, + 1.491193819907494, + 1.5429471558891237, + 1.5969022529898211, + 1.65422129095532, + 1.7115017799660563, + 1.7706725778989494, + 1.8293954259715974, + 1.889620110974647, + 1.9530865509295836, + 2.006006945972331, + 2.0636515859514475, + 2.1225354549242184, + 2.1805768779013306, + 2.238998434972018, + 2.2916413109051064, + 2.344082240946591, + 2.3954746779054403, + 2.446562292985618, + 2.496897619916126, + 2.549939688993618, + 2.600447877892293, + 2.6551392739638686, + 2.715530873974785, + 2.773885642993264, + 2.8301455869805068, + 2.8815415099961683, + 2.9349037829088047, + 2.99384187895339, + 3.05228151998017, + 3.1034186449833214, + 3.1551497819600627, + 3.2090246978914365, + 3.2667788899270818, + 3.3242066520033404, + 3.3768868600018322, + 3.4281413418939337, + 3.479827754897997, + 3.5308966279262677, + 3.5818563379580155, + 3.632917407900095, + 3.6837265589274466, + 3.735453457920812, + 3.7860345949884504, + 3.837789124925621, + 3.888690839987248, + 3.939681874937378, + 3.991065537906252, + 4.042977901990525, + 4.093607019982301, + 4.145356873981655, + 4.198019695933908, + 4.25680830294732, + 4.315354548976757, + 4.376439692918211, + 4.436623846995644, + 4.497959699947387, + 4.550293765962124, + 4.601843216922134, + 4.653032731963322, + 4.704352293978445, + 4.756186975981109, + 4.8074612859636545, + 4.8588553939480335, + 4.9097777309361845, + 4.962218754924834, + 5.015147202997468, + 5.067145174951293, + 5.119024685933255, + 5.1693626859923825, + 5.221440908964723, + 5.272011790890247, + 5.322779226931743, + 5.374018198926933, + 5.427108990959823, + 5.487806266988628, + 5.5485465249512345, + 5.609990530996583, + 5.6713737739482895, + 5.737171294982545, + 5.801017761928961, + 5.8541721479268745, + 5.914232449955307, + 5.97253548388835, + 6.031400688923895, + 6.081378027913161, + 6.132743002963252, + 6.184089893940836, + 6.2356113529531285, + 6.288325399975292, + 6.341336342971772, + 6.393494191928767, + 6.443943120888434, + 6.497301280964166, + 6.555328155984171, + 6.607023719931021, + 6.658518586889841, + 6.710812227916904, + 6.762636994943023, + 6.814353138906881, + 6.866602981928736, + 6.917817982961424, + 6.969102191971615, + 7.019934076932259, + 7.072282852954231, + 7.124472082941793, + 7.1757117349188775, + 7.228993092896417, + 7.279729037894867, + 7.331853454001248, + 7.382345812977292, + 7.428879803977907 + ], + "chunk_gaps_ms": { + "count": 135, + "p50": 52.12441610638052, + "p95": 61.33585295174271, + "max": 65.79752103425562 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 53.3, + "per_stream_tps": 27.8, + "ttft_s_max": 0.38, + "timing_valid": true, + "end_to_end_tps": 52.575569935379995, + "makespan_s": 9.738363286014646, + "steady_tps": 55.8, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 53.254263802741555, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 9.738363286014646, + "end_to_end_tps": 52.575569935379995, + "timing_valid": true, + "steady_tps": 55.756486407484346, + "per_stream_tps": [ + 26.6, + 28.9 + ], + "ttft_s": [ + 0.16, + 0.38 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + } + ], + "client_intervals": [ + { + "start": 723745.611858655, + "first": 723745.773182685, + "last": 723755.349880654, + "end": 723755.350030708, + "elapsed_s": 9.738172053010203, + "chunk_offsets_s": [ + 0.16132403002120554, + 0.22255941305775195, + 0.4621854319702834, + 0.5540986149571836, + 0.64938631397672, + 0.7421109339920804, + 0.8343563739908859, + 0.9253603320103139, + 1.0183057880494744, + 1.111608335049823, + 1.205070630996488, + 1.299247078015469, + 1.3942371240118518, + 1.489195268950425, + 1.5802541399607435, + 1.676872348994948, + 1.7759174760431051, + 1.8755986449541524, + 1.9741204190067947, + 2.069256435963325, + 2.165733697009273, + 2.260922963032499, + 2.355205732048489, + 2.454400310991332, + 2.5480552569497377, + 2.646278879023157, + 2.7421717919642106, + 2.8364110209513456, + 2.9302998690400273, + 3.0252242219867185, + 3.119209159980528, + 3.216308129951358, + 3.3128046019701287, + 3.4101612040540203, + 3.507842650054954, + 3.6047757599735633, + 3.7009339779615402, + 3.7989210470113903, + 3.8962552109733224, + 3.9937710190424696, + 4.091984860016964, + 4.186770964995958, + 4.2826658060075715, + 4.3790688960580155, + 4.47621850494761, + 4.573681381996721, + 4.671665793051943, + 4.766442280029878, + 4.861459534033202, + 4.956060020951554, + 5.050977329025045, + 5.143047395977192, + 5.233425628975965, + 5.325890091015026, + 5.421653643948957, + 5.513835474965163, + 5.609353177016601, + 5.706776313949376, + 5.80106623400934, + 5.896655822987668, + 5.992043777019717, + 6.088062320952304, + 6.179410620010458, + 6.275024133035913, + 6.367116449051537, + 6.460941178957, + 6.556200194056146, + 6.654399640043266, + 6.7491228020517156, + 6.846541946055368, + 6.943627721979283, + 7.0376070940401405, + 7.133620423963293, + 7.231281628948636, + 7.329224859015085, + 7.425812722998671, + 7.520637916051783, + 7.613201822037809, + 7.708480625995435, + 7.804433393990621, + 7.899285795050673, + 7.9929612670093775, + 8.086275437031873, + 8.17837465996854, + 8.272502155974507, + 8.370276664034463, + 8.468115031020716, + 8.561712661990896, + 8.652978798025288, + 8.74658122798428, + 8.84223789605312, + 8.93827435595449, + 9.03283412300516, + 9.127216530963778, + 9.214660804020241, + 9.277553120977245, + 9.327920106006786, + 9.379683095030487, + 9.432460585027002, + 9.484946675016545, + 9.536721586016938, + 9.587637571967207, + 9.639378154999577, + 9.69033519900404, + 9.738021999015473 + ], + "chunk_gaps_ms": { + "count": 104, + "p50": 94.92083051009104, + "p95": 98.22362207341939, + "max": 239.62601891253144 + } + }, + { + "start": 723745.611667422, + "first": 723745.99049664, + "last": 723754.826479937, + "end": 723754.826648986, + "elapsed_s": 9.214981563971378, + "chunk_offsets_s": [ + 0.3788292179815471, + 0.462578131002374, + 0.5544172619702294, + 0.6497967629693449, + 0.7423683210508898, + 0.8343844360206276, + 0.9256266900338233, + 1.018302668002434, + 1.1118420399725437, + 1.2053136579925194, + 1.299390819040127, + 1.3944743949687108, + 1.4894630019553006, + 1.580502780037932, + 1.6771228449651971, + 1.775919695966877, + 1.875842040986754, + 1.974075224949047, + 2.0695178529713303, + 2.165994458948262, + 2.261359018040821, + 2.355458306032233, + 2.4546366869471967, + 2.54828925000038, + 2.646289622061886, + 2.742167565971613, + 2.836411032010801, + 2.9305539949564263, + 3.025217117043212, + 3.1192370919743553, + 3.2162964349845424, + 3.3128171500284225, + 3.41015489900019, + 3.508093626005575, + 3.604901988990605, + 3.7010765969753265, + 3.7991648910101503, + 3.896384494029917, + 3.9939152860315517, + 4.0922385710291564, + 4.1870132740587, + 4.282783639035188, + 4.379321935004555, + 4.476469464949332, + 4.573921867995523, + 4.671799761010334, + 4.766698166029528, + 4.861711374018341, + 4.956065659062006, + 5.050986403948627, + 5.1430056360550225, + 5.233690902008675, + 5.3258838469628245, + 5.421655811020173, + 5.513854608987458, + 5.609491606010124, + 5.706905515980907, + 5.801361219026148, + 5.896800745045766, + 5.992282505030744, + 6.088193041970953, + 6.179644071031362, + 6.275274342973717, + 6.367394339991733, + 6.461195289972238, + 6.556449556024745, + 6.6546511319465935, + 6.749254622962326, + 6.846803282038309, + 6.94386682996992, + 7.037840240984224, + 7.1338788360590115, + 7.231668949010782, + 7.32959442504216, + 7.42606316099409, + 7.5208899839781225, + 7.613488678005524, + 7.708732872037217, + 7.804558292031288, + 7.899383462034166, + 7.993109032977372, + 8.086525924969465, + 8.178611885057762, + 8.27276120998431, + 8.37052712100558, + 8.468380748992786, + 8.561969575006515, + 8.653243161039427, + 8.746817272040062, + 8.842368112993427, + 8.938527691992931, + 9.0330892970087, + 9.12748044601176, + 9.214812514954247 + ], + "chunk_gaps_ms": { + "count": 93, + "p50": 95.24419403169304, + "p95": 98.23318396229297, + "max": 99.92234501987696 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.3, + "per_stream_tps": 18.3, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 68.78482603081082, + "makespan_s": 14.887004286982119, + "steady_tps": 73.2, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.27583609119345, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 14.887004286982119, + "end_to_end_tps": 68.78482603081082, + "timing_valid": true, + "steady_tps": 73.20910326028157, + "per_stream_tps": [ + 18.3, + 18.9, + 17.6, + 18.3 + ], + "ttft_s": [ + 0.89, + 0.16, + 0.38, + 0.63 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0a24c104609f", + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "28cedcb814fc", + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767" + } + ], + "client_intervals": [ + { + "start": 723755.352303088, + "first": 723756.241662687, + "last": 723770.187849424, + "end": 723770.188101553, + "elapsed_s": 14.835798465064727, + "chunk_offsets_s": [ + 0.8893595990957692, + 1.010279196081683, + 1.1367503580404446, + 1.2587723151082173, + 1.381514938082546, + 1.5138889790978283, + 1.6460699510062113, + 1.7788600320927799, + 1.914682780043222, + 2.050377082079649, + 2.186639215098694, + 2.3198054881067947, + 2.456387059064582, + 2.5975350780645385, + 2.7398041510023177, + 2.8812671171035618, + 3.023075925069861, + 3.1648566210642457, + 3.3091442240402102, + 3.4474999440135434, + 3.5912742640357465, + 3.735306912101805, + 3.8769700480625033, + 4.01873248803895, + 4.160907015088014, + 4.3026167540811, + 4.44572657905519, + 4.587328249006532, + 4.726790662040003, + 4.868645745096728, + 5.0098706050775945, + 5.155559829086997, + 5.297993154032156, + 5.436930124065839, + 5.579597736010328, + 5.725669057108462, + 5.868786893086508, + 6.014444410102442, + 6.160838908050209, + 6.306606966070831, + 6.45732206501998, + 6.600185276009142, + 6.748426023055799, + 6.896284055081196, + 7.041441867011599, + 7.185952301020734, + 7.330158451106399, + 7.474690402043052, + 7.6171821020543575, + 7.757395174005069, + 7.8927702840883285, + 8.035253921058029, + 8.177331376005895, + 8.322121791075915, + 8.469090898055583, + 8.611472709104419, + 8.759214945021085, + 8.903463372029364, + 9.041843558079563, + 9.183644436998293, + 9.3239419630263, + 9.46572779910639, + 9.60462418710813, + 9.74944160610903, + 9.888470313046128, + 10.02610638202168, + 10.171966260066256, + 10.316354467067868, + 10.453797165071592, + 10.590488905087113, + 10.727737572044134, + 10.86858003004454, + 11.00556147005409, + 11.147413942031562, + 11.29098198004067, + 11.428928329027258, + 11.566389249055646, + 11.70127752108965, + 11.84030438400805, + 11.979023039108142, + 12.114018514053896, + 12.246934262104332, + 12.377400064026006, + 12.516076062107459, + 12.659395482041873, + 12.800770140020177, + 12.943087123101577, + 13.073217107099481, + 13.211203119019046, + 13.344966305070557, + 13.483497303095646, + 13.626226170104928, + 13.74533630500082, + 13.862543465103954, + 13.980551907094195, + 14.101143458043225, + 14.2255797610851, + 14.351095135090873, + 14.470434930059128, + 14.580399122089148, + 14.675808017025702, + 14.763895065058023, + 14.835546336020343 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 140.99523850018159, + "p95": 146.39449794776738, + "max": 150.71509894914925 + } + }, + { + "start": 723755.352440452, + "first": 723755.514684262, + "last": 723768.977791189, + "end": 723768.977930889, + "elapsed_s": 13.62549043702893, + "chunk_offsets_s": [ + 0.16224381001666188, + 0.223852758994326, + 0.4672896560514346, + 0.7307420580182225, + 1.009782689041458, + 1.1362593340454623, + 1.2583522200584412, + 1.3809669989859685, + 1.5134346020640805, + 1.6456309329951182, + 1.7782741200644523, + 1.9140660989796743, + 2.049927105079405, + 2.1866878940491006, + 2.319332606974058, + 2.4559265490388498, + 2.597021760069765, + 2.739264477044344, + 2.8807262440677732, + 3.0222956190118566, + 3.164204088039696, + 3.308510520029813, + 3.4471480160718784, + 3.5903505050810054, + 3.7346975670661777, + 3.8766589420847595, + 4.018105274066329, + 4.160384768038057, + 4.3026882549747825, + 4.445785355987027, + 4.58645610907115, + 4.726378866005689, + 4.868202492012642, + 5.009396907989867, + 5.154869956080802, + 5.297432229039259, + 5.436256985994987, + 5.579131904058158, + 5.725242419051938, + 5.86830643506255, + 6.013869773014449, + 6.160519825061783, + 6.3060401460388675, + 6.45700819301419, + 6.5997223110171035, + 6.747932063066401, + 6.895713270059787, + 7.040810678037815, + 7.185164278023876, + 7.329653179971501, + 7.4740648390725255, + 7.6166666629724205, + 7.756882069050334, + 7.892406756989658, + 8.034469366073608, + 8.176601652987301, + 8.321695273043588, + 8.468852822086774, + 8.61112071806565, + 8.758555368054658, + 8.902885627001524, + 9.041474552010186, + 9.182707485044375, + 9.323415923048742, + 9.465132554993033, + 9.603935550083406, + 9.74902910203673, + 9.887911837082356, + 10.025818136055022, + 10.171219435986131, + 10.316438902984373, + 10.453073602053337, + 10.590585074038245, + 10.727248645038344, + 10.86809748201631, + 11.004910442978144, + 11.146999187069014, + 11.29012862208765, + 11.42829474399332, + 11.566032750997692, + 11.70038374804426, + 11.839764259988442, + 11.97839277004823, + 12.113584617036395, + 12.246413922053762, + 12.376697516068816, + 12.515414617024362, + 12.658988827024586, + 12.80014631303493, + 12.942387890070677, + 13.072677175048739, + 13.210426266072318, + 13.344404102070257, + 13.482877225033008, + 13.625350737012923 + ], + "chunk_gaps_ms": { + "count": 94, + "p50": 141.51557098375633, + "p95": 148.2097520492971, + "max": 279.0406310232356 + } + }, + { + "start": 723755.352535592, + "first": 723755.733463179, + "last": 723770.238432939, + "end": 723770.238823556, + "elapsed_s": 14.886287963949144, + "chunk_offsets_s": [ + 0.38092758692801, + 0.4675716880010441, + 0.730948255979456, + 1.0102758649736643, + 1.1367155419429764, + 1.2587501719826832, + 1.3815357559360564, + 1.513857198995538, + 1.6460415909532458, + 1.7788325839210302, + 1.9146017630118877, + 2.0503758369013667, + 2.1868747759144753, + 2.3193382879253477, + 2.456216872902587, + 2.5971158919855952, + 2.7396222489187494, + 2.8809402009937912, + 3.0226419949904084, + 3.1647036999929696, + 3.308967534918338, + 3.4474251510109752, + 3.590764299966395, + 3.735310422955081, + 3.87678444001358, + 4.018688149983063, + 4.160732309916057, + 4.302315118955448, + 4.4455519849434495, + 4.586858858936466, + 4.726612839964218, + 4.86846277595032, + 5.009534951997921, + 5.155223184963688, + 5.297707021934912, + 5.436926887952723, + 5.579566526925191, + 5.7253896539332345, + 5.8687419689958915, + 6.014268987928517, + 6.160654262988828, + 6.306501910905354, + 6.457298327004537, + 6.600114764994942, + 6.74835816596169, + 6.8962766689946875, + 7.041264480911195, + 7.185531726921909, + 7.329867151915096, + 7.474745749961585, + 7.617141967988573, + 7.757362408912741, + 7.89259179495275, + 8.035131240962073, + 8.177402492961846, + 8.322051687981002, + 8.468915655976161, + 8.611428439966403, + 8.759221190935932, + 8.903053120011464, + 9.041755329933949, + 9.183213686919771, + 9.323902333970182, + 9.465735649922863, + 9.60418096696958, + 9.749258035910316, + 9.888296718942001, + 10.025554495980032, + 10.171489239903167, + 10.315964030916803, + 10.453667487949133, + 10.59005930100102, + 10.727702103904448, + 10.868562354939058, + 11.005566037958488, + 11.147458812920377, + 11.290531636914238, + 11.428496658918448, + 11.566213817917742, + 11.70077830599621, + 11.840141488006338, + 11.978597151930444, + 12.113882908946835, + 12.246755187981762, + 12.376961419940926, + 12.515889105969109, + 12.659234920982271, + 12.8003344850149, + 12.94264222390484, + 13.072827812982723, + 13.210781481931917, + 13.344787023961544, + 13.483040700899437, + 13.625807633972727, + 13.745202283957042, + 13.862359994906, + 13.980424938956276, + 14.100729539990425, + 14.225127839948982, + 14.350911682005972, + 14.470248071942478, + 14.57977630593814, + 14.67550295998808, + 14.763439641916193, + 14.83552898094058, + 14.885897347005084 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 140.89901908300817, + "p95": 147.79275096952915, + "max": 279.3276089942083 + } + }, + { + "start": 723755.351819269, + "first": 723755.97751672, + "last": 723769.931952602, + "end": 723769.932740117, + "elapsed_s": 14.58092084794771, + "chunk_offsets_s": [ + 0.6256974509451538, + 0.7314168920274824, + 1.010455415933393, + 1.1369576799916103, + 1.2590267210034654, + 1.3816624849569052, + 1.5141109559917822, + 1.6462965159444138, + 1.778960719006136, + 1.9147542669670656, + 2.0506030620308593, + 2.187505956972018, + 2.3197636229451746, + 2.456501060980372, + 2.597577331936918, + 2.740116039989516, + 2.8815776630071923, + 3.0229676619637758, + 3.165072681964375, + 3.309341615997255, + 3.4477239010157064, + 3.5911802910268307, + 3.7353671479504555, + 3.8771606469526887, + 4.018786252941936, + 4.161074955016375, + 4.303480261005461, + 4.44653171102982, + 4.5871452880091965, + 4.727057671989314, + 4.868879092042334, + 5.009973749984056, + 5.155563852982596, + 5.298111116979271, + 5.436919825035147, + 5.579903119010851, + 5.725988249992952, + 5.868973477976397, + 6.0145633519859985, + 6.1610877140192315, + 6.306797122000717, + 6.457677172031254, + 6.6005551209673285, + 6.7485044749919325, + 6.896403844933957, + 7.041478235041723, + 7.185852727969177, + 7.330339535954408, + 7.4748409379972145, + 7.617329617030919, + 7.757656586007215, + 7.89297413197346, + 8.035155851976015, + 8.177274534944445, + 8.322398207965307, + 8.469712355989031, + 8.611666462966241, + 8.759224141016603, + 8.903300862992182, + 9.042176114046015, + 9.183390327030793, + 9.324098333949223, + 9.465812615002505, + 9.604707947932184, + 9.749776815995574, + 9.888586909975857, + 10.026046573999338, + 10.171900472952984, + 10.316511310986243, + 10.453761512995698, + 10.590577660943381, + 10.727660913951695, + 10.868768943008035, + 11.005581634002738, + 11.147424784954637, + 11.290815592044964, + 11.428961889934726, + 11.566461034002714, + 11.701062644016929, + 11.840478871017694, + 11.979059068951756, + 12.114160635974258, + 12.247205768944696, + 12.377373727969825, + 12.516092875041068, + 12.65965328796301, + 12.80081104603596, + 12.943094398942776, + 13.073402373003773, + 13.211109464988112, + 13.34508786094375, + 13.483537546009757, + 13.626086279982701, + 13.745594553998671, + 13.862772169988602, + 13.98082577297464, + 14.101261585950851, + 14.22558092197869, + 14.351330950041302, + 14.470804804004729, + 14.580133332987316 + ], + "chunk_gaps_ms": { + "count": 100, + "p50": 141.10134349903092, + "p95": 147.31414802372456, + "max": 279.03852390591055 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 72.2, + "per_stream_tps": 12.3, + "ttft_s_max": 21.23, + "timing_valid": true, + "end_to_end_tps": 72.0420621546337, + "makespan_s": 28.427836998947896, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 72.17409783817314, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 28.427836998947896, + "end_to_end_tps": 72.0420621546337, + "timing_valid": true, + "per_stream_tps": [ + 12.4, + 35.4, + 12.3, + 12.5, + 11.4, + 12.2, + 11.7, + 12.3 + ], + "ttft_s": [ + 1.48, + 21.23, + 1.8, + 0.62, + 0.38, + 1.17, + 0.16, + 0.89 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0a24c104609f", + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "28cedcb814fc", + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "22e8f3f09dfa", + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c28174794bf0", + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c6d6ec2a35a8", + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "1a4e0d70467b", + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66" + } + ], + "client_intervals": [ + { + "start": 723770.243858271, + "first": 723771.72416407, + "last": 723792.350870091, + "end": 723792.351633793, + "elapsed_s": 22.107775521930307, + "chunk_offsets_s": [ + 1.4803057989338413, + 1.6422223749104887, + 1.9811206019949168, + 2.165289163007401, + 2.357482051011175, + 2.5473226929316297, + 2.733685156912543, + 2.928660159930587, + 3.1168125169351697, + 3.3121917869430035, + 3.508615814964287, + 3.701384061947465, + 3.8952694319887087, + 4.0921744459774345, + 4.283273638924584, + 4.47843231796287, + 4.68394948693458, + 4.89135638193693, + 5.0934672019211575, + 5.296090442920104, + 5.499964823946357, + 5.702028714935295, + 5.907805123948492, + 6.110294793965295, + 6.311033576959744, + 6.511279886006378, + 6.711496984004043, + 6.911649003974162, + 7.11058662191499, + 7.316959374933504, + 7.521237593959086, + 7.725411735940725, + 7.930059504928067, + 8.135454172967002, + 8.33737190195825, + 8.53689199197106, + 8.737404162995517, + 8.942912155995145, + 9.150899790925905, + 9.359447940951213, + 9.563101614941843, + 9.770735017955303, + 9.976989216986112, + 10.17715605290141, + 10.38181143894326, + 10.584137373953126, + 10.787557213916443, + 10.989977105986327, + 11.19116031494923, + 11.389255185960792, + 11.595210712985136, + 11.795719369896688, + 11.993874874897301, + 12.1986981210066, + 12.401066983002238, + 12.598449787939899, + 12.80108525394462, + 13.002278449945152, + 13.201528275967576, + 13.401726093958132, + 13.60112617095001, + 13.800678202998824, + 13.998699640971608, + 14.196629648911767, + 14.394432202912867, + 14.59117081691511, + 14.79242880793754, + 14.993673368007876, + 15.198603746946901, + 15.39647994400002, + 15.60240181197878, + 15.80544370401185, + 16.00572606199421, + 16.20820002700202, + 16.40944827592466, + 16.61042014299892, + 16.814058351912536, + 17.01407604198903, + 17.215997090912424, + 17.411785557982512, + 17.61366220097989, + 17.821155951009132, + 18.029095954960212, + 18.232336220913567, + 18.44159540499095, + 18.645838356926106, + 18.849399637896568, + 19.051832634955645, + 19.25108575099148, + 19.455435827956535, + 19.661299473955296, + 19.864748178981245, + 20.071184766944498, + 20.274247310007922, + 20.480825171922334, + 20.68825941393152, + 20.884101934963837, + 21.07246803189628, + 21.421615850995295, + 21.61927080398891, + 21.804267322993837, + 21.967796107986942, + 22.107011819956824 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 201.5673170099035, + "p95": 207.94000395108014, + "max": 349.1478190990165 + } + }, + { + "start": 723770.242650098, + "first": 723791.472485585, + "last": 723798.669107981, + "end": 723798.66926017, + "elapsed_s": 28.426610071910545, + "chunk_offsets_s": [ + 21.229835486970842, + 21.42240529891569, + 21.619779113912955, + 21.805007662973367, + 21.96847230894491, + 22.108145730919205, + 22.227334170951508, + 22.342331512947567, + 22.45236464997288, + 22.554532452952117, + 22.641573679982685, + 22.72398213390261, + 22.78852282394655, + 22.850625050952658, + 22.908680578926578, + 22.96747631696053, + 23.02063785889186, + 23.074526182957925, + 23.133249614969827, + 23.19112089590635, + 23.25183288892731, + 23.310665876953863, + 23.37232604797464, + 23.442700007930398, + 23.508532167994417, + 23.57444498699624, + 23.641197404940613, + 23.705797143978998, + 23.764228242915124, + 23.815506254904903, + 23.867287058965303, + 23.91789194091689, + 23.969827634980902, + 24.02180647593923, + 24.074140943936072, + 24.125648954999633, + 24.17692789400462, + 24.22744896193035, + 24.280920770950615, + 24.33128969790414, + 24.382934178924188, + 24.435304051963612, + 24.485890443902463, + 24.538924669963308, + 24.59856010391377, + 24.655685513978824, + 24.70636294397991, + 24.758884412935004, + 24.81211804889608, + 24.87204642489087, + 24.928818158921786, + 24.980631905957125, + 25.033147306996398, + 25.087284106994048, + 25.1471924779471, + 25.20750280492939, + 25.267733927001245, + 25.329294575960375, + 25.39621852489654, + 25.464907777961344, + 25.525839747977443, + 25.578363959910348, + 25.631996970972978, + 25.682542219990864, + 25.734307806938887, + 25.78585373889655, + 25.83680700394325, + 25.888219888904132, + 25.940272054984234, + 25.991041292902082, + 26.042167740990408, + 26.093005557893775, + 26.144000504980795, + 26.197197706904262, + 26.25608211196959, + 26.313768162974156, + 26.37184716691263, + 26.427755274926312, + 26.48778590292204, + 26.553233577986248, + 26.618174792965874, + 26.68374350492377, + 26.75252137996722, + 26.8203332539415, + 26.886571904993616, + 26.953101485967636, + 27.020096955937333, + 27.085543519002385, + 27.151462289970368, + 27.21491649397649, + 27.274020513985306, + 27.333105359924957, + 27.39142127893865, + 27.454324038000777, + 27.51951701089274, + 27.58570888196118, + 27.647483788896352, + 27.700199497980066, + 27.75967659498565, + 27.817794461967424, + 27.876319195958786, + 27.936681844992563, + 28.003590846899897, + 28.069428750895895, + 28.137726987944916, + 28.203521839925088, + 28.266867725993507, + 28.323690747958608, + 28.377947939909063, + 28.426457882975228 + ], + "chunk_gaps_ms": { + "count": 109, + "p50": 58.88440506532788, + "p95": 119.18844003230333, + "max": 197.37381499726325 + } + }, + { + "start": 723770.241675204, + "first": 723772.042295602, + "last": 723792.796964753, + "end": 723792.797209624, + "elapsed_s": 22.55553441995289, + "chunk_offsets_s": [ + 1.8006203980185091, + 1.9835434469860047, + 2.1673727970337495, + 2.359405845985748, + 2.549308493034914, + 2.7361005709972233, + 2.930878513958305, + 3.1189528460381553, + 3.314309938927181, + 3.5107495239935815, + 3.703481012955308, + 3.897521393955685, + 4.094400570960715, + 4.285312213934958, + 4.480522025027312, + 4.686174380942248, + 4.893411073018797, + 5.095535258995369, + 5.298116656020284, + 5.502003108966164, + 5.704173775040545, + 5.9102974539855495, + 6.112680446007289, + 6.313333494006656, + 6.51349354092963, + 6.713836138951592, + 6.913692259928212, + 7.11260710703209, + 7.31903582403902, + 7.52353052503895, + 7.727557388017885, + 7.932321097003296, + 8.13777291297447, + 8.339986435021274, + 8.5394075330114, + 8.739217260037549, + 8.9444955450017, + 9.15248565399088, + 9.361045171972364, + 9.564834407996386, + 9.772316755028442, + 9.978534352965653, + 10.179557575029321, + 10.383426573942415, + 10.585861191968434, + 10.789069485967048, + 10.991703307023272, + 11.192991439951584, + 11.390808403026313, + 11.596791481017135, + 11.797093481989577, + 11.995568949030712, + 12.20096049993299, + 12.403022620943375, + 12.600022588041611, + 12.80296002898831, + 13.00404166104272, + 13.20324394502677, + 13.403423406998627, + 13.602826792979613, + 13.801949916989543, + 14.000353156006895, + 14.198051554965787, + 14.395941304974258, + 14.592640309943818, + 14.79405310493894, + 14.995231352979317, + 15.200114027946256, + 15.398276568041183, + 15.604070447967388, + 15.806996008963324, + 16.007170174038038, + 16.20992315595504, + 16.411536503001116, + 16.612756224931218, + 16.815703238011338, + 17.01563645002898, + 17.2178162089549, + 17.413280687993392, + 17.615243904991075, + 17.823007705970667, + 18.031566935940646, + 18.23487813700922, + 18.443458175985143, + 18.64769089501351, + 18.852431048988365, + 19.05453404597938, + 19.253752572927624, + 19.45818059099838, + 19.663758464041166, + 19.867311300942674, + 20.07380972802639, + 20.2766574639827, + 20.48275902098976, + 20.690881703980267, + 20.886732124025002, + 21.074937626020983, + 21.423899660934694, + 21.62129071401432, + 21.806500138016418, + 21.96981923293788, + 22.109610110986978, + 22.22839482501149, + 22.343386522028595, + 22.453461252036504, + 22.555289548938163 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 201.28813292831182, + "p95": 207.99010898917913, + "max": 348.96203491371125 + } + }, + { + "start": 723770.242159973, + "first": 723770.861360675, + "last": 723791.315906843, + "end": 723791.316069008, + "elapsed_s": 21.07390903495252, + "chunk_offsets_s": [ + 0.619200701941736, + 0.7316215279279277, + 1.0155554090160877, + 1.3223435149993747, + 1.643631331040524, + 1.9828878259286284, + 2.1662912049796432, + 2.359029799932614, + 2.5489012849284336, + 2.735071804956533, + 2.9299916790332645, + 3.11800678295549, + 3.3134700739756227, + 3.5100346439285204, + 3.7025202879449353, + 3.8971130680292845, + 4.093157954979688, + 4.284453158034012, + 4.479434886015952, + 4.68527570401784, + 4.892568854033016, + 5.094702756032348, + 5.297268675989471, + 5.501173292985186, + 5.7033441989915445, + 5.909767820034176, + 6.112290574004874, + 6.312426717020571, + 6.512421114020981, + 6.712954432936385, + 6.912381703034043, + 7.111504629952833, + 7.317862113006413, + 7.522479070001282, + 7.72658776701428, + 7.931381857022643, + 8.136605286970735, + 8.338935598963872, + 8.538161413976923, + 8.739003740018234, + 8.944070545956492, + 9.152057169005275, + 9.360612625954673, + 9.56449222099036, + 9.771956546930596, + 9.978096209932119, + 10.178717412985861, + 10.383247405989096, + 10.585591807961464, + 10.788968699984252, + 10.99145004095044, + 11.192569687962532, + 11.390432734042406, + 11.596438693930395, + 11.79752073995769, + 11.995318786939606, + 12.200529410038143, + 12.403228229959495, + 12.599739055964164, + 12.803135480964556, + 13.003637930960394, + 13.202955842018127, + 13.40299014293123, + 13.602481588022783, + 13.801862124004401, + 13.999915607040748, + 14.1977429370163, + 14.395739294937812, + 14.59233225195203, + 14.793914576992393, + 14.994878169032745, + 15.199745062971488, + 15.397874213987961, + 15.603717839927413, + 15.806683633942157, + 16.006795036024414, + 16.209604864940047, + 16.411645113956183, + 16.61218774493318, + 16.81548594404012, + 17.015200900030322, + 17.21755428693723, + 17.41291568998713, + 17.615151643985882, + 17.823158551938832, + 18.030366674996912, + 18.23391895298846, + 18.443568360991776, + 18.647293195943348, + 18.85147595894523, + 19.053474692977034, + 19.252693539950997, + 19.457303949980997, + 19.662604716955684, + 19.866303917951882, + 20.07254720700439, + 20.27549445501063, + 20.482471787952818, + 20.68967408093158, + 20.88536737393588, + 21.07374687003903 + ], + "chunk_gaps_ms": { + "count": 100, + "p50": 202.20343000255525, + "p95": 208.55545694939792, + "max": 339.2564948881045 + } + }, + { + "start": 723770.242070265, + "first": 723770.62175125, + "last": 723792.966544233, + "end": 723792.966961248, + "elapsed_s": 22.724890982965007, + "chunk_offsets_s": [ + 0.37968098500277847, + 0.46162438695318997, + 0.7316030759830028, + 1.0155939549440518, + 1.3223870160290971, + 1.6435984240379184, + 1.9828070340445265, + 2.166689460980706, + 2.3582320760469884, + 2.5482695379760116, + 2.7355556320399046, + 2.9303033569594845, + 3.118048924021423, + 3.31380442797672, + 3.5100196410203353, + 3.70288590004202, + 3.89740030199755, + 4.093552349950187, + 4.284720662981272, + 4.479615380056202, + 4.685583694954403, + 4.892781054018997, + 5.094907794962637, + 5.297397542046383, + 5.501399260945618, + 5.703514748020098, + 5.909728239057586, + 6.112202384043485, + 6.312614056048915, + 6.512565442011692, + 6.713092349004, + 6.912587590049952, + 7.111746206996031, + 7.3180714070331305, + 7.522729872958735, + 7.726755536044948, + 7.931582932011224, + 8.136945697944611, + 8.339100842014886, + 8.538562361965887, + 8.738975379965268, + 8.94453669898212, + 9.15255180397071, + 9.36108812096063, + 9.56485009402968, + 9.772648751968518, + 9.978378596017137, + 10.178458043024875, + 10.383639196050353, + 10.585761338006705, + 10.789448051946238, + 10.991642693057656, + 11.192784358980134, + 11.390950817032717, + 11.597073073033243, + 11.797136213979684, + 11.995627654949203, + 12.200440731015988, + 12.402711219969206, + 12.600088956998661, + 12.80302136100363, + 13.004156048991717, + 13.20323298801668, + 13.403411937993951, + 13.602778946049511, + 13.802084360038862, + 14.00018925103359, + 14.198098534019664, + 14.396042075008154, + 14.592922764015384, + 14.794122135033831, + 14.99532232095953, + 15.20016276894603, + 15.398161021992564, + 15.604104012949392, + 15.807105064974166, + 16.007239461992867, + 16.209916954976507, + 16.411483705043793, + 16.611589664011262, + 16.815771089983173, + 17.015697058057413, + 17.217581007047556, + 17.413405181025155, + 17.614815296023153, + 17.822482559946366, + 18.03096010896843, + 18.23452973703388, + 18.443114510038868, + 18.6477477659937, + 18.851624193019234, + 19.053808677010238, + 19.253434955957346, + 19.457824515993707, + 19.663196329958737, + 19.866582590038888, + 20.07328571600374, + 20.276097599999048, + 20.482287283055484, + 20.690221066004597, + 20.886587833054364, + 21.07457648194395, + 21.42358022101689, + 21.620604568975978, + 21.806145773967728, + 21.969367244048044, + 22.108873881050386, + 22.22803830297198, + 22.343028384028003, + 22.45324826601427, + 22.554848273051903, + 22.64226177195087, + 22.724473968031816 + ], + "chunk_gaps_ms": { + "count": 112, + "p50": 201.30515046184883, + "p95": 269.9786890298128, + "max": 349.00373907294124 + } + }, + { + "start": 723770.241523447, + "first": 723771.414916438, + "last": 723792.350725468, + "end": 723792.351062736, + "elapsed_s": 22.109539289027452, + "chunk_offsets_s": [ + 1.1733929910697043, + 1.3225595940602943, + 1.6440076300641522, + 1.9836500689852983, + 2.166847452055663, + 2.3590465110028163, + 2.549140732968226, + 2.736307856044732, + 2.930516524007544, + 3.118731999071315, + 3.3140565709909424, + 3.510598725057207, + 3.703391482005827, + 3.8979048939654604, + 4.09424075006973, + 4.2857105530565605, + 4.48100166907534, + 4.686358180013485, + 4.89375161298085, + 5.095768449013121, + 5.298229751060717, + 5.502265942981467, + 5.7042883730027825, + 5.90988146699965, + 6.112159098032862, + 6.313523594057187, + 6.513424052973278, + 6.713901153067127, + 6.913579807966016, + 7.112656275043264, + 7.319106737966649, + 7.5236386300530285, + 7.727662167046219, + 7.932320969062857, + 8.137384089990519, + 8.339942511054687, + 8.539433195022866, + 8.739567161072046, + 8.94520606205333, + 9.153315220028162, + 9.361703791073523, + 9.565351997036487, + 9.773147345054895, + 9.979207538999617, + 10.179451573058031, + 10.38422351097688, + 10.586381662054919, + 10.789831052999943, + 10.99227664002683, + 11.193368065985851, + 11.391430685063824, + 11.59738232602831, + 11.797760504065081, + 11.996116605005227, + 12.201217617024668, + 12.403009278001264, + 12.600741214002483, + 12.803601294988766, + 13.00456568703521, + 13.20389826304745, + 13.404097215970978, + 13.603423624997959, + 13.802810366032645, + 14.000826991046779, + 14.199007343966514, + 14.396634386037476, + 14.593546033021994, + 14.79462451604195, + 14.995920340996236, + 15.201023736968637, + 15.398777654045261, + 15.6048032050021, + 15.807864270056598, + 16.008020238019526, + 16.210502550005913, + 16.412223385996185, + 16.612417516997084, + 16.81627249403391, + 17.016458517988212, + 17.218297951039858, + 17.41400337801315, + 17.615602054051124, + 17.823252257076092, + 18.030749427038245, + 18.234714859048836, + 18.44396417797543, + 18.648231036029756, + 18.85179166099988, + 19.05421248904895, + 19.253826422966085, + 19.457849471014924, + 19.663332678028382, + 19.8667876990512, + 20.07324286806397, + 20.27630718506407, + 20.482992251054384, + 20.690439478028566, + 20.886156677035615, + 21.074531313031912, + 21.423588478006423, + 21.620962933055125, + 21.80623165203724, + 21.969563842983916, + 22.109202021034434 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 201.72083599027246, + "p95": 208.10915797483176, + "max": 349.05716497451067 + } + }, + { + "start": 723770.241765743, + "first": 723770.40412011, + "last": 723792.210995438, + "end": 723792.211138717, + "elapsed_s": 21.969372974010184, + "chunk_offsets_s": [ + 0.16235436697024852, + 0.22037481400184333, + 0.46197839302476496, + 0.7319697119528428, + 1.0157461259514093, + 1.3225491939811036, + 1.6439825519919395, + 1.983329317998141, + 2.1668243119493127, + 2.358410911052488, + 2.5486371510196477, + 2.7354169179452583, + 2.930209106998518, + 3.1183009709930047, + 3.3137628550175577, + 3.5102766449563205, + 3.702858397969976, + 3.8976217459421605, + 4.0936000880319625, + 4.284956241026521, + 4.4798785540042445, + 4.6859547949861735, + 4.8930195569992065, + 5.095146344974637, + 5.298090361990035, + 5.501664464944042, + 5.703780959942378, + 5.909966086037457, + 6.112449393025599, + 6.312875807052478, + 6.512926310999319, + 6.713435948942788, + 6.912833367008716, + 7.111995373037644, + 7.318318078992888, + 7.522976401029155, + 7.727114295004867, + 7.931829508044757, + 8.137050525052473, + 8.339288895018399, + 8.538726542028598, + 8.739200928946957, + 8.944609194993973, + 9.152508786995895, + 9.361057496047579, + 9.564834387972951, + 9.77241249801591, + 9.978565225028433, + 10.178875774028711, + 10.383419631049037, + 10.585841553052887, + 10.789405015995726, + 10.991672234958969, + 11.193012607982382, + 11.39089676097501, + 11.59674891200848, + 11.797226644004695, + 11.995583944953978, + 12.201027545030229, + 12.403242627973668, + 12.599988845991902, + 12.803072310052812, + 13.00408436905127, + 13.203207837999798, + 13.403445663978346, + 13.60296865797136, + 13.802067828015424, + 14.000207019038498, + 14.198066298034973, + 14.396198014030233, + 14.592779553961009, + 14.794042423018254, + 14.995194924995303, + 15.20007198897656, + 15.398309731972404, + 15.604183148010634, + 15.807019946048968, + 16.00713102798909, + 16.209908779943362, + 16.411847632960416, + 16.611774767981842, + 16.815808327984996, + 17.015657764975913, + 17.21760722703766, + 17.41324440599419, + 17.61582595203072, + 17.823318070033565, + 18.030922875972465, + 18.234116432024166, + 18.443152672960423, + 18.648190272040665, + 18.852153945015743, + 19.054041240015067, + 19.25394592003431, + 19.457765180966817, + 19.663050660979934, + 19.86660170694813, + 20.072894279961474, + 20.276006410946138, + 20.482970278942958, + 20.690126406028867, + 20.885857710032724, + 21.07425893796608, + 21.42324766202364, + 21.62061421701219, + 21.805933658964932, + 21.96922969503794 + ], + "chunk_gaps_ms": { + "count": 106, + "p50": 202.12164148688316, + "p95": 269.9913189280778, + "max": 348.98872405756265 + } + }, + { + "start": 723770.241423171, + "first": 723771.131429889, + "last": 723791.863167069, + "end": 723791.863309133, + "elapsed_s": 21.621885961969383, + "chunk_offsets_s": [ + 0.8900067179929465, + 1.0164905439596623, + 1.3231987749459222, + 1.644821499940008, + 1.9836993099888787, + 2.167111051036045, + 2.3594619870418683, + 2.549513390986249, + 2.736293238005601, + 2.931030777981505, + 3.119098552968353, + 3.3145213129464537, + 3.510955972946249, + 3.7036599449347705, + 3.8979110750369728, + 4.0947219809750095, + 4.285860602976754, + 4.480650895042345, + 4.686499707982875, + 4.893744091037661, + 5.095941444044001, + 5.298492845031433, + 5.502307347021997, + 5.704307263949886, + 5.910430895979516, + 6.112681797938421, + 6.313538245973177, + 6.51355875399895, + 6.7140387169783935, + 6.9135566300246865, + 7.1129774149740115, + 7.319330501020886, + 7.523827904020436, + 7.727907611988485, + 7.932533795014024, + 8.13768095895648, + 8.33985752996523, + 8.539261933998205, + 8.73980398895219, + 8.94527296803426, + 9.153469110024162, + 9.361846887972206, + 9.565364873036742, + 9.77306891803164, + 9.979117807000875, + 10.179519133991562, + 10.384181651985273, + 10.586536176037043, + 10.790056561934762, + 10.992333134985529, + 11.193645511986688, + 11.391646937001497, + 11.597681662999094, + 11.798101222957484, + 11.996352429036051, + 12.201014557969756, + 12.403777641942725, + 12.600940197939053, + 12.803629806963727, + 13.004745034035295, + 13.203917803009972, + 13.404124991968274, + 13.603484836989082, + 13.802691765944473, + 14.000889113987796, + 14.198872608016245, + 14.396805851953104, + 14.593524620984681, + 14.794820158975199, + 14.996059792931192, + 15.200945031014271, + 15.398952701943927, + 15.604867869988084, + 15.807931796996854, + 16.007986768963747, + 16.210518348030746, + 16.412257588934153, + 16.61273049993906, + 16.816452180035412, + 17.016611226950772, + 17.218350014998578, + 17.414148204028606, + 17.61560806503985, + 17.82319419295527, + 18.03133974096272, + 18.235204830998555, + 18.44412158895284, + 18.64785430696793, + 18.85232907801401, + 19.054679450928234, + 19.254188086022623, + 19.458280646940693, + 19.66394085600041, + 19.867473257007077, + 20.07369331398513, + 20.27677506103646, + 20.48316448694095, + 20.690911303041503, + 20.88658852595836, + 21.075158060994, + 21.42425718693994, + 21.62174389802385 + ], + "chunk_gaps_ms": { + "count": 101, + "p50": 201.999916927889, + "p95": 208.37777794804424, + "max": 349.0991259459406 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 70.6, + "per_stream_tps": 70.6, + "ttft_s_max": 0.11, + "timing_valid": true, + "end_to_end_tps": 68.81210534106044, + "makespan_s": 3.720275651081465, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 70.62948416378526, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 3.720275651081465, + "end_to_end_tps": 68.81210534106044, + "timing_valid": true, + "per_stream_tps": [ + 70.6 + ], + "ttft_s": [ + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 723814.208571369, + "first": 723814.318311962, + "last": 723817.928702198, + "end": 723817.92884702, + "elapsed_s": 3.720275651081465, + "chunk_offsets_s": [ + 0.10974059300497174, + 0.16961614903993905, + 0.22756165801547468, + 0.2890846390509978, + 0.3549921161029488, + 0.4222280631074682, + 0.48937487101648003, + 0.5550045070704073, + 0.6193226421019062, + 0.6862665310036391, + 0.751395209110342, + 0.8176933340728283, + 0.8824796440312639, + 0.9469221309991553, + 1.0118330490076914, + 1.07597451005131, + 1.1413034460274503, + 1.206256544101052, + 1.2632688710000366, + 1.3147827410139143, + 1.3675475850468501, + 1.4218675820156932, + 1.482424233108759, + 1.5419760720105842, + 1.5997238130075857, + 1.6590423160232604, + 1.724846656084992, + 1.790956040029414, + 1.853925021016039, + 1.9163964811014012, + 1.998079500044696, + 2.078238194109872, + 2.160965441027656, + 2.245315015083179, + 2.3260593420127407, + 2.410648065037094, + 2.493359712068923, + 2.577426773030311, + 2.661416868097149, + 2.746480151079595, + 2.830749198095873, + 2.9139873560052365, + 2.996986636077054, + 3.0809436440467834, + 3.164816276053898, + 3.2481572460383177, + 3.332302119000815, + 3.4156788650434464, + 3.4994919860037044, + 3.5808657680172473, + 3.6636222160886973, + 3.7201308290241286 + ], + "chunk_gaps_ms": { + "count": 51, + "p50": 66.109383944422, + "p95": 84.34957405552268, + "max": 85.06328298244625 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 81.7, + "per_stream_tps": 41.6, + "ttft_s_max": 0.27, + "timing_valid": true, + "end_to_end_tps": 80.66172781763856, + "makespan_s": 6.347496066009626, + "steady_tps": 83.5, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 81.74677646771583, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 6.347496066009626, + "end_to_end_tps": 80.66172781763856, + "timing_valid": true, + "steady_tps": 83.47482955997891, + "per_stream_tps": [ + 42.0, + 41.2 + ], + "ttft_s": [ + 0.27, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 723817.930241506, + "first": 723818.203213873, + "last": 723824.277448083, + "end": 723824.277590459, + "elapsed_s": 6.347348953015171, + "chunk_offsets_s": [ + 0.2729723670054227, + 0.3540442860685289, + 0.4398757250746712, + 0.5242314420174807, + 0.6143848000792786, + 0.7034932910464704, + 0.7925414720084518, + 0.878189405077137, + 0.9614472340326756, + 1.0450544770574197, + 1.1324448940576985, + 1.2180870020529255, + 1.308610555017367, + 1.3971949350088835, + 1.4863653980428353, + 1.5777959739789367, + 1.6645369890611619, + 1.7563974090153351, + 1.8456518379971385, + 1.9372823730809614, + 2.0288071680115536, + 2.1178942209808156, + 2.204323454061523, + 2.2923608269775286, + 2.381333676050417, + 2.469222187064588, + 2.5538740570191294, + 2.639884442090988, + 2.725213936995715, + 2.809948747046292, + 2.8930917150573805, + 2.979119120980613, + 3.072286464041099, + 3.1588600870454684, + 3.2473331730579957, + 3.335293611045927, + 3.4186757240677252, + 3.5049956360599026, + 3.5938542400253937, + 3.6821672570658848, + 3.767698366078548, + 3.8581947330385447, + 3.9478634510887787, + 4.035069989040494, + 4.121656468021683, + 4.210417552036233, + 4.298253738088533, + 4.384747916017659, + 4.472145301057026, + 4.559973950032145, + 4.646955349016935, + 4.733876626007259, + 4.819255767040886, + 4.906809624051675, + 4.994200813001953, + 5.082292909035459, + 5.169432773021981, + 5.256853688042611, + 5.34351981699001, + 5.429781911079772, + 5.517278304090723, + 5.6064446159871295, + 5.6942511920351535, + 5.780169402016327, + 5.866828922065906, + 5.953218940994702, + 6.037474194075912, + 6.121107024024241, + 6.2078789949882776, + 6.290203288081102, + 6.347206577076577 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 87.17320096911862, + "p95": 91.52479493059218, + "max": 93.16734306048602 + } + }, + { + "start": 723817.930094393, + "first": 723818.038669968, + "last": 723824.220565196, + "end": 723824.220635005, + "elapsed_s": 6.29054061206989, + "chunk_offsets_s": [ + 0.10857557505369186, + 0.16759275807999074, + 0.35441422602161765, + 0.44025034399237484, + 0.5248037140117958, + 0.6146661849925295, + 0.7039711039979011, + 0.7927766470238566, + 0.8784575030440465, + 0.9618542350362986, + 1.0456406400771812, + 1.1327261360129341, + 1.218405910069123, + 1.308955576037988, + 1.3975875959731638, + 1.4866652260534465, + 1.57820625603199, + 1.6648418629774824, + 1.7564006260363385, + 1.8461218259762973, + 1.9377120430581272, + 2.029071738012135, + 2.118210747023113, + 2.2048424059757963, + 2.292683894978836, + 2.3816292650299147, + 2.4694713309872895, + 2.554204691085033, + 2.64038866199553, + 2.7255397790577263, + 2.8103616520529613, + 2.8937916529830545, + 2.97959579504095, + 3.072605220018886, + 3.159220193978399, + 3.2477733860723674, + 3.3356377379968762, + 3.419134746072814, + 3.5053972579771653, + 3.594196296064183, + 3.682574868085794, + 3.7682301090098917, + 3.8584760760422796, + 3.9485470399959013, + 4.0355080839945, + 4.122055765008554, + 4.21081707207486, + 4.298802827019244, + 4.385157934972085, + 4.472596454084851, + 4.560342182056047, + 4.647300387034193, + 4.734196349047124, + 4.819680936052464, + 4.907222145004198, + 4.994537326972932, + 5.08277507906314, + 5.169826097087935, + 5.257321759010665, + 5.3437868220498785, + 5.430191355990246, + 5.517560702050105, + 5.6068342860089615, + 5.6946272070053965, + 5.780509100994095, + 5.866783577017486, + 5.953222561976872, + 6.037764096981846, + 6.1215943730203435, + 6.20809926406946, + 6.29047080304008 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 87.06825698027387, + "p95": 91.55876305885613, + "max": 186.8214679416269 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 115.4, + "per_stream_tps": 30.1, + "ttft_s_max": 0.67, + "timing_valid": true, + "end_to_end_tps": 114.4309649062791, + "makespan_s": 8.948626806028187, + "steady_tps": 121.9, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 115.39961835388979, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 8.948626806028187, + "end_to_end_tps": 114.4309649062791, + "timing_valid": true, + "steady_tps": 121.93963418318616, + "per_stream_tps": [ + 30.8, + 30.3, + 29.9, + 29.7 + ], + "ttft_s": [ + 0.67, + 0.46, + 0.27, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 723824.281811094, + "first": 723824.955306696, + "last": 723833.228781484, + "end": 723833.228948696, + "elapsed_s": 8.947137602022849, + "chunk_offsets_s": [ + 0.6734956019790843, + 0.7929088629316539, + 0.9147179760038853, + 1.0344395469874144, + 1.1541121510090306, + 1.2711550940293819, + 1.3870451250113547, + 1.500952498987317, + 1.6151132659288123, + 1.7298536039888859, + 1.851268911967054, + 1.9774386240169406, + 2.1055669779889286, + 2.231575579964556, + 2.3562043249839917, + 2.4829713399522007, + 2.6099893879145384, + 2.7376012669410557, + 2.857864292920567, + 2.9847494799178094, + 3.110943100997247, + 3.23409859102685, + 3.357596445013769, + 3.4779396220110357, + 3.5981814669212326, + 3.714790872996673, + 3.830071516917087, + 3.9422530609881505, + 4.0584972089855, + 4.178362620994449, + 4.293452351004817, + 4.411344563006423, + 4.529361139982939, + 4.649420875939541, + 4.765486423973925, + 4.883201393997297, + 5.0032536189537495, + 5.1242658930132166, + 5.246017335914075, + 5.366022269940004, + 5.484676457941532, + 5.602291670977138, + 5.720670541981235, + 5.844762426917441, + 5.965680274995975, + 6.086493946029805, + 6.207266934914514, + 6.328133406932466, + 6.446836840943433, + 6.562966285971925, + 6.682914056931622, + 6.802051639999263, + 6.922471150988713, + 7.043596233008429, + 7.166343648917973, + 7.287217861972749, + 7.404132773983292, + 7.5225882979575545, + 7.644061038969085, + 7.763223109999672, + 7.883923302986659, + 8.005440879962407, + 8.125997922965325, + 8.241113942931406, + 8.357518407981843, + 8.474470839952119, + 8.591395342024043, + 8.705760975019075, + 8.806702691013925, + 8.890389485983178, + 8.94697038992308 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 119.90659148432314, + "p95": 126.88518699724227, + "max": 128.12835397198796 + } + }, + { + "start": 723824.280626151, + "first": 723824.743146863, + "last": 723833.172052121, + "end": 723833.172226914, + "elapsed_s": 8.89160076295957, + "chunk_offsets_s": [ + 0.4625207119388506, + 0.5688627569470555, + 0.7939288780326024, + 0.9156008099671453, + 1.0354549350449815, + 1.154955561971292, + 1.272188624017872, + 1.388141177012585, + 1.5019639020320028, + 1.6162044720258564, + 1.7307079669553787, + 1.8523217790061608, + 1.9783823229372501, + 2.106643161037937, + 2.2326501220231876, + 2.357316895038821, + 2.4837828089948744, + 2.611061298986897, + 2.7382848280249164, + 2.8587463019648567, + 2.985545107978396, + 3.112036966951564, + 3.235498875961639, + 3.359008371946402, + 3.478984114015475, + 3.598988114972599, + 3.716175592970103, + 3.831702725030482, + 3.943168280995451, + 4.059147938969545, + 4.179244744009338, + 4.294135635951534, + 4.4120057879481465, + 4.530693640932441, + 4.650790843996219, + 4.766506726038642, + 4.884131607948802, + 5.004159256001003, + 5.125125361024402, + 5.2468828989658505, + 5.3669541659764946, + 5.485584886977449, + 5.603733554016799, + 5.721742952009663, + 5.845692311995663, + 5.966816017986275, + 6.087429265026003, + 6.208345519029535, + 6.329234887030907, + 6.4483241570414975, + 6.564422313007526, + 6.683977850014344, + 6.803450625971891, + 6.9234945609932765, + 7.044670460978523, + 7.167733059031889, + 7.288154980982654, + 7.405216927989386, + 7.52371606498491, + 7.64504410000518, + 7.76459670602344, + 7.88528537703678, + 8.006927768001333, + 8.126975354971364, + 8.242554803029634, + 8.358522844035178, + 8.475404334021732, + 8.59281117701903, + 8.707002831040882, + 8.807846009964123, + 8.89142597001046 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 119.98987151309848, + "p95": 127.22352903801948, + "max": 225.0661210855469 + } + }, + { + "start": 723824.281079466, + "first": 723824.555430334, + "last": 723833.087952969, + "end": 723833.088110988, + "elapsed_s": 8.807031522039324, + "chunk_offsets_s": [ + 0.27435086795594543, + 0.35613957000896335, + 0.5688022549729794, + 0.7938075079582632, + 0.9153931819600984, + 1.035045969998464, + 1.1545489389682189, + 1.271779307979159, + 1.3874920200323686, + 1.5015563239576295, + 1.6159339590230957, + 1.7304615740431473, + 1.8518160639796406, + 1.9778815660392866, + 2.106146158068441, + 2.2321524809813127, + 2.356819238048047, + 2.4834945540642366, + 2.6105656509753317, + 2.738043721066788, + 2.8584870980121195, + 2.98524967499543, + 3.1115151409758255, + 3.2351950380252674, + 3.358618338010274, + 3.4786054500145838, + 3.598583503975533, + 3.7157763889990747, + 3.831017339020036, + 3.9427678959909827, + 4.0587558100232854, + 4.178480284055695, + 4.2934655860299245, + 4.411611011018977, + 4.530285896034911, + 4.650042206980288, + 4.766132398042828, + 4.8834280939772725, + 5.003753368975595, + 5.124446996953338, + 5.246474476065487, + 5.3665512460283935, + 5.485176304006018, + 5.602889214060269, + 5.721467911964282, + 5.844763851026073, + 5.96600162202958, + 6.087322263978422, + 6.207624517963268, + 6.328510919003747, + 6.447260383982211, + 6.563461182988249, + 6.683327884064056, + 6.802546184975654, + 6.9227321080397815, + 7.044068563031033, + 7.166841289028525, + 7.287443405017257, + 7.404492730041966, + 7.522966413991526, + 7.64435371698346, + 7.763753989012912, + 7.884473314974457, + 8.005944072967395, + 8.12624044297263, + 8.241378072998486, + 8.357475613011047, + 8.474414511001669, + 8.591487520025112, + 8.706386661040597, + 8.80687350302469 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 119.9223775183782, + "p95": 127.47807009145617, + "max": 225.00525298528373 + } + }, + { + "start": 723824.28032189, + "first": 723824.389930762, + "last": 723832.987702813, + "end": 723832.9877893, + "elapsed_s": 8.707467410014942, + "chunk_offsets_s": [ + 0.10960887197870761, + 0.16981118195690215, + 0.35707547899801284, + 0.5693719829432666, + 0.7943287040106952, + 0.9160748770227656, + 1.035886409925297, + 1.155544570996426, + 1.2726024989970028, + 1.388591399998404, + 1.5023852699669078, + 1.6166589820059016, + 1.7312850679736584, + 1.8529543409822509, + 1.9788197439629585, + 2.1070166550343856, + 2.2330232799286023, + 2.3578701009973884, + 2.4844168230192736, + 2.6114401179365814, + 2.739041712018661, + 2.8594901759643108, + 2.9864792230073363, + 3.112601430970244, + 3.2354989019222558, + 3.3590108229545876, + 3.4791911899810657, + 3.599619925953448, + 3.7163587469840422, + 3.831632948014885, + 3.9438587150070816, + 4.060230821953155, + 4.179381493944675, + 4.294670174014755, + 4.412568366969936, + 4.530624101986177, + 4.651016357936896, + 4.766750918002799, + 4.884639094001614, + 5.005044127930887, + 5.12570329697337, + 5.247598749003373, + 5.367547256988473, + 5.486247438006103, + 5.603823740966618, + 5.722449516993947, + 5.84578099893406, + 5.966949341935106, + 6.088149277959019, + 6.208710107021034, + 6.329584621009417, + 6.4484168159542605, + 6.564491565921344, + 6.684326307964511, + 6.8034941899823025, + 6.9238932459848, + 7.045055132941343, + 7.168092023930512, + 7.28855819592718, + 7.405669788015075, + 7.523861992987804, + 7.645436016959138, + 7.764390285941772, + 7.885503673925996, + 8.007024199003354, + 8.127242843969725, + 8.24237027601339, + 8.358753561973572, + 8.475802151951939, + 8.592494916985743, + 8.707380922976881 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 120.1995059964247, + "p95": 128.19691107142717, + "max": 224.9567210674286 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 128.8, + "per_stream_tps": 21.7, + "ttft_s_max": 12.01, + "timing_valid": true, + "end_to_end_tps": 128.3746663734432, + "makespan_s": 15.953303388087079, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 128.78144573939778, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 15.953303388087079, + "end_to_end_tps": 128.3746663734432, + "timing_valid": true, + "per_stream_tps": [ + 64.7, + 22.1, + 21.4, + 21.8, + 21.6, + 21.6, + 22.4, + 21.5 + ], + "ttft_s": [ + 12.01, + 1.15, + 0.28, + 0.91, + 0.11, + 0.68, + 1.42, + 0.47 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 723833.231519057, + "first": 723845.242963019, + "last": 723849.184615415, + "end": 723849.184712002, + "elapsed_s": 15.953192945104092, + "chunk_offsets_s": [ + 12.011443961993791, + 12.173577911104076, + 12.324765903060324, + 12.462092144065537, + 12.585079815005884, + 12.695095029077493, + 12.784560092026368, + 12.85094446700532, + 12.915118602104485, + 12.97951075900346, + 13.046226163045503, + 13.111247805994935, + 13.175818042014726, + 13.24034224206116, + 13.303965914063156, + 13.370192500995472, + 13.434949787100777, + 13.492309321998619, + 13.543831121060066, + 13.594615553040057, + 13.648403366096318, + 13.708658057032153, + 13.767926283995621, + 13.825575071037747, + 13.88547744101379, + 13.951015301048756, + 14.016064124065451, + 14.07976627908647, + 14.141879337024875, + 14.223927694023587, + 14.304195499047637, + 14.387459296034649, + 14.472176969051361, + 14.552939477027394, + 14.638138876995072, + 14.72141823801212, + 14.806333405082114, + 14.890681305085309, + 14.975510676042177, + 15.06044005509466, + 15.144245151081122, + 15.22743451606948, + 15.311934224097058, + 15.395972589030862, + 15.480741058010608, + 15.564796650083736, + 15.64829224999994, + 15.731917038094252, + 15.81270453799516, + 15.896411386085674, + 15.95309635810554 + ], + "chunk_gaps_ms": { + "count": 50, + "p50": 80.77500393847004, + "p95": 137.32624100521207, + "max": 162.13394911028445 + } + }, + { + "start": 723833.231817427, + "first": 723834.385772507, + "last": 723845.92653608, + "end": 723845.926628626, + "elapsed_s": 12.694811198976822, + "chunk_offsets_s": [ + 1.1539550800807774, + 1.3096156340325251, + 1.5883128310088068, + 1.7556774700060487, + 1.9188069449737668, + 2.081462593981996, + 2.2428728650556877, + 2.403467167983763, + 2.569417965016328, + 2.739745849976316, + 2.9162021160591394, + 3.096301309997216, + 3.278171846992336, + 3.457295638974756, + 3.6351701400708407, + 3.814669720013626, + 3.9916515260702, + 4.166874884045683, + 4.345595658989623, + 4.52167999104131, + 4.701339906081557, + 4.876283988007344, + 5.043287058011629, + 5.210354599985294, + 5.373980703996494, + 5.530848148046061, + 5.687791355070658, + 5.8431955120759085, + 6.000206368975341, + 6.157984570018016, + 6.31685586005915, + 6.47526595601812, + 6.632839858997613, + 6.789258633973077, + 6.947769365971908, + 7.106737507972866, + 7.267044499982148, + 7.4251663100440055, + 7.585102748009376, + 7.74453699903097, + 7.904216401046142, + 8.063796770991758, + 8.223084280034527, + 8.383886384079233, + 8.545076386071742, + 8.706146982032806, + 8.867826158064418, + 9.027962543070316, + 9.18844680103939, + 9.3477420930285, + 9.508446473977529, + 9.667857203050517, + 9.82641793997027, + 9.987602903041989, + 10.14717215206474, + 10.306624272023328, + 10.468715792987496, + 10.62868043105118, + 10.789058692986146, + 10.947615217999555, + 11.107373783015646, + 11.26839253201615, + 11.42954273999203, + 11.588720179046504, + 11.747388424002565, + 11.90361539798323, + 12.173482206999324, + 12.324539307039231, + 12.461858932976611, + 12.584678402054124, + 12.694718652986921 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.34262697212398, + "p95": 180.0991939380765, + "max": 278.6971969762817 + } + }, + { + "start": 723833.232465033, + "first": 723833.512182125, + "last": 723845.40523096, + "end": 723845.405357204, + "elapsed_s": 12.172892171074636, + "chunk_offsets_s": [ + 0.2797170920530334, + 0.3604266180191189, + 0.5733379180310294, + 0.799565946101211, + 1.0471252060960978, + 1.3085815770318732, + 1.58701280108653, + 1.7547854110598564, + 1.9180688080377877, + 2.0806066850200295, + 2.2421612211037427, + 2.4026327970204875, + 2.568877201061696, + 2.739185161073692, + 2.9154694780008867, + 3.0951730750966817, + 3.2775891470955685, + 3.4565747750457376, + 3.6344154099933803, + 3.8136853020405397, + 3.9906276050023735, + 4.165980193065479, + 4.345111733069643, + 4.52117535402067, + 4.700529706082307, + 4.875287640024908, + 5.042711285990663, + 5.208602222031914, + 5.373010324081406, + 5.529816330992617, + 5.686723637045361, + 5.84216280002147, + 5.999189899070188, + 6.15678346902132, + 6.316823431989178, + 6.474122863030061, + 6.631752859102562, + 6.787652797065675, + 6.946289704064839, + 7.106134706060402, + 7.266450435039587, + 7.424445021082647, + 7.584307920071296, + 7.743938797037117, + 7.9036148129962385, + 8.063212773995474, + 8.222497311071493, + 8.383287739008665, + 8.54449393809773, + 8.705558209097944, + 8.867022565100342, + 9.027256233035587, + 9.187552378047258, + 9.347146022017114, + 9.50774786004331, + 9.667154594091699, + 9.82583119906485, + 9.987004325026646, + 10.146262136055157, + 10.305930783040822, + 10.467901778989471, + 10.627928724046797, + 10.788487805053592, + 10.946923662093468, + 11.10677072207909, + 11.26761234703008, + 11.42830246000085, + 11.587764372001402, + 11.746192336082458, + 11.903228285023943, + 12.172765927040018 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.64597549848258, + "p95": 247.55925999488682, + "max": 278.4312240546569 + } + }, + { + "start": 723833.232013096, + "first": 723834.138670998, + "last": 723845.816539609, + "end": 723845.816614019, + "elapsed_s": 12.584600923000835, + "chunk_offsets_s": [ + 0.906657902058214, + 1.0484449670184404, + 1.309550118050538, + 1.5879378729732707, + 1.75534668401815, + 1.9188900360604748, + 2.081517504993826, + 2.2428402840159833, + 2.403153161983937, + 2.569723710999824, + 2.739136402029544, + 2.9163421069970354, + 3.096322029014118, + 3.278112265979871, + 3.4568996450398117, + 3.634936437010765, + 3.814531932002865, + 3.9912941249785945, + 4.166794206015766, + 4.345266730990261, + 4.521821486996487, + 4.701482216012664, + 4.876452294993214, + 5.0427378880558535, + 5.209767932072282, + 5.373696448048577, + 5.530895082978532, + 5.687384719029069, + 5.842893033986911, + 5.999917817069218, + 6.157682274002582, + 6.317093068966642, + 6.475109304999933, + 6.632488753995858, + 6.788707436993718, + 6.94704952603206, + 7.106475439970382, + 7.267264413065277, + 7.425162106985226, + 7.585046665044501, + 7.744753062026575, + 7.904325525974855, + 8.06409441307187, + 8.223162470036186, + 8.384053342044353, + 8.545309604029171, + 8.706350457039662, + 8.867911105044186, + 9.02803488704376, + 9.188292875071056, + 9.348102407064289, + 9.50868283805903, + 9.6679716600338, + 9.82664741796907, + 9.987679801997729, + 10.147427794989198, + 10.30628020002041, + 10.468281183973886, + 10.628736787009984, + 10.789322467986494, + 10.947870239033364, + 11.10755805298686, + 11.267945845029317, + 11.429027335019782, + 11.58847728103865, + 11.747151134070009, + 11.903863560059108, + 12.173139316029847, + 12.324538954999298, + 12.461561850970611, + 12.584526513004676 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.42169753927737, + "p95": 181.79023696575314, + "max": 278.3877549227327 + } + }, + { + "start": 723833.231619445, + "first": 723833.343824156, + "last": 723845.135352172, + "end": 723845.135705492, + "elapsed_s": 11.904086046968587, + "chunk_offsets_s": [ + 0.1122047109529376, + 0.17210413597058505, + 0.36134985997341573, + 0.5742905309889466, + 0.8006778269773349, + 1.048145995941013, + 1.3096623909659684, + 1.5882256359327585, + 1.7558006460312754, + 1.9189650809857994, + 2.0817450979957357, + 2.2427546490216628, + 2.403624612954445, + 2.5693454910069704, + 2.7398394329939038, + 2.9159948129672557, + 3.096418370027095, + 3.2781516530085355, + 3.4572471189312637, + 3.6354637329932302, + 3.8148342670174316, + 3.991784904967062, + 4.167031522956677, + 4.345864078961313, + 4.521917704027146, + 4.701644751010463, + 4.876225702930242, + 5.043823863961734, + 5.210820849984884, + 5.373645551968366, + 5.531156325014308, + 5.688055373961106, + 5.843224911950529, + 6.00000201293733, + 6.1575879459269345, + 6.317115620011464, + 6.474791790009476, + 6.6323756269412115, + 6.78853836201597, + 6.947178207919933, + 7.106745620956644, + 7.267139057978056, + 7.425126631977037, + 7.584911391022615, + 7.744648025021888, + 7.904319440945983, + 8.064674561028369, + 8.223177184932865, + 8.384042241959833, + 8.545188779011369, + 8.706251467927359, + 8.867823728942312, + 9.028001743019558, + 9.188601888949051, + 9.34813634492457, + 9.50850829994306, + 9.66789682302624, + 9.826502481941134, + 9.987699202960357, + 10.147300636977889, + 10.30659171694424, + 10.468535842024721, + 10.628712515928783, + 10.789153627934866, + 10.94768732402008, + 11.107480806997046, + 11.268414667923935, + 11.42922307702247, + 11.5884620109573, + 11.747137748985551, + 11.903732726932503 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.8367330627516, + "p95": 226.38729598838836, + "max": 278.5632449667901 + } + }, + { + "start": 723833.231948123, + "first": 723833.911960393, + "last": 723845.693516195, + "end": 723845.69369432, + "elapsed_s": 12.461746197077446, + "chunk_offsets_s": [ + 0.6800122699933127, + 0.800118547049351, + 1.0476000510388985, + 1.309136592084542, + 1.5875834930920973, + 1.7551537780091166, + 1.9183061280054972, + 2.080981916980818, + 2.242316304007545, + 2.4030595329822972, + 2.5687323060119525, + 2.739104144042358, + 2.9154179020551965, + 3.095902413013391, + 3.2776381220901385, + 3.4565203139791265, + 3.6348718360532075, + 3.8139731300761923, + 3.991086650057696, + 4.166637517046183, + 4.345142011065036, + 4.521429402055219, + 4.701105780084617, + 4.875761258997954, + 5.043297922005877, + 5.21072685404215, + 5.373440994997509, + 5.530682094045915, + 5.687693494022824, + 5.8429977090563625, + 5.999943679082207, + 6.1577036450617015, + 6.316916360054165, + 6.47485942998901, + 6.632626477978192, + 6.78871152701322, + 6.94715059408918, + 7.1063063950277865, + 7.267278416082263, + 7.425540292053483, + 7.585166839999147, + 7.744766411022283, + 7.904352643992752, + 8.063923593028449, + 8.223292057984509, + 8.384159177076072, + 8.545212425058708, + 8.706284058978781, + 8.868089989060536, + 9.028150242054835, + 9.187785314046778, + 9.348095301073045, + 9.50854056503158, + 9.668138315086253, + 9.826602130080573, + 9.987786359037273, + 10.147537837037817, + 10.30657112505287, + 10.468299968983047, + 10.628763289074413, + 10.78933425701689, + 10.947829896002077, + 11.107877734000795, + 11.267896053032018, + 11.429298829985783, + 11.588455497985706, + 11.747167041059583, + 11.903655386995524, + 12.172861286089756, + 12.324467058992013, + 12.461568072088994 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.51714401692152, + "p95": 247.4815039895475, + "max": 278.4469010075554 + } + }, + { + "start": 723833.232123858, + "first": 723834.648967655, + "last": 723846.015724674, + "end": 723846.015864307, + "elapsed_s": 12.783740448998287, + "chunk_offsets_s": [ + 1.4168437970802188, + 1.587287885020487, + 1.7550395510625094, + 1.9180888390401378, + 2.0807291640667245, + 2.2423523840261623, + 2.4027657330734655, + 2.5684763920726255, + 2.739281919086352, + 2.9156005780678242, + 3.0956635270267725, + 3.2775097300764173, + 3.45645408204291, + 3.6350394600303844, + 3.8139102200511843, + 3.9910020870156586, + 4.166409739060327, + 4.344874392030761, + 4.521477261092514, + 4.700797408004291, + 4.875540556036867, + 5.042758385068737, + 5.208987188059837, + 5.372971392003819, + 5.530122148105875, + 5.687099403003231, + 5.842574148089625, + 5.99946003803052, + 6.157153584063053, + 6.316663115052506, + 6.474742439109832, + 6.63255206705071, + 6.788631940027699, + 6.947247846052051, + 7.1063194430898875, + 7.26739554409869, + 7.425253721070476, + 7.58514457300771, + 7.745060804998502, + 7.904432443086989, + 8.063936176011339, + 8.22317611402832, + 8.384151824051514, + 8.545084165059961, + 8.706297968048602, + 8.868004619027488, + 9.028168927063234, + 9.188234352041036, + 9.348301596008241, + 9.508423571009189, + 9.66815571906045, + 9.82659383106511, + 9.987783395103179, + 10.14742178900633, + 10.306552644004114, + 10.468462771037593, + 10.628653568099253, + 10.789242045022547, + 10.947805392090231, + 11.10790799302049, + 11.267999808071181, + 11.428531730081886, + 11.587914252071641, + 11.746494974009693, + 11.903107415069826, + 12.17321716004517, + 12.324369409005158, + 12.46125454211142, + 12.584162529092282, + 12.694233427057043, + 12.783600816037506 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.14314151834697, + "p95": 179.32014691177756, + "max": 270.10974497534335 + } + }, + { + "start": 723833.231408614, + "first": 723833.700512817, + "last": 723845.556587091, + "end": 723845.55668598, + "elapsed_s": 12.325277365976945, + "chunk_offsets_s": [ + 0.4691042030462995, + 0.5744632290443406, + 0.8007977230008692, + 1.0487528890371323, + 1.309801337076351, + 1.588218625052832, + 1.7560534779913723, + 1.9192671769997105, + 2.0819126300048083, + 2.243481583078392, + 2.403800879023038, + 2.5700912160100415, + 2.740119651076384, + 2.91686583601404, + 3.0966695500537753, + 3.2785365900490433, + 3.4575643229763955, + 3.635635588085279, + 3.814819609047845, + 3.9921750320354477, + 4.167319978005253, + 4.346114257001318, + 4.522463559056632, + 4.701898969011381, + 4.876738983090036, + 5.0436479119816795, + 5.210298344027251, + 5.374351795064285, + 5.531187060056254, + 5.688102925079875, + 5.843395163072273, + 6.000564029091038, + 6.158352048019879, + 6.317664841073565, + 6.475507355062291, + 6.6332074330421165, + 6.789170571020804, + 6.947722957003862, + 7.106777556007728, + 7.26763907703571, + 7.425687061040662, + 7.585919747012667, + 7.745524780009873, + 7.904853371088393, + 8.064426649012603, + 8.223728246986866, + 8.384740873007104, + 8.545881388010457, + 8.70686233707238, + 8.868676525074989, + 9.028767545009032, + 9.188379729050212, + 9.348937698989175, + 9.509185158996843, + 9.66882631299086, + 9.827099801041186, + 9.98836915998254, + 10.148176143062301, + 10.307200515992008, + 10.46903109701816, + 10.62940018903464, + 10.789678920060396, + 10.948421440087259, + 11.108561981003731, + 11.268502283026464, + 11.429874016088434, + 11.589035988086835, + 11.74783807399217, + 11.904512123088352, + 12.173472744994797, + 12.325178476981819 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.46353097772226, + "p95": 247.9551660362631, + "max": 278.417287976481 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 34.9, + "per_stream_tps": 34.9, + "ttft_s_max": 0.16, + "timing_valid": true, + "end_to_end_tps": 34.25549711861656, + "makespan_s": 7.473253099014983, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 34.875469208452365, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 7.473253099014983, + "end_to_end_tps": 34.25549711861656, + "timing_valid": true, + "per_stream_tps": [ + 34.9 + ], + "ttft_s": [ + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 723868.564497789, + "first": 723868.725610153, + "last": 723876.037339738, + "end": 723876.037750888, + "elapsed_s": 7.473253099014983, + "chunk_offsets_s": [ + 0.16111236403230578, + 0.2231533370213583, + 0.28339546697679907, + 0.34597520099487156, + 0.4111162209883332, + 0.4776321080280468, + 0.5402725029271096, + 0.5933647679630667, + 0.6508624110138044, + 0.7080513789551333, + 0.7641197629272938, + 0.8154002409428358, + 0.8670506739290431, + 0.918012346024625, + 0.9690536760026589, + 1.0214666259707883, + 1.0721574280178174, + 1.124875737936236, + 1.1764527650084347, + 1.227403228986077, + 1.2775897990213707, + 1.3303883579792455, + 1.3899995170067996, + 1.44955218792893, + 1.5060828640125692, + 1.5555974469752982, + 1.6082571409642696, + 1.6680495030013844, + 1.7261540440376848, + 1.7851973440265283, + 1.8456213949248195, + 1.9105484950123355, + 1.9717707600211725, + 2.0236461650347337, + 2.080652995966375, + 2.139706449001096, + 2.192476019030437, + 2.243013971950859, + 2.29486750299111, + 2.347902143956162, + 2.398962529026903, + 2.4519103629281744, + 2.50715434900485, + 2.558228819980286, + 2.6090812489856035, + 2.6595825699623674, + 2.7103054269682616, + 2.7616830019978806, + 2.811874978011474, + 2.8633951619267464, + 2.915382827981375, + 2.964707170962356, + 3.016920496011153, + 3.068569491035305, + 3.119010868947953, + 3.1697916869306937, + 3.2237594180041924, + 3.280208049924113, + 3.332312152022496, + 3.3830847509671003, + 3.435237811994739, + 3.4850680269300938, + 3.536309403949417, + 3.587142698932439, + 3.6386206549359486, + 3.6893963089678437, + 3.74087918701116, + 3.7916189449606463, + 3.845904455985874, + 3.895843056961894, + 3.9466917369281873, + 3.999132739030756, + 4.049735659034923, + 4.100033928989433, + 4.151955366018228, + 4.203186115948483, + 4.254060011007823, + 4.304025938967243, + 4.355550769949332, + 4.406888170982711, + 4.458570595015772, + 4.5098893359536305, + 4.562677790992893, + 4.615627020946704, + 4.6725562799256295, + 4.7246044089552015, + 4.776810826966539, + 4.829840683960356, + 4.88713213801384, + 4.944302742020227, + 4.99494710797444, + 5.045956135028973, + 5.09854916902259, + 5.1546124359592795, + 5.205191547982395, + 5.255149138974957, + 5.305786916986108, + 5.356623720028438, + 5.407971472945064, + 5.459477892960422, + 5.512141234008595, + 5.56888377701398, + 5.619664742960595, + 5.669956887024455, + 5.722324060974643, + 5.780066587030888, + 5.839265662012622, + 5.8952306270366535, + 5.946682867943309, + 6.000070650014095, + 6.059703817940317, + 6.11641606898047, + 6.17170426202938, + 6.231472256942652, + 6.296531299012713, + 6.359570596017875, + 6.41156841092743, + 6.4688712320057675, + 6.528201807988808, + 6.5851219430333, + 6.635379484971054, + 6.6883955670055, + 6.739205433987081, + 6.791255314019509, + 6.842580095981248, + 6.895798316923901, + 6.952139721950516, + 7.009763523936272, + 7.066572075942531, + 7.115992769016884, + 7.168056688038632, + 7.2197950349655, + 7.271686405991204, + 7.32236400898546, + 7.372731677023694, + 7.423094691941515, + 7.47284194896929 + ], + "chunk_gaps_ms": { + "count": 136, + "p50": 52.04900453099981, + "p95": 62.579734018072486, + "max": 66.51588703971356 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 49.9, + "per_stream_tps": 25.3, + "ttft_s_max": 0.38, + "timing_valid": true, + "end_to_end_tps": 49.33394520505358, + "makespan_s": 10.378249658970162, + "steady_tps": 50.7, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 49.91788947119949, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 10.378249658970162, + "end_to_end_tps": 49.33394520505358, + "timing_valid": true, + "steady_tps": 50.69863504744092, + "per_stream_tps": [ + 25.5, + 25.1 + ], + "ttft_s": [ + 0.38, + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 723876.040181864, + "first": 723876.422209471, + "last": 723886.417918103, + "end": 723886.418329587, + "elapsed_s": 10.378147722920403, + "chunk_offsets_s": [ + 0.3820276069454849, + 0.4657400199212134, + 0.5575283099897206, + 0.638066187966615, + 0.7242125229677185, + 0.8174366779858246, + 0.903078859904781, + 0.9897402679780498, + 1.071668598917313, + 1.159795293933712, + 1.2485759349074215, + 1.3360349799040705, + 1.4237320389365777, + 1.5066081209806725, + 1.601033762912266, + 1.6958697939990088, + 1.7907017800025642, + 1.8798588289646432, + 1.970242392970249, + 2.0640836869133636, + 2.1533908209530637, + 2.241262316936627, + 2.3331829509697855, + 2.429848098894581, + 2.5185295189730823, + 2.6065422559622675, + 2.6969680639449507, + 2.7873073959490284, + 2.8722477939445525, + 2.9600810389965773, + 3.0551972329849377, + 3.1461551039246842, + 3.2405811669304967, + 3.3344605040038005, + 3.4247922159265727, + 3.5109914019703865, + 3.593281944980845, + 3.6820434569381177, + 3.763642636942677, + 3.8536674439674243, + 3.9403513479046524, + 4.0285744779976085, + 4.119254988967441, + 4.210096779977903, + 4.3047009479487315, + 4.397682966897264, + 4.4893156898906454, + 4.580703270970844, + 4.671669703908265, + 4.761736440937966, + 4.853288537939079, + 4.942210181965493, + 5.0345165389589965, + 5.124415574944578, + 5.21888084791135, + 5.308152718003839, + 5.4034949879860505, + 5.498326672939584, + 5.591523521929048, + 5.687237607897259, + 5.7800748099107295, + 5.874108948977664, + 5.9647396869258955, + 6.057626595953479, + 6.153546300949529, + 6.246883231913671, + 6.335031003924087, + 6.433358262991533, + 6.531139036989771, + 6.624734915909357, + 6.71580389898736, + 6.803494336898439, + 6.898399444995448, + 6.994871494942345, + 7.0847616218961775, + 7.175955040962435, + 7.266305725905113, + 7.36116485693492, + 7.4556058859452605, + 7.546895828912966, + 7.63660170789808, + 7.725505513953976, + 7.820103628910147, + 7.911130732973106, + 8.00105471094139, + 8.095629210933112, + 8.192504696897231, + 8.286478180903941, + 8.372615007916465, + 8.462235422921367, + 8.555881616892293, + 8.646066147950478, + 8.729593632975593, + 8.815882284892723, + 8.909013184951618, + 8.99837469495833, + 9.09083944989834, + 9.183263471000828, + 9.277559405891225, + 9.36639792099595, + 9.45717829791829, + 9.546467546955682, + 9.63610505394172, + 9.725425511947833, + 9.813522573909722, + 9.90148345194757, + 9.994195296894759, + 10.085851608891971, + 10.178002653992735, + 10.260696287965402, + 10.328704224899411, + 10.377736238995567 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 90.6307379482314, + "p95": 95.9197049960494, + "max": 98.327259067446 + } + }, + { + "start": 723876.040079928, + "first": 723876.201140002, + "last": 723886.36796176, + "end": 723886.368140948, + "elapsed_s": 10.328061020001769, + "chunk_offsets_s": [ + 0.16106007399503142, + 0.22247708798386157, + 0.46540320303756744, + 0.5573333100182936, + 0.6377677799900994, + 0.7238707010401413, + 0.8172490489669144, + 0.9033926490228623, + 0.9894707850180566, + 1.0719519749982283, + 1.1598029530141503, + 1.2487791370367631, + 1.336248159990646, + 1.4240593579597771, + 1.5069484720006585, + 1.6013701480114833, + 1.6958591910079122, + 1.790381110040471, + 1.879645421053283, + 1.9700118920300156, + 2.063806607038714, + 2.1532578440383077, + 2.240906305029057, + 2.3328369490336627, + 2.429657298955135, + 2.5184226849814877, + 2.606192111968994, + 2.696757868048735, + 2.7871972039574757, + 2.8720334409736097, + 2.959747796994634, + 3.0547643559984863, + 3.1458504049805924, + 3.240370266023092, + 3.3340652169426903, + 3.4245359170017764, + 3.510740651981905, + 3.5931986790383235, + 3.6814950299449265, + 3.7634647879749537, + 3.85350931098219, + 3.940074805985205, + 4.02897031896282, + 4.1190737979486585, + 4.210019927006215, + 4.304301308002323, + 4.397291433997452, + 4.4888622789876536, + 4.58056775003206, + 4.671380328014493, + 4.761476197047159, + 4.853145141038112, + 4.941943967016414, + 5.034355310956016, + 5.124043092946522, + 5.218606309965253, + 5.307883230037987, + 5.403357938979752, + 5.497939990018494, + 5.591159853036515, + 5.687444373033941, + 5.7798462129430845, + 5.8737633869750425, + 5.964627400971949, + 6.0573711739853024, + 6.153215655009262, + 6.246695335954428, + 6.33432163100224, + 6.433282876969315, + 6.5307989900466055, + 6.624441522988491, + 6.715478859958239, + 6.803142201039009, + 6.898007962969132, + 6.994594529969618, + 7.084515342023224, + 7.1757790530100465, + 7.26606543501839, + 7.360813912004232, + 7.455976405995898, + 7.546645105001517, + 7.6362277349689975, + 7.725751954945736, + 7.819972184021026, + 7.910750478040427, + 8.000690585002303, + 8.095405486994423, + 8.192041179980151, + 8.28633647901006, + 8.372247174964286, + 8.462541303946637, + 8.555529473000206, + 8.645708141033538, + 8.729019387043081, + 8.815468293963932, + 8.908756752032787, + 8.998156900983304, + 9.090576146962121, + 9.182911669020541, + 9.277163331978954, + 9.366219806019217, + 9.45683048805222, + 9.546064210939221, + 9.63581675896421, + 9.725089407991618, + 9.813899178989232, + 9.901313751004636, + 9.993874449050054, + 10.085743156960234, + 10.177904319949448, + 10.2605980100343, + 10.327881832025014 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 90.61068203300238, + "p95": 96.58656700048596, + "max": 242.92611505370587 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.0, + "per_stream_tps": 17.8, + "ttft_s_max": 0.9, + "timing_valid": true, + "end_to_end_tps": 68.55453390507776, + "makespan_s": 14.937013522954658, + "steady_tps": 71.9, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.04775504144398, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 14.937013522954658, + "end_to_end_tps": 68.55453390507776, + "timing_valid": true, + "steady_tps": 71.94337546243555, + "per_stream_tps": [ + 17.7, + 17.9, + 18.2, + 17.5 + ], + "ttft_s": [ + 0.38, + 0.63, + 0.9, + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 723886.421027904, + "first": 723886.805050258, + "last": 723901.241296103, + "end": 723901.241598069, + "elapsed_s": 14.820570164942183, + "chunk_offsets_s": [ + 0.38402235391549766, + 0.4708960809512064, + 0.7379888769937679, + 1.0177560219308361, + 1.1395506149856374, + 1.2633611370110884, + 1.3873833159450442, + 1.5149193319957703, + 1.635996498982422, + 1.7577367949998006, + 1.8806477210018784, + 2.005702171009034, + 2.1286069150082767, + 2.2498259419808164, + 2.377100521000102, + 2.5077373649692163, + 2.645329838967882, + 2.780672669992782, + 2.9164635239867494, + 3.0481732089538127, + 3.1787939969217405, + 3.3084215900162235, + 3.4367938609793782, + 3.5686663730302826, + 3.7048431899165735, + 3.8384818530175835, + 3.965570420026779, + 4.0926653479691595, + 4.218643272994086, + 4.342139747925103, + 4.471737507963553, + 4.604538828949444, + 4.738885495928116, + 4.869769721990451, + 4.999875492998399, + 5.130390550941229, + 5.253079597023316, + 5.3723165799165145, + 5.489654619945213, + 5.61089978099335, + 5.731230383971706, + 5.859613273991272, + 5.986799077014439, + 6.114014016930014, + 6.245671977987513, + 6.378533354960382, + 6.511118883965537, + 6.641682780929841, + 6.7725060529774055, + 6.903695608954877, + 7.0335568089503795, + 7.164205373963341, + 7.292079147999175, + 7.416429591947235, + 7.548819008981809, + 7.677655367995612, + 7.806983029004186, + 7.938743120990694, + 8.072315337019973, + 8.205610723933205, + 8.336657083942555, + 8.46661994594615, + 8.593370223999955, + 8.720604722970165, + 8.851244976976886, + 8.980924939969555, + 9.108548454008996, + 9.23946252092719, + 9.371845050016418, + 9.5064887519693, + 9.639400289976038, + 9.761989103979431, + 9.891381029970944, + 10.022412639926188, + 10.154284553951584, + 10.28660666802898, + 10.413742188015021, + 10.542737956973724, + 10.674796028994024, + 10.805064447922632, + 10.933251710026525, + 11.058940535993315, + 11.185754662961699, + 11.314716359018348, + 11.443412280990742, + 11.57093685597647, + 11.704220011015423, + 11.843804005999118, + 11.978041530004703, + 12.108840919914655, + 12.234686160925776, + 12.361862893914804, + 12.484195154975168, + 12.604448064928874, + 12.728529892978258, + 12.853460106998682, + 12.980613866006024, + 13.107860573916696, + 13.237354498007335, + 13.36474567395635, + 13.492577679920942, + 13.621721862000413, + 13.751441685948521, + 13.876694369013421, + 13.998120544943959, + 14.118385458015837, + 14.243491668952629, + 14.37286978692282, + 14.496272619930096, + 14.61810488102492, + 14.729203397990204, + 14.820268198964186 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 128.8363590138033, + "p95": 135.79085399396718, + "max": 279.7671449370682 + } + }, + { + "start": 723886.420055695, + "first": 723887.049902048, + "last": 723901.30805756, + "end": 723901.308170314, + "elapsed_s": 14.888114618952386, + "chunk_offsets_s": [ + 0.6298463529674336, + 0.7384762440342456, + 1.0183640309842303, + 1.1401609829626977, + 1.2637736370088533, + 1.3879999399650842, + 1.515998663031496, + 1.6372061469592154, + 1.758951568044722, + 1.8814495210535824, + 2.0059593400219455, + 2.128794015967287, + 2.250273652956821, + 2.3778374020475894, + 2.508579624001868, + 2.6461521750316024, + 2.7815113130491227, + 2.916494371020235, + 3.0489178559510037, + 3.1794247010257095, + 3.309149786015041, + 3.4374095340026543, + 3.5694691439857706, + 3.7054022869560868, + 3.839658480952494, + 3.966747293015942, + 4.09343018499203, + 4.219288698979653, + 4.343146729050204, + 4.47279846703168, + 4.6055492899613455, + 4.739824061049148, + 4.870786607032642, + 5.000788792967796, + 5.131399177014828, + 5.254012265009806, + 5.37332387000788, + 5.490662537980825, + 5.611831643967889, + 5.7322720469674096, + 5.860528819961473, + 5.987733865040354, + 6.115078459959477, + 6.246605011983775, + 6.379467612016015, + 6.512129334034398, + 6.642615156015381, + 6.773444107035175, + 6.904631737968884, + 7.034495173022151, + 7.164911697967909, + 7.292410769034177, + 7.4173468459630385, + 7.549743450013921, + 7.6787025820231065, + 7.807800099020824, + 7.939519732957706, + 8.073219442972913, + 8.206541731022298, + 8.337822976987809, + 8.467756657977588, + 8.594781595049426, + 8.721787714981474, + 8.852295679971576, + 8.98185028100852, + 9.10957403597422, + 9.24037377606146, + 9.372595816035755, + 9.507236514007673, + 9.640147394035012, + 9.763037301017903, + 9.892271358054131, + 10.023341965978034, + 10.15521388000343, + 10.28764804196544, + 10.414505135966465, + 10.543637371971272, + 10.675816916977055, + 10.806114683975466, + 10.934151681954972, + 11.060080041061155, + 11.186899769003503, + 11.315475821029395, + 11.444992159027606, + 11.572055642027408, + 11.705395065015182, + 11.845114086056128, + 11.979064646991901, + 12.109596807044, + 12.235695463954471, + 12.362875774037093, + 12.485223855008371, + 12.605455458979122, + 12.729541233973578, + 12.854490987956524, + 12.981623959960416, + 13.108867492992431, + 13.238364864955656, + 13.365499039995484, + 13.493697131983936, + 13.622893249965273, + 13.75244988698978, + 13.877629552036524, + 13.99926311604213, + 14.118919976055622, + 14.24442690005526, + 14.374047924997285, + 14.497194717056118, + 14.619008249021135, + 14.730135645950213, + 14.821999932988547, + 14.888001865008846 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 128.57605202589184, + "p95": 134.98305797111243, + "max": 279.88778694998473 + } + }, + { + "start": 723886.420655156, + "first": 723887.316789542, + "last": 723901.356826425, + "end": 723901.35694301, + "elapsed_s": 14.936287853983231, + "chunk_offsets_s": [ + 0.89613438595552, + 1.0183437729720026, + 1.139639773988165, + 1.2633619089610875, + 1.3880268179345876, + 1.5151828569360077, + 1.6359521079575643, + 1.7585874439682811, + 1.8812667590100318, + 2.0055902019375935, + 2.128260975005105, + 2.249850510037504, + 2.3779238399583846, + 2.508196891983971, + 2.6457475499482825, + 2.7810962579678744, + 2.9165792919229716, + 3.048655046033673, + 3.1790043390356004, + 3.308669927995652, + 3.4368823419790715, + 3.5686275250045583, + 3.7045826009707525, + 3.839171301922761, + 3.966073198011145, + 4.093270716024563, + 4.218875613994896, + 4.342832303023897, + 4.47231121500954, + 4.605154138989747, + 4.739494925015606, + 4.870378184947185, + 5.000426077982411, + 5.130942358984612, + 5.253662155941129, + 5.372822955017909, + 5.490298646967858, + 5.61105874099303, + 5.731752375955693, + 5.860337001970038, + 5.9876869870349765, + 6.114851891994476, + 6.246264185989276, + 6.379363748943433, + 6.511825944995508, + 6.642227303003892, + 6.77314382197801, + 6.90420093794819, + 7.034094695933163, + 7.164659671019763, + 7.2921931489836425, + 7.416966482996941, + 7.549304352025501, + 7.678178961970843, + 7.807554343016818, + 7.939255784964189, + 8.072830394958146, + 8.20607289893087, + 8.337491473997943, + 8.467280866927467, + 8.59395566699095, + 8.721341250929981, + 8.851814336958341, + 8.981452876934782, + 9.10915514093358, + 9.239977912977338, + 9.37231293192599, + 9.506963282939978, + 9.640031054033898, + 9.762148696929216, + 9.89185301400721, + 10.023117806995288, + 10.15493192395661, + 10.287224079016596, + 10.414172076038085, + 10.543249886017293, + 10.675280760973692, + 10.80564376001712, + 10.933923903037794, + 11.059208947932348, + 11.18603570596315, + 11.315146530978382, + 11.44389307196252, + 11.571649011923, + 11.704924703924917, + 11.844559420016594, + 11.978711221017875, + 12.109502571984194, + 12.23534346499946, + 12.362418815959245, + 12.484723481931724, + 12.6050900160335, + 12.72909104602877, + 12.854059636942111, + 12.981167122954503, + 13.108397951931693, + 13.237885155016556, + 13.365070816944353, + 13.493152510025539, + 13.622580847935751, + 13.752094403025694, + 13.877128196996637, + 13.998725584940985, + 14.118461019010283, + 14.244014477939345, + 14.373657632037066, + 14.496760759036988, + 14.618332169950008, + 14.72975848603528, + 14.821330219972879, + 14.887804762925953, + 14.936171268927865 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 128.5846260143444, + "p95": 134.65035101398826, + "max": 139.63471609167755 + } + }, + { + "start": 723886.419929487, + "first": 723886.584441727, + "last": 723901.150620004, + "end": 723901.150790494, + "elapsed_s": 14.730861007003114, + "chunk_offsets_s": [ + 0.16451223997864872, + 0.22658963792491704, + 0.47182539489585906, + 0.7389930379576981, + 1.01866830594372, + 1.1405844709370285, + 1.264689847943373, + 1.3882376499241218, + 1.5160704129375517, + 1.6368191669462249, + 1.7591628489317372, + 1.8820924459723756, + 2.006577724008821, + 2.1294864889932796, + 2.2508853849722072, + 2.37847822799813, + 2.5088826349237934, + 2.646500019938685, + 2.781880871974863, + 2.9171319419983774, + 3.0493539589224383, + 3.1796978489728644, + 3.3094500119332224, + 3.4376579099334776, + 3.5694572179345414, + 3.7056113919243217, + 3.8398586669936776, + 3.9667438420001417, + 4.0939478209475055, + 4.219567752908915, + 4.3435130649013445, + 4.473002290003933, + 4.605834208894521, + 4.740175526007079, + 4.871153279906139, + 5.0011890979949385, + 5.131715215975419, + 5.254342358908616, + 5.373582545900717, + 5.491113154916093, + 5.6118919079890475, + 5.732516716001555, + 5.8608037799131125, + 5.9881698199315, + 6.115385852986947, + 6.247027404955588, + 6.379794938955456, + 6.5125167509540915, + 6.643003167002462, + 6.773821612936445, + 6.904975708923303, + 7.034790442907251, + 7.165119229000993, + 7.292865872965194, + 7.417646633926779, + 7.55006642290391, + 7.678990134969354, + 7.808243947918527, + 7.940013592946343, + 8.073588716913946, + 8.206830673967488, + 8.338166957953945, + 8.467973609920591, + 8.59473762800917, + 8.722024238901213, + 8.852572690928355, + 8.982215856900439, + 9.10991604893934, + 9.2407361069927, + 9.37300099991262, + 9.507656201953068, + 9.64068187493831, + 9.762915515922941, + 9.892545769922435, + 10.023757180897519, + 10.155594014911912, + 10.287914442946203, + 10.414929967955686, + 10.543929877923802, + 10.676042506005615, + 10.806402453919873, + 10.934573050006293, + 11.060101190931164, + 11.186642803950235, + 11.316055159899406, + 11.444840186974034, + 11.572591071948409, + 11.705613427911885, + 11.844991307007149, + 11.979700075928122, + 12.110300198895857, + 12.236318938899785, + 12.363192616961896, + 12.48549136298243, + 12.605768755893223, + 12.729868552996777, + 12.854845742927864, + 12.98194905789569, + 13.10917836392764, + 13.238655220950022, + 13.365745447925292, + 13.493925093906, + 13.623351138900034, + 13.752865928923711, + 13.877913934993558, + 13.999530982924625, + 14.119419219903648, + 14.244822793989442, + 14.374324898933992, + 14.497546033002436, + 14.618955996935256, + 14.730690516997129 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 128.9999099681154, + "p95": 136.15417398978025, + "max": 279.67526798602194 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 71.6, + "per_stream_tps": 12.4, + "ttft_s_max": 20.97, + "timing_valid": true, + "end_to_end_tps": 71.49847345532, + "makespan_s": 28.64396819996182, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 71.61895045235113, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 28.64396819996182, + "end_to_end_tps": 71.49847345532, + "timing_valid": true, + "per_stream_tps": [ + 12.8, + 12.3, + 33.2, + 12.6, + 12.4, + 12.3, + 12.5, + 12.3 + ], + "ttft_s": [ + 1.78, + 0.62, + 20.97, + 1.47, + 0.89, + 0.16, + 1.17, + 0.38 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 723901.361440517, + "first": 723903.140187766, + "last": 723923.067117057, + "end": 723923.067200801, + "elapsed_s": 21.705760283977725, + "chunk_offsets_s": [ + 1.7787472490454093, + 1.951588321942836, + 2.128147993935272, + 2.2996873760130256, + 2.4754667839733884, + 2.6540840789675713, + 2.828331131953746, + 3.002766308025457, + 3.174267729045823, + 3.350517460028641, + 3.5305301930056885, + 3.7136490809498355, + 3.8987512949388474, + 4.086826525977813, + 4.276815184042789, + 4.466929213958792, + 4.653457393986173, + 4.8403589859372005, + 5.027361479005776, + 5.215636049048044, + 5.401207052986138, + 5.590149127994664, + 5.777709332993254, + 5.962455929955468, + 6.139519362011924, + 6.322031422983855, + 6.508056010934524, + 6.696836877963506, + 6.884331497945823, + 7.0723271439783275, + 7.260613605962135, + 7.441466199001297, + 7.6222405940061435, + 7.799064279999584, + 7.971398968016729, + 8.141386306961067, + 8.313902811030857, + 8.489552980987355, + 8.665335605968721, + 8.849583224044181, + 9.032514111953788, + 9.216850899974816, + 9.402974077034742, + 9.589623530977406, + 9.776087785023265, + 9.962062934995629, + 10.146005104994401, + 10.326821577968076, + 10.508238445036113, + 10.692638239939697, + 10.87190923304297, + 11.051982732024044, + 11.236314720008522, + 11.42086468404159, + 11.607500990037806, + 11.791460381937213, + 11.979595592012629, + 12.162525968975388, + 12.343029654934071, + 12.524478752049617, + 12.704585075960495, + 12.884772298042662, + 13.070245316019282, + 13.255065585952252, + 13.436976048047654, + 13.619877125951461, + 13.796221237047575, + 13.979948216001503, + 14.167287293006666, + 14.347556483000517, + 14.529508176958188, + 14.712660650955513, + 14.896325879963115, + 15.081146587966941, + 15.26649531594012, + 15.447672378038988, + 15.630879202042706, + 15.812781285028905, + 15.994157072040252, + 16.171386695001274, + 16.350470589008182, + 16.53638070996385, + 16.72611278295517, + 16.916745738009922, + 17.107500556041487, + 17.29988199402578, + 17.485350925009698, + 17.66417934501078, + 17.837747366982512, + 18.016368216020055, + 18.193680475000292, + 18.369487583986484, + 18.54662424104754, + 18.724517815979198, + 18.90119154099375, + 19.07883575395681, + 19.26160921494011, + 19.44503539800644, + 19.627717016963288, + 19.80512837797869, + 19.977777285967022, + 20.14847606094554, + 20.317913612001576, + 20.486698135966435, + 20.652679148945026, + 20.81013644603081, + 21.129384455969557, + 21.28191259293817, + 21.415038569946773, + 21.529819658957422, + 21.625592563999817, + 21.705676540033892 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 181.91046209540218, + "p95": 189.98865806497633, + "max": 319.2480099387467 + } + }, + { + "start": 723901.360641621, + "first": 723901.983378058, + "last": 723922.643678167, + "end": 723922.643832523, + "elapsed_s": 21.283190901973285, + "chunk_offsets_s": [ + 0.6227364368969575, + 0.7312923849094659, + 1.0097732989815995, + 1.3075354458997026, + 1.6213880099821836, + 1.9524385499535128, + 2.128392886952497, + 2.3005758119979873, + 2.4763400269439444, + 2.654670527903363, + 2.829443697934039, + 3.0033919509733096, + 3.175113879959099, + 3.3509589369641617, + 3.5310521268984303, + 3.7144867558963597, + 3.8993502099765465, + 4.087662433972582, + 4.2778804959962144, + 4.467771001975052, + 4.65376389794983, + 4.841119605000131, + 5.028090157895349, + 5.216546975891106, + 5.402172111906111, + 5.590875979978591, + 5.779184689978138, + 5.9630540299694985, + 6.1403550379909575, + 6.322698739939369, + 6.508565750904381, + 6.697340386919677, + 6.884807597962208, + 7.0730140139348805, + 7.260883044917136, + 7.442106056958437, + 7.622888893936761, + 7.799351337947883, + 7.972032927908003, + 8.14213560090866, + 8.314328110893257, + 8.489924706984311, + 8.665792618994601, + 8.850335304974578, + 9.03313283692114, + 9.21757469989825, + 9.403556878911331, + 9.590570631902665, + 9.77672572794836, + 9.962824773974717, + 10.146631189971231, + 10.327655208995566, + 10.508960279985331, + 10.69329885195475, + 10.872531073982827, + 11.052535915980116, + 11.23721160297282, + 11.42185907298699, + 11.607625456992537, + 11.792208843980916, + 11.979827384930104, + 12.163019030936994, + 12.343385791988112, + 12.525186519953422, + 12.705211848951876, + 12.885605496936478, + 13.07101449498441, + 13.255726954899728, + 13.437194509897381, + 13.619895188952796, + 13.796752224909142, + 13.980636288993992, + 14.168049481930211, + 14.348209128947929, + 14.530504306894727, + 14.713843157980591, + 14.897740392945707, + 15.082183613907546, + 15.267457563895732, + 15.44928383093793, + 15.632107223966159, + 15.813993830000982, + 15.99547811690718, + 16.172534878947772, + 16.351675662910566, + 16.537733013974503, + 16.727513374993578, + 16.91805061791092, + 17.108726074919105, + 17.301328195957467, + 17.486228956957348, + 17.66465916892048, + 17.839265188900754, + 18.017534847953357, + 18.19505089998711, + 18.370928609976545, + 18.548116008983925, + 18.72571986890398, + 18.90267547289841, + 19.080229350947775, + 19.262709803995676, + 19.44635268091224, + 19.62859373097308, + 19.806262581958435, + 19.97917956591118, + 20.149732261896133, + 20.318939752993174, + 20.487941717961803, + 20.65408664289862, + 20.811449292930774, + 21.130565363913774, + 21.283036545966752 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 182.4804530479014, + "p95": 192.60212103836238, + "max": 331.0505399713293 + } + }, + { + "start": 723901.36081419, + "first": 723922.329985778, + "last": 723930.00331069, + "end": 723930.003483805, + "elapsed_s": 28.642669614986517, + "chunk_offsets_s": [ + 20.969171587959863, + 21.130619331030175, + 21.282807455048896, + 21.416182041051798, + 21.530543179949746, + 21.626400516950525, + 21.706882882048376, + 21.771424527978525, + 21.827808127971366, + 21.878203109954484, + 21.92882545304019, + 21.980237366980873, + 22.030948081985116, + 22.082494604983367, + 22.135009645018727, + 22.186531201004982, + 22.23822298704181, + 22.290623696986586, + 22.34200394200161, + 22.392781240982004, + 22.443614106043242, + 22.495909685036168, + 22.55585748597514, + 22.615076881018467, + 22.67129719303921, + 22.722125930944458, + 22.77510983403772, + 22.83540377195459, + 22.894410936976783, + 22.953405085019767, + 23.011259827995673, + 23.069524175021797, + 23.13136255298741, + 23.183701672940515, + 23.2410599719733, + 23.300848884973675, + 23.353266126010567, + 23.40543212997727, + 23.456725724041462, + 23.509473968995735, + 23.561500206938945, + 23.614460548036732, + 23.66944752796553, + 23.72094022994861, + 23.771273985039443, + 23.82207315193955, + 23.872653455939144, + 23.92386018903926, + 23.974122802959755, + 24.025243256939575, + 24.07722257601563, + 24.1275430789683, + 24.17863475601189, + 24.230677737970836, + 24.28135670698248, + 24.331980603048578, + 24.38552733696997, + 24.442653249949217, + 24.493177155964077, + 24.543956786976196, + 24.595951316994615, + 24.646000290988013, + 24.696566074038856, + 24.74828058097046, + 24.799587978981435, + 24.850399728980847, + 24.902627686969936, + 24.95347449905239, + 25.006115535041317, + 25.056240811012685, + 25.107829681946896, + 25.16033097205218, + 25.21105110796634, + 25.26218337600585, + 25.31431439705193, + 25.36567130300682, + 25.416443716036156, + 25.466637984965928, + 25.517966976971366, + 25.569994117948227, + 25.622159627033398, + 25.67288336204365, + 25.72578561899718, + 25.77837485796772, + 25.834655512939207, + 25.886355223017745, + 25.939396178000607, + 25.991746490006335, + 26.049679256044328, + 26.106843622983433, + 26.156181014026515, + 26.20730609993916, + 26.26044343202375, + 26.31636228901334, + 26.366452001966536, + 26.41707798303105, + 26.467554296017624, + 26.51831106794998, + 26.569153615040705, + 26.620536318048835, + 26.672751646954566, + 26.730941788991913, + 26.782437829999253, + 26.833101322990842, + 26.8862069810275, + 26.944688464980572, + 27.002471270971, + 27.058273104950786, + 27.11052681703586, + 27.163249623961747, + 27.224468937027268, + 27.281854939996265, + 27.338386212009937, + 27.39762714400422, + 27.46279224404134, + 27.524984941002913, + 27.578062172979116, + 27.63491006800905, + 27.69374589493964, + 27.751977127045393, + 27.802412642049603, + 27.854965378995985, + 27.906577310990542, + 27.95882124302443, + 28.009360139025375, + 28.063446513027884, + 28.120764710009098, + 28.178261411958374, + 28.235773775959387, + 28.28519474796485, + 28.337632591021247, + 28.38912477705162, + 28.44091428501997, + 28.491412705974653, + 28.542013401980512, + 28.593254402978346, + 28.642496500047855 + ], + "chunk_gaps_ms": { + "count": 136, + "p50": 52.16575652593747, + "p95": 65.16510003712028, + "max": 161.44774307031184 + } + }, + { + "start": 723901.360503472, + "first": 723902.826061296, + "last": 723922.98714371, + "end": 723922.987223519, + "elapsed_s": 21.626720046973787, + "chunk_offsets_s": [ + 1.4655578240053728, + 1.6221479279920459, + 1.95276648493018, + 2.1287580189527944, + 2.300746670924127, + 2.476440866012126, + 2.6548470120178536, + 2.828755986993201, + 3.003146881936118, + 3.174763996968977, + 3.3507477599196136, + 3.530896120937541, + 3.7141870600171387, + 3.899402532959357, + 4.087222407921217, + 4.277479419950396, + 4.4675134680001065, + 4.654352220939472, + 4.841540319961496, + 5.028527710936032, + 5.216828185948543, + 5.402370865922421, + 5.591051696916111, + 5.778884329018183, + 5.963040768983774, + 6.140687495004386, + 6.322881934000179, + 6.508776112925261, + 6.69730714999605, + 6.885316999978386, + 7.073190187918954, + 7.261211688979529, + 7.441911828005686, + 7.62327883800026, + 7.799280041945167, + 7.971950516919605, + 8.14218557998538, + 8.314683760982007, + 8.490458105923608, + 8.666103983996436, + 8.850562375970185, + 9.033234326983802, + 9.2177531439811, + 9.403779860935174, + 9.590741997002624, + 9.77694456197787, + 9.963193306000903, + 10.146859035012312, + 10.327390420949087, + 10.509140209993348, + 10.693537603016011, + 10.87288517691195, + 11.052986905910075, + 11.23728321492672, + 11.421880370937288, + 11.608247024007142, + 11.792280034977011, + 11.980251897941343, + 12.163654814008623, + 12.344010129920207, + 12.525518978014588, + 12.705703146988526, + 12.885669684968889, + 13.07122055196669, + 13.255950373015366, + 13.437433530925773, + 13.620112707954831, + 13.79711275396403, + 13.98092068801634, + 14.168296742951497, + 14.34845551091712, + 14.530673550907522, + 14.713798014912754, + 14.89782757498324, + 15.08189901697915, + 15.267946864012629, + 15.448862390010618, + 15.632011655950919, + 15.813819792005233, + 15.995832170010544, + 16.1724823549157, + 16.35151333990507, + 16.537548512918875, + 16.727593482937664, + 16.91809915693011, + 17.108941218000837, + 17.3013026349945, + 17.485947965993546, + 17.66484917490743, + 17.839173988904804, + 18.017574299010448, + 18.19503163092304, + 18.370903727947734, + 18.54806744295638, + 18.72577313601505, + 18.902719064964913, + 19.080330190015957, + 19.26288161694538, + 19.446538278949447, + 19.628467560978606, + 19.806300587952137, + 19.979242147994228, + 20.149837033008225, + 20.31929265591316, + 20.488043657969683, + 20.654311202000827, + 20.811559278983623, + 21.130973584018648, + 21.282642934005708, + 21.416155065991916, + 21.530901159974746, + 21.626640237984248 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 182.0123780053109, + "p95": 190.25701202917844, + "max": 330.61855693813413 + } + }, + { + "start": 723901.359727726, + "first": 723902.248828084, + "last": 723922.777207412, + "end": 723922.777344639, + "elapsed_s": 21.4176169129787, + "chunk_offsets_s": [ + 0.8891003580065444, + 1.0106058469973505, + 1.308660460053943, + 1.6221666680648923, + 1.9530656520510092, + 2.129941735067405, + 2.301107684033923, + 2.4768440930638462, + 2.655455682077445, + 2.8296842250274494, + 3.0041230430360883, + 3.1755903150187805, + 3.3516893080668524, + 3.53189416998066, + 3.7149091090541333, + 3.9000093530630693, + 4.088245327002369, + 4.278976947069168, + 4.4683332910062745, + 4.654404471046291, + 4.84167218406219, + 5.028749313089065, + 5.217735604033805, + 5.402654018020257, + 5.591505578020588, + 5.779130041017197, + 5.96367220103275, + 6.1409713990287855, + 6.323233596980572, + 6.5091582119930536, + 6.6978221259778365, + 6.8853933800710365, + 7.073586352053098, + 7.262082242988981, + 7.442407428054139, + 7.623293898068368, + 7.799683141987771, + 7.972391106071882, + 8.142655351082794, + 8.315351381083019, + 8.49106406304054, + 8.666808938025497, + 8.850808560033329, + 9.033857190981507, + 9.218231022008695, + 9.404215379036032, + 9.591065632062964, + 9.777258667978458, + 9.963530707056634, + 10.147237112978473, + 10.328333580982871, + 10.509504916029982, + 10.693841743050143, + 10.873353848000988, + 11.05312902701553, + 11.2376791840652, + 11.422501523047686, + 11.60809269407764, + 11.793514127028175, + 11.980866583064198, + 12.163553546997719, + 12.344063690048642, + 12.52557101903949, + 12.70573157898616, + 12.886093736975454, + 13.07159337203484, + 13.256135870004073, + 13.438863837043755, + 13.620469653978944, + 13.797485047020018, + 13.981231448007748, + 14.168599062017165, + 14.348910251050256, + 14.531008179066703, + 14.714504503994249, + 14.898537214030512, + 15.083221402019262, + 15.268448187038302, + 15.450124563998543, + 15.63293208007235, + 15.814875269075856, + 15.996668739011511, + 16.173413612996228, + 16.352552158990875, + 16.538617168087512, + 16.728183400002308, + 16.918997996021062, + 17.109680218040012, + 17.302402490051463, + 17.487109928042628, + 17.665831275982782, + 17.840133352088742, + 18.017999132047407, + 18.195917012984864, + 18.37178823107388, + 18.548928310978226, + 18.726670240052044, + 18.903544320026413, + 19.081181116052903, + 19.263551334035583, + 19.44735381298233, + 19.629306219983846, + 19.80711254104972, + 19.980130341020413, + 20.15068700304255, + 20.320032210089266, + 20.488957887981087, + 20.6550504360348, + 20.81245572399348, + 21.13194915605709, + 21.283810560009442, + 21.417479685973376 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 182.26219795178622, + "p95": 190.81459601875395, + "max": 330.89898398611695 + } + }, + { + "start": 723901.359639902, + "first": 723901.519229492, + "last": 723922.171712712, + "end": 723922.171788742, + "elapsed_s": 20.812148839933798, + "chunk_offsets_s": [ + 0.15958959003910422, + 0.22305585397407413, + 0.46585051994770765, + 0.7320418070303276, + 1.0109534539515153, + 1.3082014509709552, + 1.6220460169715807, + 1.9528016790281981, + 2.130086982040666, + 2.301039602025412, + 2.4767667959677055, + 2.6554518989287317, + 2.829721870017238, + 3.004060238949023, + 3.175722589949146, + 3.3515006189700216, + 3.531657104031183, + 3.7148774249944836, + 3.9001609849510714, + 4.087974512018263, + 4.278266820940189, + 4.468039114959538, + 4.654420948005281, + 4.84143360494636, + 5.028699040994979, + 5.217782703926787, + 5.402786970953457, + 5.591640757978894, + 5.779177300981246, + 5.963607877958566, + 6.141026512952521, + 6.3234097169479355, + 6.509404381969944, + 6.697810841025785, + 6.885354990954511, + 7.073573234025389, + 7.261626370018348, + 7.442211790941656, + 7.623260805965401, + 7.799668325926177, + 7.972377898986451, + 8.142523239017464, + 8.315397161990404, + 8.490997441927902, + 8.666618415038101, + 8.85084586800076, + 9.033785630948842, + 9.21820741193369, + 9.40416966797784, + 9.591058501973748, + 9.777196060982533, + 9.963481431943364, + 10.147105637006462, + 10.328363321023062, + 10.50949462002609, + 10.693770944955759, + 10.873059169040062, + 11.053121699020267, + 11.237671390990727, + 11.422918232972734, + 11.608225876931101, + 11.793497895007022, + 11.980654902989045, + 12.163419359945692, + 12.344108516001143, + 12.525744281010702, + 12.705782303004526, + 12.886149686994031, + 13.071635828004219, + 13.256127298926003, + 13.438673153985292, + 13.620636465027928, + 13.797520564985462, + 13.981257881969213, + 14.168596334988251, + 14.349061399931088, + 14.53113310993649, + 14.714171243016608, + 14.898218116955832, + 15.083024075021967, + 15.268347796984017, + 15.449670176953077, + 15.633179722935893, + 15.814831349998713, + 15.996300995000638, + 16.173310348996893, + 16.352489955024794, + 16.5386010769289, + 16.72800282703247, + 16.91859753895551, + 17.109340043971315, + 17.301805511000566, + 17.48704491602257, + 17.665164690930396, + 17.83983791898936, + 18.018253840971738, + 18.195680900942534, + 18.371405270998366, + 18.54856732301414, + 18.726275637978688, + 18.903147064032964, + 19.08092374796979, + 19.2634607499931, + 19.447039976948872, + 19.62969808699563, + 19.806845834013075, + 19.97962138603907, + 20.150417446973734, + 20.319844041951, + 20.48859318997711, + 20.65461803902872, + 20.812072810018435 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 182.65811004675925, + "p95": 242.79466597363353, + "max": 330.75566205661744 + } + }, + { + "start": 723901.35999062, + "first": 723902.529187675, + "last": 723922.891442185, + "end": 723922.891519419, + "elapsed_s": 21.531528798979707, + "chunk_offsets_s": [ + 1.1691970549291, + 1.3080230460036546, + 1.621737791923806, + 1.9525460939621553, + 2.129514248925261, + 2.300789001979865, + 2.47651927289553, + 2.6551537959603593, + 2.8291062829084694, + 3.003954273997806, + 3.1752087629865855, + 3.3513826999114826, + 3.5313546509714797, + 3.714572519995272, + 3.8996822979534045, + 4.087675240938552, + 4.278195287915878, + 4.467729477910325, + 4.654211801942438, + 4.841175943962298, + 5.028432717896067, + 5.217273592948914, + 5.402536236913875, + 5.591340439976193, + 5.779024290968664, + 5.963213108945638, + 6.140636419993825, + 6.32309289497789, + 6.508947490947321, + 6.697613455937244, + 6.885086529888213, + 7.073270007967949, + 7.261597874923609, + 7.442026949953288, + 7.6232480889884755, + 7.799371842993423, + 7.972199115902185, + 8.142428589984775, + 8.315139450947754, + 8.490705743897706, + 8.666227591922507, + 8.850652189925313, + 9.033531372901052, + 9.218045321991667, + 9.403698306996375, + 9.590857414994389, + 9.777275912929326, + 9.963666169904172, + 10.147182693937793, + 10.328591569908895, + 10.509567016968504, + 10.693834181991406, + 10.873632093891501, + 11.053379927994683, + 11.237724230973981, + 11.422753623919562, + 11.60855109395925, + 11.792973042000085, + 11.9802147669252, + 12.163486911915243, + 12.344212304917164, + 12.525725132902153, + 12.705773335997947, + 12.886132248910144, + 13.071622269926593, + 13.25625080394093, + 13.438773910980672, + 13.621080599958077, + 13.797570118913427, + 13.981231910991482, + 14.16877178591676, + 14.348894485970959, + 14.531123700900935, + 14.714277692954056, + 14.898240954964422, + 15.082877606968395, + 15.268148935982026, + 15.449902167893015, + 15.63262897497043, + 15.814576544915326, + 15.996176237938926, + 16.173254569992423, + 16.35236024193, + 16.538313149940223, + 16.72788225999102, + 16.91866579989437, + 17.10931901494041, + 17.301899500889704, + 17.486383665003814, + 17.66508718393743, + 17.839791960897855, + 18.01814922899939, + 18.19550479191821, + 18.3716166469967, + 18.54880919493735, + 18.726216472918168, + 18.903168015996926, + 19.080761641962454, + 19.263326053973287, + 19.44712488690857, + 19.629097906989045, + 19.80677371600177, + 19.979719175957143, + 20.15031273290515, + 20.319631567923352, + 20.488664750941098, + 20.654696589917876, + 20.812040719902143, + 21.131321235909127, + 21.28357958898414, + 21.41676584794186, + 21.53145156498067 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 182.30668897740543, + "p95": 190.6532150460407, + "max": 330.8083020383492 + } + }, + { + "start": 723901.359515605, + "first": 723901.740494205, + "last": 723922.490740046, + "end": 723922.490836473, + "elapsed_s": 21.131320867920294, + "chunk_offsets_s": [ + 0.38097860000561923, + 0.46623460098635405, + 0.732450962997973, + 1.010704256943427, + 1.3087514489889145, + 1.6224666279740632, + 1.9533964969450608, + 2.1294571210164577, + 2.3019182489952073, + 2.4773377829696983, + 2.6557556280167773, + 2.8305423730053008, + 3.004581694956869, + 3.176417332026176, + 3.352350024972111, + 3.532634094939567, + 3.715438876999542, + 3.900522999931127, + 4.08863543800544, + 4.279116119025275, + 4.468774864915758, + 4.654942433931865, + 4.842107526026666, + 5.0292517869966105, + 5.2176248979521915, + 5.403514242963865, + 5.592285620979965, + 5.779984786990099, + 5.964422298013233, + 6.141827140003443, + 6.323634019936435, + 6.510128360008821, + 6.698667054995894, + 6.886127186007798, + 7.074353622971103, + 7.261670338921249, + 7.443035954958759, + 7.623866123962216, + 7.800676850951277, + 7.973651540931314, + 8.143642436014488, + 8.315375230973586, + 8.491382948937826, + 8.667444625985809, + 8.851427205954678, + 9.03463514696341, + 9.21859104395844, + 9.404466000967659, + 9.591410503955558, + 9.77754224895034, + 9.963701014989056, + 10.147378209978342, + 10.328440519981086, + 10.509775985963643, + 10.69398886198178, + 10.873444066965021, + 11.053394714021124, + 11.237941715982743, + 11.422520439955406, + 11.608547332929447, + 11.793529152986594, + 11.980611857026815, + 12.16343886195682, + 12.344160321983509, + 12.525829592021182, + 12.7058035149239, + 12.886234933976084, + 13.071656776010059, + 13.256394685944542, + 13.43872354796622, + 13.620628445991315, + 13.797536812955514, + 13.98124094796367, + 14.16867454093881, + 14.34899701597169, + 14.531357454019599, + 14.714249145006761, + 14.898292901925743, + 15.083189302007668, + 15.268771347007714, + 15.449914881028235, + 15.632736083935015, + 15.814866991015151, + 15.996179614914581, + 16.173511522007175, + 16.35267654596828, + 16.53842034493573, + 16.728175587952137, + 16.918890006025322, + 17.109517364995554, + 17.30199658300262, + 17.48722359596286, + 17.665493936976418, + 17.839922384009697, + 18.018445205991156, + 18.195734036969952, + 18.37159398093354, + 18.54864497296512, + 18.726537987939082, + 18.903177076019347, + 19.081090249004774, + 19.263479418004863, + 19.447094292961992, + 19.629881075932644, + 19.807149578933604, + 19.979883702937514, + 20.150496994960122, + 20.31987933802884, + 20.488778259954415, + 20.654695620993152, + 20.812118484987877, + 21.13122444099281 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 182.78678297065198, + "p95": 266.2163620116189, + "max": 330.92986897099763 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + } + ] +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/hostwatch-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/hostwatch-rank0.log new file mode 100644 index 0000000000..e250c840d6 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/hostwatch-rank0.log @@ -0,0 +1,387 @@ +02:20:38 114 +02:20:40 112 +02:20:43 112 +02:20:45 112 +02:20:47 111 +02:20:49 110 +02:20:51 109 +02:20:53 109 +02:20:55 109 +02:20:57 108 +02:20:59 107 +02:21:01 107 +02:21:03 105 +02:21:05 105 +02:21:07 105 +02:21:09 104 +02:21:11 104 +02:21:13 102 +02:21:15 102 +02:21:17 102 +02:21:19 101 +02:21:22 101 +02:21:24 99 +02:21:26 99 +02:21:28 98 +02:21:30 98 +02:21:32 98 +02:21:34 97 +02:21:36 97 +02:21:38 97 +02:21:40 95 +02:21:42 95 +02:21:44 94 +02:21:46 94 +02:21:48 94 +02:21:50 93 +02:21:52 93 +02:21:54 92 +02:21:56 91 +02:21:58 91 +02:22:01 90 +02:22:03 90 +02:22:05 90 +02:22:07 88 +02:22:09 88 +02:22:11 88 +02:22:13 87 +02:22:15 87 +02:22:17 85 +02:22:19 85 +02:22:21 84 +02:22:23 84 +02:22:25 84 +02:22:27 83 +02:22:29 83 +02:22:31 82 +02:22:33 81 +02:22:35 81 +02:22:38 81 +02:22:40 79 +02:22:42 79 +02:22:44 78 +02:22:46 78 +02:22:48 78 +02:22:50 77 +02:22:52 77 +02:22:54 75 +02:22:56 75 +02:22:58 75 +02:23:00 74 +02:23:02 74 +02:23:04 72 +02:23:06 72 +02:23:08 72 +02:23:10 71 +02:23:12 71 +02:23:15 71 +02:23:17 70 +02:23:19 70 +02:23:21 68 +02:23:23 68 +02:23:25 67 +02:23:27 67 +02:23:29 66 +02:23:31 65 +02:23:33 65 +02:23:35 65 +02:23:37 64 +02:23:39 64 +02:23:41 62 +02:23:43 62 +02:23:45 62 +02:23:47 61 +02:23:49 61 +02:23:52 60 +02:23:54 60 +02:23:56 59 +02:23:58 58 +02:24:00 58 +02:24:02 57 +02:24:04 57 +02:24:06 57 +02:24:08 55 +02:24:10 55 +02:24:12 55 +02:24:14 54 +02:24:16 54 +02:24:18 54 +02:24:20 50 +02:24:22 50 +02:24:24 50 +02:24:26 50 +02:24:28 50 +02:24:31 50 +02:24:33 36 +02:24:35 12 +02:24:37 12 +02:24:39 12 +02:24:41 12 +02:24:43 12 +02:24:45 12 +02:24:47 12 +02:24:49 12 +02:24:51 12 +02:24:53 12 +02:24:55 12 +02:24:57 12 +02:24:59 12 +02:25:01 12 +02:25:04 12 +02:25:06 12 +02:25:08 12 +02:25:10 12 +02:25:12 12 +02:25:14 12 +02:25:16 12 +02:25:18 12 +02:25:20 12 +02:25:22 12 +02:25:24 12 +02:25:26 12 +02:25:28 12 +02:25:30 12 +02:25:32 12 +02:25:35 12 +02:25:37 12 +02:25:39 12 +02:25:41 12 +02:25:43 12 +02:25:45 12 +02:25:47 12 +02:25:49 12 +02:25:51 12 +02:25:53 12 +02:25:55 12 +02:25:57 12 +02:25:59 12 +02:26:01 12 +02:26:03 12 +02:26:05 12 +02:26:08 12 +02:26:10 12 +02:26:12 12 +02:26:14 12 +02:26:16 12 +02:26:18 12 +02:26:20 12 +02:26:22 12 +02:26:24 12 +02:26:26 12 +02:26:28 12 +02:26:30 12 +02:26:32 12 +02:26:34 12 +02:26:36 12 +02:26:39 12 +02:26:41 12 +02:26:43 12 +02:26:45 12 +02:26:47 12 +02:26:49 12 +02:26:51 12 +02:26:53 12 +02:26:55 12 +02:26:57 12 +02:26:59 12 +02:27:01 12 +02:27:03 12 +02:27:05 12 +02:27:07 12 +02:27:10 12 +02:27:12 12 +02:27:14 12 +02:27:16 12 +02:27:18 12 +02:27:20 12 +02:27:22 12 +02:27:24 12 +02:27:26 12 +02:27:28 12 +02:27:30 12 +02:27:32 12 +02:27:34 12 +02:27:36 12 +02:27:39 12 +02:27:41 12 +02:27:43 12 +02:27:45 12 +02:27:47 12 +02:27:49 12 +02:27:51 12 +02:27:53 12 +02:27:55 12 +02:27:57 12 +02:27:59 12 +02:28:01 12 +02:28:03 12 +02:28:05 12 +02:28:07 12 +02:28:10 12 +02:28:12 12 +02:28:14 12 +02:28:16 12 +02:28:18 12 +02:28:20 12 +02:28:22 12 +02:28:24 12 +02:28:26 12 +02:28:28 12 +02:28:30 12 +02:28:32 12 +02:28:34 12 +02:28:36 12 +02:28:38 12 +02:28:41 12 +02:28:43 12 +02:28:45 12 +02:28:47 12 +02:28:49 12 +02:28:51 12 +02:28:53 12 +02:28:55 12 +02:28:57 12 +02:28:59 12 +02:29:01 12 +02:29:03 12 +02:29:06 12 +02:29:08 12 +02:29:10 12 +02:29:12 12 +02:29:14 12 +02:29:16 12 +02:29:18 12 +02:29:20 12 +02:29:22 12 +02:29:24 12 +02:29:26 12 +02:29:28 12 +02:29:30 12 +02:29:33 12 +02:29:35 12 +02:29:37 12 +02:29:39 12 +02:29:41 12 +02:29:43 12 +02:29:45 12 +02:29:47 12 +02:29:49 12 +02:29:51 12 +02:29:53 12 +02:29:55 12 +02:29:57 12 +02:29:59 12 +02:30:02 12 +02:30:04 12 +02:30:06 12 +02:30:08 12 +02:30:10 12 +02:30:12 12 +02:30:14 12 +02:30:16 12 +02:30:18 12 +02:30:20 12 +02:30:22 12 +02:30:24 12 +02:30:26 12 +02:30:28 12 +02:30:30 12 +02:30:33 12 +02:30:35 12 +02:30:37 12 +02:30:39 12 +02:30:41 12 +02:30:43 12 +02:30:45 12 +02:30:47 12 +02:30:49 12 +02:30:51 12 +02:30:53 12 +02:30:55 12 +02:30:57 12 +02:30:59 12 +02:31:01 12 +02:31:04 12 +02:31:06 12 +02:31:08 12 +02:31:10 12 +02:31:12 12 +02:31:14 12 +02:31:16 12 +02:31:18 12 +02:31:20 12 +02:31:22 12 +02:31:24 12 +02:31:26 12 +02:31:28 12 +02:31:30 12 +02:31:33 12 +02:31:35 12 +02:31:37 12 +02:31:39 12 +02:31:41 12 +02:31:43 12 +02:31:45 12 +02:31:47 12 +02:31:49 12 +02:31:51 12 +02:31:53 12 +02:31:55 12 +02:31:57 12 +02:31:59 12 +02:32:01 12 +02:32:04 12 +02:32:06 12 +02:32:08 12 +02:32:10 12 +02:32:12 12 +02:32:14 12 +02:32:16 12 +02:32:18 12 +02:32:20 12 +02:32:22 12 +02:32:24 12 +02:32:26 12 +02:32:28 12 +02:32:30 12 +02:32:33 12 +02:32:35 12 +02:32:37 12 +02:32:39 12 +02:32:41 12 +02:32:43 12 +02:32:45 12 +02:32:47 12 +02:32:49 12 +02:32:51 12 +02:32:53 12 +02:32:55 12 +02:32:57 12 +02:32:59 12 +02:33:01 12 +02:33:04 12 +02:33:06 12 +02:33:08 12 +02:33:10 12 +02:33:12 12 +02:33:14 12 +02:33:16 12 +02:33:18 12 +02:33:20 12 +02:33:22 12 +02:33:24 12 +02:33:26 12 +02:33:28 12 +02:33:30 12 +02:33:32 12 +02:33:35 12 +02:33:37 12 +02:33:39 12 +02:33:41 12 +02:33:43 12 +02:33:45 12 +02:33:47 12 +02:33:49 12 +02:33:51 12 +02:33:53 38 +02:33:55 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/hostwatch-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/hostwatch-rank1.log new file mode 100644 index 0000000000..99b050db7a --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/hostwatch-rank1.log @@ -0,0 +1,386 @@ +02:20:39 114 +02:20:41 112 +02:20:43 112 +02:20:45 110 +02:20:47 110 +02:20:49 110 +02:20:51 109 +02:20:53 109 +02:20:56 108 +02:20:58 107 +02:21:00 107 +02:21:02 105 +02:21:04 105 +02:21:06 105 +02:21:08 104 +02:21:10 104 +02:21:12 103 +02:21:14 103 +02:21:16 103 +02:21:18 101 +02:21:20 101 +02:21:22 101 +02:21:24 100 +02:21:26 100 +02:21:28 99 +02:21:30 99 +02:21:32 98 +02:21:35 97 +02:21:37 97 +02:21:39 96 +02:21:41 95 +02:21:43 95 +02:21:45 94 +02:21:47 94 +02:21:49 93 +02:21:51 93 +02:21:53 93 +02:21:55 91 +02:21:57 91 +02:21:59 91 +02:22:01 90 +02:22:03 90 +02:22:05 89 +02:22:07 89 +02:22:10 87 +02:22:12 87 +02:22:14 87 +02:22:16 85 +02:22:18 85 +02:22:20 85 +02:22:22 84 +02:22:24 84 +02:22:26 83 +02:22:28 83 +02:22:30 83 +02:22:32 81 +02:22:34 81 +02:22:36 80 +02:22:38 80 +02:22:40 80 +02:22:42 78 +02:22:44 78 +02:22:47 77 +02:22:49 77 +02:22:51 77 +02:22:53 75 +02:22:55 75 +02:22:57 76 +02:22:59 74 +02:23:01 74 +02:23:03 73 +02:23:05 73 +02:23:07 73 +02:23:09 71 +02:23:11 71 +02:23:13 71 +02:23:15 70 +02:23:17 70 +02:23:19 70 +02:23:21 68 +02:23:24 69 +02:23:26 67 +02:23:28 67 +02:23:30 67 +02:23:32 66 +02:23:34 65 +02:23:36 65 +02:23:38 64 +02:23:40 64 +02:23:42 64 +02:23:44 63 +02:23:46 63 +02:23:48 61 +02:23:50 61 +02:23:52 60 +02:23:54 60 +02:23:56 60 +02:23:58 58 +02:24:01 58 +02:24:03 58 +02:24:05 57 +02:24:07 57 +02:24:09 57 +02:24:11 56 +02:24:13 56 +02:24:15 54 +02:24:17 54 +02:24:19 53 +02:24:21 52 +02:24:23 50 +02:24:25 50 +02:24:27 50 +02:24:29 50 +02:24:31 50 +02:24:33 48 +02:24:35 11 +02:24:37 11 +02:24:40 11 +02:24:42 12 +02:24:44 12 +02:24:46 12 +02:24:48 12 +02:24:50 12 +02:24:52 12 +02:24:54 12 +02:24:56 12 +02:24:58 12 +02:25:00 12 +02:25:02 12 +02:25:04 12 +02:25:06 12 +02:25:09 12 +02:25:11 12 +02:25:13 12 +02:25:15 12 +02:25:17 12 +02:25:19 12 +02:25:21 12 +02:25:23 12 +02:25:25 12 +02:25:27 12 +02:25:29 12 +02:25:31 12 +02:25:33 12 +02:25:35 12 +02:25:37 12 +02:25:40 12 +02:25:42 12 +02:25:44 12 +02:25:46 12 +02:25:48 12 +02:25:50 12 +02:25:52 12 +02:25:54 12 +02:25:56 12 +02:25:58 12 +02:26:00 12 +02:26:02 12 +02:26:04 12 +02:26:06 12 +02:26:08 12 +02:26:11 12 +02:26:13 12 +02:26:15 12 +02:26:17 12 +02:26:19 12 +02:26:21 12 +02:26:23 12 +02:26:25 12 +02:26:27 12 +02:26:29 12 +02:26:31 12 +02:26:33 12 +02:26:35 12 +02:26:37 12 +02:26:39 12 +02:26:41 12 +02:26:44 12 +02:26:46 12 +02:26:48 12 +02:26:50 12 +02:26:52 12 +02:26:54 12 +02:26:56 12 +02:26:58 12 +02:27:00 12 +02:27:02 12 +02:27:04 12 +02:27:06 12 +02:27:08 12 +02:27:10 12 +02:27:12 12 +02:27:15 12 +02:27:17 12 +02:27:19 12 +02:27:21 12 +02:27:23 12 +02:27:25 12 +02:27:27 12 +02:27:29 12 +02:27:31 12 +02:27:33 12 +02:27:35 12 +02:27:37 12 +02:27:39 12 +02:27:41 12 +02:27:43 12 +02:27:46 12 +02:27:48 12 +02:27:50 12 +02:27:52 12 +02:27:54 12 +02:27:56 12 +02:27:58 12 +02:28:00 12 +02:28:02 12 +02:28:04 12 +02:28:06 12 +02:28:08 12 +02:28:10 12 +02:28:13 12 +02:28:15 12 +02:28:17 12 +02:28:19 12 +02:28:21 12 +02:28:23 12 +02:28:25 12 +02:28:27 12 +02:28:29 12 +02:28:31 12 +02:28:33 12 +02:28:35 12 +02:28:37 12 +02:28:39 12 +02:28:42 12 +02:28:44 12 +02:28:46 12 +02:28:48 12 +02:28:50 12 +02:28:52 12 +02:28:54 12 +02:28:56 12 +02:28:58 12 +02:29:00 12 +02:29:03 12 +02:29:05 12 +02:29:07 12 +02:29:09 12 +02:29:11 12 +02:29:13 12 +02:29:15 12 +02:29:17 12 +02:29:19 12 +02:29:21 12 +02:29:23 12 +02:29:25 12 +02:29:27 12 +02:29:29 12 +02:29:31 12 +02:29:33 12 +02:29:36 12 +02:29:38 12 +02:29:40 12 +02:29:42 12 +02:29:44 12 +02:29:46 12 +02:29:48 12 +02:29:50 12 +02:29:52 12 +02:29:54 12 +02:29:56 12 +02:29:58 12 +02:30:00 12 +02:30:02 12 +02:30:04 12 +02:30:07 12 +02:30:09 12 +02:30:11 12 +02:30:13 12 +02:30:15 12 +02:30:17 12 +02:30:19 12 +02:30:21 12 +02:30:23 12 +02:30:25 12 +02:30:27 12 +02:30:29 12 +02:30:31 12 +02:30:33 12 +02:30:35 12 +02:30:38 12 +02:30:40 12 +02:30:42 12 +02:30:44 12 +02:30:46 12 +02:30:48 12 +02:30:50 12 +02:30:52 12 +02:30:54 12 +02:30:56 12 +02:30:58 12 +02:31:00 12 +02:31:02 12 +02:31:04 12 +02:31:07 12 +02:31:09 12 +02:31:11 12 +02:31:13 12 +02:31:15 12 +02:31:17 12 +02:31:19 12 +02:31:21 12 +02:31:23 12 +02:31:25 12 +02:31:27 12 +02:31:29 12 +02:31:31 12 +02:31:33 12 +02:31:35 12 +02:31:38 12 +02:31:40 12 +02:31:42 12 +02:31:44 12 +02:31:46 12 +02:31:48 12 +02:31:50 12 +02:31:52 12 +02:31:54 12 +02:31:56 12 +02:31:58 12 +02:32:00 12 +02:32:02 12 +02:32:04 12 +02:32:07 12 +02:32:09 12 +02:32:11 12 +02:32:13 12 +02:32:15 12 +02:32:17 12 +02:32:19 12 +02:32:21 12 +02:32:23 12 +02:32:25 12 +02:32:27 12 +02:32:29 12 +02:32:31 12 +02:32:33 12 +02:32:35 12 +02:32:37 12 +02:32:40 12 +02:32:42 12 +02:32:44 12 +02:32:46 12 +02:32:48 12 +02:32:50 12 +02:32:52 12 +02:32:54 12 +02:32:56 12 +02:32:58 12 +02:33:00 12 +02:33:02 12 +02:33:04 12 +02:33:06 12 +02:33:09 12 +02:33:11 12 +02:33:13 12 +02:33:15 12 +02:33:17 12 +02:33:19 12 +02:33:21 12 +02:33:23 12 +02:33:25 12 +02:33:27 12 +02:33:29 12 +02:33:31 12 +02:33:33 12 +02:33:35 12 +02:33:37 12 +02:33:39 12 +02:33:42 12 +02:33:44 12 +02:33:46 12 +02:33:48 12 +02:33:50 12 +02:33:52 12 +02:33:54 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/hostwatch-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/hostwatch-rank2.log new file mode 100644 index 0000000000..c46285e4f6 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/hostwatch-rank2.log @@ -0,0 +1,386 @@ +02:20:40 113 +02:20:42 112 +02:20:44 112 +02:20:46 110 +02:20:48 110 +02:20:50 109 +02:20:52 109 +02:20:54 109 +02:20:56 108 +02:20:58 107 +02:21:01 106 +02:21:03 107 +02:21:05 105 +02:21:07 105 +02:21:09 105 +02:21:11 103 +02:21:13 103 +02:21:15 102 +02:21:17 102 +02:21:19 101 +02:21:21 101 +02:21:23 100 +02:21:25 100 +02:21:27 100 +02:21:29 98 +02:21:31 98 +02:21:33 97 +02:21:35 97 +02:21:37 97 +02:21:40 96 +02:21:42 96 +02:21:44 95 +02:21:46 94 +02:21:48 94 +02:21:50 93 +02:21:52 93 +02:21:54 92 +02:21:56 92 +02:21:58 91 +02:22:00 90 +02:22:02 89 +02:22:04 89 +02:22:06 89 +02:22:08 88 +02:22:10 88 +02:22:12 87 +02:22:14 86 +02:22:17 86 +02:22:19 85 +02:22:21 85 +02:22:23 84 +02:22:25 84 +02:22:27 84 +02:22:29 83 +02:22:31 83 +02:22:33 81 +02:22:35 81 +02:22:37 80 +02:22:39 80 +02:22:41 79 +02:22:43 79 +02:22:45 79 +02:22:47 77 +02:22:49 77 +02:22:51 76 +02:22:53 76 +02:22:56 75 +02:22:58 75 +02:23:00 75 +02:23:02 74 +02:23:04 73 +02:23:06 73 +02:23:08 72 +02:23:10 72 +02:23:12 71 +02:23:14 71 +02:23:16 70 +02:23:18 70 +02:23:20 70 +02:23:22 69 +02:23:24 68 +02:23:26 67 +02:23:28 67 +02:23:30 67 +02:23:33 66 +02:23:35 66 +02:23:37 66 +02:23:39 64 +02:23:41 64 +02:23:43 63 +02:23:45 63 +02:23:47 62 +02:23:49 62 +02:23:51 61 +02:23:53 60 +02:23:55 60 +02:23:57 59 +02:23:59 59 +02:24:01 59 +02:24:03 57 +02:24:05 55 +02:24:07 55 +02:24:10 55 +02:24:12 55 +02:24:14 55 +02:24:16 54 +02:24:18 42 +02:24:20 42 +02:24:22 42 +02:24:24 42 +02:24:26 42 +02:24:28 42 +02:24:30 42 +02:24:32 42 +02:24:34 34 +02:24:36 17 +02:24:38 17 +02:24:40 17 +02:24:42 17 +02:24:44 17 +02:24:47 17 +02:24:49 17 +02:24:51 17 +02:24:53 17 +02:24:55 17 +02:24:57 17 +02:24:59 17 +02:25:01 17 +02:25:03 17 +02:25:05 17 +02:25:07 17 +02:25:09 17 +02:25:11 17 +02:25:13 17 +02:25:15 17 +02:25:18 17 +02:25:20 17 +02:25:22 17 +02:25:24 17 +02:25:26 17 +02:25:28 17 +02:25:30 17 +02:25:32 17 +02:25:34 17 +02:25:36 17 +02:25:38 17 +02:25:40 17 +02:25:42 17 +02:25:44 17 +02:25:46 17 +02:25:49 17 +02:25:51 17 +02:25:53 17 +02:25:55 17 +02:25:57 17 +02:25:59 17 +02:26:01 17 +02:26:03 17 +02:26:05 17 +02:26:07 17 +02:26:09 17 +02:26:11 17 +02:26:13 17 +02:26:15 17 +02:26:17 17 +02:26:20 17 +02:26:22 17 +02:26:24 17 +02:26:26 17 +02:26:28 17 +02:26:30 17 +02:26:32 17 +02:26:34 17 +02:26:36 17 +02:26:38 17 +02:26:40 17 +02:26:42 17 +02:26:44 17 +02:26:46 17 +02:26:49 17 +02:26:51 17 +02:26:53 17 +02:26:55 17 +02:26:57 17 +02:26:59 17 +02:27:01 17 +02:27:03 17 +02:27:05 17 +02:27:07 17 +02:27:09 17 +02:27:11 17 +02:27:13 17 +02:27:15 17 +02:27:17 17 +02:27:20 17 +02:27:22 17 +02:27:24 17 +02:27:26 17 +02:27:28 17 +02:27:30 17 +02:27:32 17 +02:27:34 17 +02:27:36 17 +02:27:38 17 +02:27:40 17 +02:27:42 17 +02:27:44 17 +02:27:46 17 +02:27:49 17 +02:27:51 17 +02:27:53 17 +02:27:55 17 +02:27:57 17 +02:27:59 17 +02:28:01 17 +02:28:03 17 +02:28:05 17 +02:28:07 17 +02:28:09 17 +02:28:11 17 +02:28:13 17 +02:28:15 17 +02:28:17 17 +02:28:20 17 +02:28:22 17 +02:28:24 17 +02:28:26 17 +02:28:28 17 +02:28:30 17 +02:28:32 17 +02:28:34 17 +02:28:36 17 +02:28:38 17 +02:28:40 17 +02:28:42 17 +02:28:44 17 +02:28:46 17 +02:28:48 17 +02:28:51 17 +02:28:53 17 +02:28:55 17 +02:28:57 17 +02:28:59 17 +02:29:01 17 +02:29:03 17 +02:29:05 17 +02:29:07 17 +02:29:09 17 +02:29:11 17 +02:29:13 17 +02:29:15 17 +02:29:17 17 +02:29:19 17 +02:29:22 17 +02:29:24 17 +02:29:26 17 +02:29:28 17 +02:29:30 17 +02:29:32 17 +02:29:34 17 +02:29:36 17 +02:29:38 17 +02:29:40 17 +02:29:42 17 +02:29:44 17 +02:29:46 17 +02:29:48 17 +02:29:50 17 +02:29:53 17 +02:29:55 17 +02:29:57 17 +02:29:59 17 +02:30:01 17 +02:30:03 17 +02:30:05 17 +02:30:07 17 +02:30:09 17 +02:30:11 17 +02:30:13 17 +02:30:15 17 +02:30:17 17 +02:30:19 17 +02:30:21 17 +02:30:24 17 +02:30:26 17 +02:30:28 17 +02:30:30 17 +02:30:32 17 +02:30:34 17 +02:30:36 17 +02:30:38 17 +02:30:40 17 +02:30:42 17 +02:30:44 17 +02:30:46 17 +02:30:48 17 +02:30:50 17 +02:30:53 17 +02:30:55 17 +02:30:57 17 +02:30:59 17 +02:31:01 17 +02:31:03 17 +02:31:05 17 +02:31:07 17 +02:31:09 17 +02:31:11 17 +02:31:13 17 +02:31:15 17 +02:31:17 17 +02:31:19 17 +02:31:21 17 +02:31:24 17 +02:31:26 17 +02:31:28 17 +02:31:30 17 +02:31:32 17 +02:31:34 17 +02:31:36 17 +02:31:38 17 +02:31:40 17 +02:31:42 17 +02:31:44 17 +02:31:46 17 +02:31:48 17 +02:31:50 17 +02:31:53 17 +02:31:55 17 +02:31:57 17 +02:31:59 17 +02:32:01 17 +02:32:03 17 +02:32:05 17 +02:32:07 17 +02:32:09 17 +02:32:11 17 +02:32:13 17 +02:32:15 17 +02:32:17 17 +02:32:19 17 +02:32:21 17 +02:32:24 17 +02:32:26 17 +02:32:28 17 +02:32:30 17 +02:32:32 17 +02:32:34 17 +02:32:36 17 +02:32:38 17 +02:32:40 17 +02:32:42 17 +02:32:44 17 +02:32:46 17 +02:32:48 17 +02:32:50 17 +02:32:52 17 +02:32:55 17 +02:32:57 17 +02:32:59 17 +02:33:01 17 +02:33:03 17 +02:33:05 17 +02:33:07 17 +02:33:09 17 +02:33:11 17 +02:33:13 17 +02:33:15 17 +02:33:17 17 +02:33:19 17 +02:33:21 17 +02:33:24 17 +02:33:26 17 +02:33:28 17 +02:33:30 17 +02:33:32 17 +02:33:34 17 +02:33:36 17 +02:33:38 17 +02:33:40 17 +02:33:42 17 +02:33:44 17 +02:33:46 17 +02:33:48 17 +02:33:50 17 +02:33:52 17 +02:33:54 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/server-rank0.log new file mode 100644 index 0000000000..bc53db1818 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/server-rank0.log @@ -0,0 +1,27335 @@ +[kvp-serve] 2026-10-22T17:20:24+00:00 rank 0 of 3 context 1048576 parallel 8 binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=all TF_GLM_MM=unset args +info: rank 0: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-a:4512:4512 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:4512:4512 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:4512:4512 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:4512:4512 [0] NCCL INFO cudaDriverVersion 13030 +host-a:4512:4512 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:4512:4512 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:4512:4512 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:4512:4512 [0] NCCL INFO Count not open libibverbs.so +host-a:4512:4512 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:4512:4512 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:4512:4512 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:4512:4512 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0xe73d7b0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:4512:4512 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0xe77eeb0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:4512:4512 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0xe7c05b0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:4512:4512 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0xe801cb0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:4512:4512 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4512:4512 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4512:4512 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4512:4512 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4512:4512 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:4512:4512 [0] NCCL INFO Initialized NET plugin IB +host-a:4512:4512 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:4512:4512 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:4512:4512 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:4512:4512 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:4512:4512 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:4512:4512 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:4512:4512 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:4512:4512 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:4512:4512 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:4512:4512 [0] NCCL INFO Using network IB +host-a:4512:4512 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:4512:4512 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0xbb060f0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x8fa3f4c721825aca - Init START +host-a:4512:4512 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:4512:4512 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:4512:4512 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:4512:4512 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:4512:4512 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:4512:4512 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:4512:4512 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:4512:4512 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:4512:4512 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:4512:4512 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:4512:4512 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:4512:4512 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:4512:4512 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:4512:4512 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:4512:4512 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:4512:4512 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:4512:4512 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:4512:4512 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:4512:4512 [0] NCCL INFO Rank 0: 6 Net devices +host-a:4512:4512 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:4512:4512 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:4512:4512 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:4512:4512 [0] NCCL INFO comm 0xbb060f0 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:4512:4512 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:4512:4512 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:4512:4512 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:4512:4512 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:4512:4512 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:4512:4512 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:4512:4512 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:4512:4512 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:4512:4512 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:4512:4512 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:4512:4529 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-a:4512:4529 [0] NCCL INFO proxy listening socket at <59137> +host-a:4512:4530 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 4 +host-a:4512:4531 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-a:4512:4512 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:4512:4512 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:4512:4512 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:4512:4512 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:4512:4512 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:4512:4512 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:4512:4512 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:4512:4512 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589c010000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589c054000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589c09c000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6807 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6808 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6807 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6807 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6808 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6808 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589c145000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589c18d000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589c1d1000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589c219000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589c25d000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1779 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1780 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1781 pkey=0 pd=0xf7589c0f2d40 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6807 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6808 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6839 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6840 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6839 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6839 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6840 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6840 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1907 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1908 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1909 pkey=0 pd=0xf7589c317c10 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6809 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6810 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6809 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6809 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6810 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6810 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6841 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6842 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6841 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ccbca ctsFifoLkey=0x1ccbca +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6841 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6842 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ccbca ctsFifoLkey=0x1ccbca +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6842 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1782 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1783 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1784 pkey=0 pd=0xf7589c0f2d40 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1910 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1911 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1912 pkey=0 pd=0xf7589c317c10 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6839 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6840 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6809 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6810 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6841 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6842 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4512 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:4512:4512 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:4512:4512 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:4512:4512 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:4512:4512 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589c5bb000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589c5fc000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589c63d000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589c67e000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1785 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1786 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1785 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1785 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1786 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1786 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1913 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1914 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1913 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19e2da ctsFifoLkey=0x19e2da +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1913 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1914 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19e2da ctsFifoLkey=0x19e2da +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1914 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1785 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1786 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1913 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1914 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1787 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1788 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1787 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1787 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1788 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1788 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1787 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1788 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1915 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1916 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1915 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b7b4 ctsFifoLkey=0x19b7b4 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1915 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1916 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b7b4 ctsFifoLkey=0x19b7b4 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1916 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1915 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1916 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4512 [0] NCCL INFO Connected all trees +host-a:4512:4512 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:4512:4512 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:4512:4512 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:4512:4512 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589c81f000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589c864000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589c8ac000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589c8f4000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6811 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6812 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6813 pkey=0 pd=0xf7589c053590 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6843 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6844 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6845 pkey=0 pd=0xf7589c2f1d60 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6814 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6815 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6816 pkey=0 pd=0xf7589c053590 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6846 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6847 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6848 pkey=0 pd=0xf7589c2f1d60 +host-a:4512:4512 [0] NCCL INFO Connected binomial trees +host-a:4512:4512 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:4512:4512 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:4512:4512 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:4512:4512 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:4512:4512 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-a:4512:4512 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:4512:4512 [0] NCCL INFO ncclCommInitRank comm 0xbb060f0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x8fa3f4c721825aca - Init COMPLETE +host-a:4512:4512 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 8.64 (kernels 0.24, alloc 0.03, bootstrap 8.06, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.29, rest 0.00) +host-a:4512:4512 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:4512:4512 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:4512:4512 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:4512:4512 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:4512:4512 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:4512:4512 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:4512:4512 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:4512:4512 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589cb16000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589cb57000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589cb9f000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589cbe3000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589cc2b000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6817 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6818 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6817 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1be6e3 ctsFifoLkey=0x1be6e3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6817 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6818 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1be6e3 ctsFifoLkey=0x1be6e3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6818 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1789 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1790 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1791 pkey=0 pd=0xf7589c0f2d40 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6849 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6850 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6849 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c2525 ctsFifoLkey=0x1c2525 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6849 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6850 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c2525 ctsFifoLkey=0x1c2525 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6850 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1917 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1918 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1917 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c0bd ctsFifoLkey=0x19c0bd +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1917 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1918 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c0bd ctsFifoLkey=0x19c0bd +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1918 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589cdfa000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6817 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6818 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6849 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6850 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1919 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1920 pkey=0 pd=0xf7589c317c10 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1921 pkey=0 pd=0xf7589c317c10 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6851 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6852 pkey=0 pd=0xf7589c2f1d60 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6853 pkey=0 pd=0xf7589c2f1d60 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf7589cf30000) +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf7589cf74000) +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1917 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1918 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1792 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1793 pkey=0 pd=0xf7589c0f2d40 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1792 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x1800bd ctsFifoLkey=0x1800bd +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1792 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1793 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x1800bd ctsFifoLkey=0x1800bd +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1793 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6819 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6820 pkey=0 pd=0xf7589c053590 oooRq=0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6821 pkey=0 pd=0xf7589c053590 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1792 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4512:4529 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1793 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, coinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 78.03 GiB; 8 streams at once, the engine's own at a 1048576-token window (12.88 GiB, in the model's), 7 more taking 1237 pages of 512 rows (7.70 GiB) as they need them, 131 MiB fixed each, of 24.6 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 16.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 1048576; loaded in 250.4s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-787db69fbfb13040d62d67873ea98e4c prompt=14 cached=0 thinking=False effort=none tokens=16 sha=e195338a9a9d finish=length rounds=6 accepted=9/13 ttft=0.110 decode=0.352 elapsed=0.462 queued=0.000 tok/s=42.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-4c98f657da95065765b43a15a00e1326 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.110 decode=5.233 elapsed=5.343 queued=0.000 tok/s=48.7 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-7c227558a0368e77575fcd37b40c7316 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=108 accepted=147/202 ttft=0.108 decode=6.164 elapsed=6.272 queued=0.000 tok/s=41.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 4, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-ccecca269a4d068e1bd1fb95d9cf3979 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=106 accepted=150/212 ttft=0.109 decode=6.224 elapsed=6.333 queued=0.000 tok/s=41.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 5, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-fc865227735bb099bff010aeea1c7c4b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=105 accepted=150/218 ttft=0.109 decode=6.219 elapsed=6.328 queued=0.000 tok/s=41.0 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-b06fc14dac0d3f669fd568d26832559f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=77 accepted=179/227 ttft=0.110 decode=4.942 elapsed=5.052 queued=0.000 tok/s=51.6 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-f0d85cfed817cebeb384682ef6d440a6 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=104 accepted=151/218 ttft=0.109 decode=6.088 elapsed=6.197 queued=0.000 tok/s=41.9 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 8, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-9cc4794a4d35a74ff33e6242d5c79369 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=106 accepted=149/206 ttft=0.109 decode=6.294 elapsed=6.402 queued=0.000 tok/s=40.5 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 9, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-112670f91b5d8c54bbb7df6c0de02e4a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=119 accepted=136/196 ttft=0.109 decode=6.734 elapsed=6.843 queued=0.000 tok/s=37.9 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 10, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-f8d04c9b858bdbccd7220675c24aa714 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.300 elapsed=11.403 queued=0.000 tok/s=22.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 11, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-692839d08bc7f147a41ac5fbb35a95c1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.220 elapsed=11.323 queued=0.000 tok/s=22.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-591b924a36f596d8d23ad41cda61d1fb prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=255 accepted=0/0 ttft=0.101 decode=11.171 elapsed=11.272 queued=0.000 tok/s=22.8 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 13, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-79f24d33e73da010724becbd1da500e3 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.232 elapsed=11.335 queued=0.000 tok/s=22.7 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 14, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-31f34c712b8d61992ecc12708d720b64 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.317 elapsed=11.419 queued=0.000 tok/s=22.5 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 15, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-249b57af7373fc9aa7f702cd6b8a892a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=255 accepted=0/0 ttft=0.104 decode=11.329 elapsed=11.433 queued=0.000 tok/s=22.5 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-36ea766a1f02d18f411b3279b4b859c9 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.303 elapsed=11.406 queued=0.000 tok/s=22.6 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 17, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-2510fa4ab698a6602ffac66dc31074f8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=255 accepted=0/0 ttft=0.104 decode=11.279 elapsed=11.383 queued=0.000 tok/s=22.6 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 18, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-173ea20a122102eb02f174720072d14f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.109 decode=5.271 elapsed=5.380 queued=0.000 tok/s=48.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 19, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-5d82e53830b8a0072b4b096f41298d95 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.274 decode=7.358 elapsed=7.632 queued=0.167 tok/s=34.7 seed=1234 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-e361b340381d7d171b6899fb438ee6c7 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=96 accepted=159/273 ttft=0.110 decode=8.610 elapsed=8.720 queued=0.000 tok/s=29.6 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 21, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-0fa7290a5933699e0911ae932b1d9404 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.467 decode=11.039 elapsed=11.506 queued=0.167 tok/s=23.1 seed=1234 +[tensorfold] done cmpl-f98d9f2f8a2124797ac8a7bba74bed63 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=89 accepted=167/263 ttft=0.110 decode=12.575 elapsed=12.685 queued=0.000 tok/s=20.3 seed=1237 +[tensorfold] done cmpl-4ed31c88d0a9e1fd711c84e80e910ca2 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=89 accepted=166/264 ttft=0.276 decode=12.489 elapsed=12.765 queued=0.167 tok/s=20.4 seed=1236 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-1d083245ee815b7f8e4f58b36d0763bc prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=96 accepted=160/285 ttft=0.684 decode=12.645 elapsed=13.329 queued=0.167 tok/s=20.2 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 26, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-5daf4412ccf88e050ca285ccb16c386f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=1.159 decode=16.021 elapsed=17.180 queued=0.162 tok/s=15.9 seed=1234 +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 33, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-47191babebe3b1d24649bac769e18dcd prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=79 accepted=176/236 ttft=1.426 decode=16.485 elapsed=17.910 queued=0.162 tok/s=15.5 seed=1238 +[tensorfold] done cmpl-63e2236d9c071fe9e8879aadfd2297f5 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=89 accepted=166/264 ttft=0.457 decode=18.574 elapsed=19.031 queued=0.164 tok/s=13.7 seed=1236 +[tensorfold] done cmpl-e93c5d0dd4bc10be0e4fa10d7c696145 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=89 accepted=167/264 ttft=0.674 decode=18.511 elapsed=19.184 queued=0.163 tok/s=13.8 seed=1237 +[tensorfold] done cmpl-ba2afd352bdfa4de815a80360f744d97 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=95 accepted=160/282 ttft=0.108 decode=19.488 elapsed=19.597 queued=0.000 tok/s=13.1 seed=1235 +[tensorfold] done cmpl-6c9bb2af0d2c1b37d67e11caf713eb8f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=91 accepted=164/269 ttft=0.909 decode=18.687 elapsed=19.596 queued=0.163 tok/s=13.6 seed=1240 +[tensorfold] done cmpl-cf386c2f8905989942a998544aec0c6d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=102 accepted=153/304 ttft=0.271 decode=20.074 elapsed=20.345 queued=0.000 tok/s=12.7 seed=1241 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-5a28c0be56c9ada810e2cb277c5e0dbd prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=99 accepted=156/225 ttft=17.289 decode=7.659 elapsed=24.948 queued=0.162 tok/s=33.3 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 34, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-dcb0aa1d817d5945def87fc6f8c341fa prompt=32 cached=0 thinking=False effort=none tokens=16 sha=3f46b826a241 finish=length rounds=10 accepted=5/13 ttft=0.159 decode=0.541 elapsed=0.700 queued=0.000 tok/s=27.7 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 35, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-7b0027520b47ca35bc0f8efd501eac45 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=135 accepted=120/176 ttft=0.162 decode=7.436 elapsed=7.597 queued=0.000 tok/s=34.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 36, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-8f1b07a31d04c68368b654992f8cd273 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=106 accepted=150/234 ttft=0.160 decode=6.541 elapsed=6.701 queued=0.000 tok/s=39.0 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 37, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-2f0234153a66c39441e7db5c94cfb15c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=133 accepted=123/191 ttft=0.159 decode=7.292 elapsed=7.451 queued=0.000 tok/s=35.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 38, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-46c280673b491f50e80d9bb4b3b698cc prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=141 accepted=115/175 ttft=0.159 decode=7.707 elapsed=7.866 queued=0.000 tok/s=33.1 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 39, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-9f4d4ed243ea887e545bbc94a99570da prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=135 accepted=120/197 ttft=0.161 decode=7.594 elapsed=7.755 queued=0.000 tok/s=33.6 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 40, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-ec7bbbca09fc49278d7f3c22fe2d4330 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=128 accepted=128/208 ttft=0.159 decode=7.341 elapsed=7.500 queued=0.000 tok/s=34.7 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 41, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-096c282a6a9488bc3edee43b005c7357 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=124 accepted=131/205 ttft=0.181 decode=7.251 elapsed=7.432 queued=0.000 tok/s=35.2 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-69a9de75c35ca83bc59b772c56d57822 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=123 accepted=132/204 ttft=0.161 decode=7.069 elapsed=7.230 queued=0.000 tok/s=36.1 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 43, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-05842dc2ee97886280c783638d3fa0b6 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.610 elapsed=11.762 queued=0.000 tok/s=22.0 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 44, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-5e8d2acbf980095eced0db6525bd0e4a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=255 accepted=0/0 ttft=0.154 decode=11.473 elapsed=11.627 queued=0.000 tok/s=22.2 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 45, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-f27614e9a703e72eb484132127c9de63 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.448 elapsed=11.600 queued=0.000 tok/s=22.3 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 46, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-4868448e9ebf8f896f00783af00cf245 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.493 elapsed=11.645 queued=0.000 tok/s=22.2 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 47, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-8250f4232afc768f24846ed6baf61963 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.391 elapsed=11.545 queued=0.000 tok/s=22.4 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-21c8d0195b1d4862f275482a63d955f3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.157 elapsed=11.309 queued=0.000 tok/s=22.9 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 49, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-1c76fbb0ded8e1b6367f22026fe9be2f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.256 elapsed=11.408 queued=0.000 tok/s=22.7 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 50, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-a53c89f5de5badd71d0c34184b08cb5f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.266 elapsed=11.419 queued=0.000 tok/s=22.6 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 51, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-d70ba7988b356b4607c4adb55e547dcf prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=135 accepted=120/176 ttft=0.160 decode=7.268 elapsed=7.428 queued=0.000 tok/s=35.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 53, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 53, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-988fd818588c5812f81a3a8642635734 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=93 accepted=162/278 ttft=0.378 decode=8.836 elapsed=9.214 queued=0.000 tok/s=28.9 seed=1235 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-77d7c086b17dbe4a95f2c3d746329e34 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=104 accepted=151/292 ttft=0.160 decode=9.577 elapsed=9.737 queued=0.000 tok/s=26.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 54, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 57, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 57, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 57, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-6802fdf7664924f26a611af0c038ae08 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=94 accepted=161/280 ttft=0.161 decode=13.463 elapsed=13.625 queued=0.000 tok/s=18.9 seed=1235 +[tensorfold] done chatcmpl-d4e09caab53de3c10c83a45304aa4b8f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=100 accepted=156/297 ttft=0.624 decode=13.955 elapsed=14.579 queued=0.222 tok/s=18.3 seed=1237 +[tensorfold] done chatcmpl-c0bbe5fbd9f3f3799e5591a47b4759b5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=102 accepted=154/303 ttft=0.888 decode=13.946 elapsed=14.835 queued=0.222 tok/s=18.3 seed=1234 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-d64711c64a9550af131526a22ffad51c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=105 accepted=151/311 ttft=0.380 decode=14.505 elapsed=14.885 queued=0.222 tok/s=17.6 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 58, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 65, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 65, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 65, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 65, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 65, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 65, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-8b9b5468e05df32d951c7e54aaabe64d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=100 accepted=156/297 ttft=0.617 decode=20.455 elapsed=21.072 queued=0.218 tok/s=12.5 seed=1237 +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 66, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-fd5bbfff41c3a49c61304123ebdcfb0d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=101 accepted=154/298 ttft=0.887 decode=20.732 elapsed=21.619 queued=0.218 tok/s=12.3 seed=1241 +[tensorfold] done chatcmpl-24a2aefe8f432ca0533fef7fee5c08a7 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=106 accepted=149/314 ttft=0.161 decode=21.807 elapsed=21.968 queued=0.000 tok/s=11.7 seed=1240 +[tensorfold] done chatcmpl-88f931a5d6954aa973997461cad52909 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=103 accepted=153/304 ttft=1.170 decode=20.936 elapsed=22.106 queued=0.217 tok/s=12.2 seed=1239 +[tensorfold] done chatcmpl-6c9e0e6f42079478d3183e53bf9a7393 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=102 accepted=154/303 ttft=1.479 decode=20.627 elapsed=22.106 queued=0.217 tok/s=12.4 seed=1234 +[tensorfold] done chatcmpl-ee051ca0aa1d2a097a4fefb77f417e30 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=105 accepted=151/311 ttft=1.797 decode=20.755 elapsed=22.552 queued=0.217 tok/s=12.3 seed=1236 +[tensorfold] done chatcmpl-698b3128dc4124cc9a3ba11871cb86d7 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=112 accepted=143/332 ttft=0.378 decode=22.345 elapsed=22.723 queued=0.218 tok/s=11.4 seed=1238 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-3872c876e0507492f103384f921dcf95 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=109 accepted=147/220 ttft=21.227 decode=7.197 elapsed=28.424 queued=0.217 tok/s=35.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 67, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-ff825dffcb08448e41b563c4ea59bf7a prompt=14 cached=0 thinking=False effort=none tokens=16 sha=3000ccd48624 finish=length rounds=5 accepted=10/11 ttft=0.109 decode=0.300 elapsed=0.409 queued=0.000 tok/s=50.0 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 68, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-12cf7e338faf783f0fbc35b99c742795 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=51 accepted=204/222 ttft=0.108 decode=3.621 elapsed=3.729 queued=0.000 tok/s=70.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 69, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-6f84e83d5d80ca0ad0aa2739505dee57 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=255 accepted=0/0 ttft=0.101 decode=11.294 elapsed=11.395 queued=0.000 tok/s=22.6 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 70, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-399a9ccfb4cee348fc4d24d845b1bff0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=51 accepted=204/222 ttft=0.109 decode=3.610 elapsed=3.719 queued=0.000 tok/s=70.6 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 71, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 72, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-6387b32302411e28b1efc8a65514bf84 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.108 decode=6.182 elapsed=6.290 queued=0.000 tok/s=41.2 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-1fdc2a7257beba75f63635ee022ac8ba prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.272 decode=6.074 elapsed=6.346 queued=0.166 tok/s=42.0 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 73, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 76, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 76, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 76, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-ce4438bff73162b348fb30894b1a2a2e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.109 decode=8.598 elapsed=8.706 queued=0.000 tok/s=29.7 +[tensorfold] done cmpl-c457e761e5ac469474fbc9b87d2a702f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.274 decode=8.533 elapsed=8.806 queued=0.168 tok/s=29.9 +[tensorfold] done cmpl-205fab088fd3c41db9b2c4be43a38812 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.461 decode=8.429 elapsed=8.890 queued=0.168 tok/s=30.3 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-98c254610f9e6b80d1374334ff764d2a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.673 decode=8.274 elapsed=8.946 queued=0.167 tok/s=30.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 78, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 84, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 84, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 84, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 84, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 84, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 84, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-ce4e62aa81b273cbff27e6f3fa027d2e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.110 decode=11.792 elapsed=11.902 queued=0.000 tok/s=21.6 +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 85, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-286f6659dabe14d70488af483eed70ee prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.279 decode=11.893 elapsed=12.172 queued=0.000 tok/s=21.4 +[tensorfold] done cmpl-a95b29d978cdff5faf36cb9c4cd6497b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.467 decode=11.856 elapsed=12.323 queued=0.169 tok/s=21.5 +[tensorfold] done cmpl-7074e0f3e2f6dc646b0da47396d21876 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.678 decode=11.782 elapsed=12.460 queued=0.169 tok/s=21.6 +[tensorfold] done cmpl-686573e218111ef796551e10c3da8791 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.905 decode=11.678 elapsed=12.583 queued=0.169 tok/s=21.8 +[tensorfold] done cmpl-40fd73e523f5d97568ac86788fda3e0e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.152 decode=11.541 elapsed=12.693 queued=0.169 tok/s=22.1 +[tensorfold] done cmpl-28f20358cf36dc7babeb7d8eaf249022 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.415 decode=11.367 elapsed=12.782 queued=0.169 tok/s=22.4 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done cmpl-72ad26f0d673de34f1ddf5b15af8fc19 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/221 ttft=12.009 decode=3.942 elapsed=15.950 queued=0.169 tok/s=64.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 86, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-9f54d823bd8a4c12abcfc3515746c49b prompt=32 cached=0 thinking=False effort=none tokens=16 sha=e3dfbb415b68 finish=length rounds=5 accepted=10/12 ttft=0.160 decode=0.311 elapsed=0.471 queued=0.000 tok/s=48.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 87, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-07a2f05c5914fde53c6adc2b6b0dcacf prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=136 accepted=119/185 ttft=0.159 decode=7.317 elapsed=7.476 queued=0.000 tok/s=34.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 88, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-53c057d0fb8fb334d65d0d24fb18b911 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=255 accepted=0/0 ttft=0.149 decode=11.277 elapsed=11.427 queued=0.000 tok/s=22.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 89, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-2673ad95e4df394c11464985a09cc233 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=136 accepted=119/185 ttft=0.160 decode=7.312 elapsed=7.472 queued=0.000 tok/s=34.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 91, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 91, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-66264fbabd51ea4d89fe40a99deaab66 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.160 decode=10.167 elapsed=10.327 queued=0.000 tok/s=25.1 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-b8b0aadef20777b2ced286dff249111f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.381 decode=9.996 elapsed=10.377 queued=0.000 tok/s=25.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 95, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 95, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 95, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-2f5d2b69385dbd6ec0f576620e9244b5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.163 decode=14.566 elapsed=14.729 queued=0.000 tok/s=17.5 +[tensorfold] done chatcmpl-74d6fb7ed9343c5443c38d7622c7e598 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.383 decode=14.437 elapsed=14.819 queued=0.224 tok/s=17.7 +[tensorfold] done chatcmpl-e7c000e5933ad430ce0250c4f712e7e3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.628 decode=14.258 elapsed=14.886 queued=0.224 tok/s=17.9 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-ae79961f3d4dabc07c2246efdf7462eb prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.895 decode=14.040 elapsed=14.935 queued=0.224 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 96, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 103, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 103, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 103, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 103, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 103, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 103, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-6ada3fe58ea1d5a98647192b0d68c085 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.158 decode=20.652 elapsed=20.811 queued=0.000 tok/s=12.3 +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 104, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-8ce10d4ab65efd4eb0e5857d867214fb prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.380 decode=20.750 elapsed=21.130 queued=0.221 tok/s=12.3 +[tensorfold] done chatcmpl-265067897c54efccfbdb03c5e23adadb prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.622 decode=20.660 elapsed=21.282 queued=0.221 tok/s=12.3 +[tensorfold] done chatcmpl-fb5cdae66cf6e1b83bababb643c3b4c2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.888 decode=20.528 elapsed=21.416 queued=0.221 tok/s=12.4 +[tensorfold] done chatcmpl-eae1aa3691a2e503bdd58f52ef0d77dc prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.167 decode=20.362 elapsed=21.529 queued=0.220 tok/s=12.5 +[tensorfold] done chatcmpl-51e95de68ca528c70dd1d2b97cef4c3e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.464 decode=20.162 elapsed=21.625 queued=0.220 tok/s=12.6 +[tensorfold] done chatcmpl-52439527c4ab2ed2be377a307a42e42b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.777 decode=19.927 elapsed=21.704 queued=0.219 tok/s=12.8 +info: kv pool idle: promised 0, free 1237 + store 0 == total 1237 +[tensorfold] done chatcmpl-4ab920fb68462ab2eb28daf80d78fbbe prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=136 accepted=119/186 ttft=20.966 decode=7.674 elapsed=28.640 queued=0.219 tok/s=33.2 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +mp_mask=0x0} +host-a:4512:4512 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/server-rank1.log new file mode 100644 index 0000000000..b7eab5678f --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/server-rank1.log @@ -0,0 +1,6646 @@ +[kvp-serve] 2026-10-22T17:20:23+00:00 rank 1 of 3 context 1048576 parallel 8 binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=all TF_GLM_MM=unset args +info: rank 1: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-b:3997:3997 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:3997:3997 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:3997:3997 [0] NCCL INFO cudaDriverVersion 13030 +host-b:3997:3997 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:3997:3997 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:3997:3997 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:3997:3997 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:3997:3997 [0] NCCL INFO Count not open libibverbs.so +host-b:3997:3997 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:3997:3997 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:3997:3997 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:3997:3997 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x1245b520 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:3997:3997 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x1249cc20 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:3997:3997 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x124de320 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:3997:3997 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x1251fa20 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:3997:3997 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:3997:3997 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:3997:3997 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:3997:3997 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:3997:3997 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:3997:3997 [0] NCCL INFO Initialized NET plugin IB +host-b:3997:3997 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:3997:3997 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:3997:3997 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:3997:3997 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:3997:3997 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:3997:3997 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:3997:3997 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:3997:3997 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:3997:3997 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:3997:3997 [0] NCCL INFO Using network IB +host-b:3997:3997 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:3997:3997 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0xf823eb0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x8fa3f4c721825aca - Init START +host-b:3997:3997 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:3997:3997 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:3997:3997 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:3997:3997 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:3997:3997 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:3997:3997 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:3997:3997 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:3997:3997 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:3997:3997 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:3997:3997 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:3997:3997 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:3997:3997 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:3997:3997 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:3997:3997 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:3997:3997 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:3997:3997 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:3997:3997 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:3997:3997 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:3997:3997 [0] NCCL INFO Rank 1: 6 Net devices +host-b:3997:3997 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:3997:3997 [0] NCCL INFO comm 0xf823eb0 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:3997:3997 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:3997:3997 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:3997:3997 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:3997:3997 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:3997:3997 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:3997:3997 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:3997:4012 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-b:3997:4012 [0] NCCL INFO proxy listening socket at <56611> +host-b:3997:4013 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-b:3997:3997 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:3997:3997 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:3997:3997 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:3997:4014 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-b:3997:3997 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:3997:3997 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:3997:3997 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:3997:3997 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:3997:3997 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656c010000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656c054000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656c09c000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2035 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2036 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2035 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2035 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2036 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2036 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6583 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6584 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6585 pkey=0 pd=0xf5656c0f2420 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2163 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2164 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2163 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2163 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2164 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2164 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656c22e000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656c276000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656c2ba000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656c302000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656c346000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2035 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2036 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6615 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6616 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6617 pkey=0 pd=0xf5656c0dcdf0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2037 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2038 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2037 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8481 ctsFifoLkey=0x1b8481 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2037 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2038 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8481 ctsFifoLkey=0x1b8481 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2038 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6586 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6587 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6588 pkey=0 pd=0xf5656c0f2420 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2165 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2166 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2165 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1ddad7 ctsFifoLkey=0x1ddad7 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2165 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2166 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1ddad7 ctsFifoLkey=0x1ddad7 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2166 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6618 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6619 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6620 pkey=0 pd=0xf5656c0dcdf0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2163 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2164 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2037 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2038 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2165 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2166 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:3997 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:3997:3997 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:3997:3997 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:3997:3997 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:3997:3997 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656c5b7000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656c5fc000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656c641000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656c686000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2039 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2040 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2041 pkey=0 pd=0xf5656c053590 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2167 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2168 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2169 pkey=0 pd=0xf5656c0f2f90 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2042 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2043 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2044 pkey=0 pd=0xf5656c053590 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2170 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2171 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2172 pkey=0 pd=0xf5656c0f2f90 +host-b:3997:3997 [0] NCCL INFO Connected all trees +host-b:3997:3997 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:3997:3997 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:3997:3997 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:3997:3997 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656c8a5000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656c8e6000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656c92a000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6589 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6590 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6589 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6589 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6590 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6590 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6621 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6622 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6621 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19cdca ctsFifoLkey=0x19cdca +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6621 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6622 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19cdca ctsFifoLkey=0x19cdca +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6622 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6591 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6592 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6591 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x177532 ctsFifoLkey=0x177532 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6591 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6592 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x177532 ctsFifoLkey=0x177532 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6592 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656ca7f000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6589 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6590 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6621 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6622 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6623 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6624 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6623 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c7c4 ctsFifoLkey=0x19c7c4 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6623 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6624 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c7c4 ctsFifoLkey=0x19c7c4 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6624 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6591 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6592 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6623 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6624 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:3997 [0] NCCL INFO Connected binomial trees +host-b:3997:3997 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:3997:3997 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:3997:3997 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:3997:3997 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:3997:3997 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-b:3997:3997 [0] NCCL INFO ncclCommInitRank comm 0xf823eb0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x8fa3f4c721825aca - Init COMPLETE +host-b:3997:3997 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 1.68 (kernels 0.25, alloc 0.03, bootstrap 1.09, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.29, rest 0.00) +host-b:3997:3997 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:3997:3997 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:3997:3997 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:3997:3997 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:3997:3997 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:3997:3997 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:3997:3997 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:3997:3997 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656cb1a000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656cb5b000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656cba0000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656cbe4000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656cc2c000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656cc70000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5656ccb8000) +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5656ccfc000) +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2045 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2046 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2045 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b7977 ctsFifoLkey=0x1b7977 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2045 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2046 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b7977 ctsFifoLkey=0x1b7977 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2046 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6593 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6594 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6595 pkey=0 pd=0xf5656c0f2420 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2173 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2174 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2173 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1decea ctsFifoLkey=0x1decea +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2173 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2174 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1decea ctsFifoLkey=0x1decea +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2174 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6625 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6626 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6627 pkey=0 pd=0xf5656c0dcdf0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6628 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6629 pkey=0 pd=0xf5656c0dcdf0 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6628 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x190f0d ctsFifoLkey=0x190f0d +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6628 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6629 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x190f0d ctsFifoLkey=0x190f0d +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6629 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2045 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2046 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2173 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2174 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6628 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6629 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2175 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2176 pkey=0 pd=0xf5656c0f2f90 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP createinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 1: rank 0 ended; leaving +d: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2177 pkey=0 pd=0xf5656c0f2f90 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6596 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6597 pkey=0 pd=0xf5656c0f2420 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6596 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x151ad7 ctsFifoLkey=0x151ad7 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6596 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6597 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x151ad7 ctsFifoLkey=0x151ad7 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6597 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2047 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2048 pkey=0 pd=0xf5656c053590 oooRq=0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2049 pkey=0 pd=0xf5656c053590 +host-b:3997:4012 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6596 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:4012 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6597 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:3997:3997 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/server-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/server-rank2.log new file mode 100644 index 0000000000..3f992c32ff --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6all/server-rank2.log @@ -0,0 +1,1347 @@ +[kvp-serve] 2026-10-22T17:20:22+00:00 rank 2 of 3 context 1048576 parallel 8 binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=all TF_GLM_MM=unset args +info: rank 2: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-c:3985:3985 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:3985:3985 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:3985:3985 [0] NCCL INFO cudaDriverVersion 13030 +host-c:3985:3985 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:3985:3985 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:3985:3985 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:3985:3985 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:3985:3985 [0] NCCL INFO Count not open libibverbs.so +host-c:3985:3985 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:3985:3985 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:3985:3985 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:3985:3985 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x1e11e5f0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:3985:3985 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x1e15fcf0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:3985:3985 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x1e1a13f0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:3985:3985 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x1e1e2af0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:3985:3985 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:3985:3985 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:3985:3985 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:3985:3985 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:3985:3985 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:3985:3985 [0] NCCL INFO Initialized NET plugin IB +host-c:3985:3985 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:3985:3985 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:3985:3985 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:3985:3985 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:3985:3985 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:3985:3985 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:3985:3985 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:3985:3985 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:3985:3985 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:3985:3985 [0] NCCL INFO Using network IB +host-c:3985:3985 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:3985:3985 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x1b4e6f80 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x8fa3f4c721825aca - Init START +host-c:3985:3985 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:3985:3985 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:3985:3985 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:3985:3985 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:3985:3985 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:3985:3985 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:3985:3985 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:3985:3985 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:3985:3985 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:3985:3985 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:3985:3985 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:3985:3985 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:3985:3985 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:3985:3985 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:3985:3985 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:3985:3985 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:3985:3985 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:3985:3985 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:3985:3985 [0] NCCL INFO Rank 2: 6 Net devices +host-c:3985:3985 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:3985:3985 [0] NCCL INFO comm 0x1b4e6f80 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:3985:3985 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:3985:3985 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:3985:3985 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:3985:3985 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:3985:3985 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:3985:3985 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:3985:4001 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-c:3985:4000 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-c:3985:4000 [0] NCCL INFO proxy listening socket at <33279> +host-c:3985:4002 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-c:3985:3985 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:3985:3985 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:3985:3985 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:3985:3985 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:3985:3985 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:3985:3985 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:3985:3985 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:3985:3985 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9c010000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9c054000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9c09c000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9c0e0000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9c128000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9c16c000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9c1b4000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9c1f8000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2035 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2036 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2035 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbcba ctsFifoLkey=0x1bbcba +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2035 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2036 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbcba ctsFifoLkey=0x1bbcba +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2036 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1779 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1780 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1781 pkey=0 pd=0xf95a9c09bf70 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2163 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2164 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2163 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2163 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2164 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2164 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1907 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1908 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1909 pkey=0 pd=0xf95a9c2f1b30 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2037 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2038 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2037 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbfbc ctsFifoLkey=0x1bbfbc +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2037 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2038 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbfbc ctsFifoLkey=0x1bbfbc +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2038 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2165 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2166 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2165 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d1c1b ctsFifoLkey=0x1d1c1b +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2165 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2166 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d1c1b ctsFifoLkey=0x1d1c1b +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2166 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2035 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2036 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2163 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2164 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2037 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2038 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1782 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1783 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1784 pkey=0 pd=0xf95a9c09bf70 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2165 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2166 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1910 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1911 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1912 pkey=0 pd=0xf95a9c2f1b30 +host-c:3985:3985 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:3985:3985 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:3985:3985 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:3985:3985 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:3985:3985 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:3985:3985 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:3985:3985 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:3985:3985 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:3985:3985 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9c5ae000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9c5ef000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9c634000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1785 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1786 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1785 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17e4a1 ctsFifoLkey=0x17e4a1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1785 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1786 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17e4a1 ctsFifoLkey=0x17e4a1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1786 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1913 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1914 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1913 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19a6a4 ctsFifoLkey=0x19a6a4 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1913 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1914 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19a6a4 ctsFifoLkey=0x19a6a4 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1914 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9c72e000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9c776000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2039 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2040 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2041 pkey=0 pd=0xf95a9c053eb0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2167 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2168 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2169 pkey=0 pd=0xf95a9c283e70 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1787 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1788 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1787 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1787 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1788 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1788 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9c905000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1785 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1786 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1913 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1914 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1787 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1788 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2042 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2043 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2044 pkey=0 pd=0xf95a9c053eb0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9c9bb000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9c9ff000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1915 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1916 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1915 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19ebe8 ctsFifoLkey=0x19ebe8 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1915 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1916 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19ebe8 ctsFifoLkey=0x19ebe8 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1916 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2170 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2171 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2172 pkey=0 pd=0xf95a9c283e70 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1915 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1916 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:3985 [0] NCCL INFO Connected all trees +host-c:3985:3985 [0] NCCL INFO Connected binomial trees +host-c:3985:3985 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:3985:3985 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:3985:3985 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:3985:3985 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:3985:3985 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-c:3985:3985 [0] NCCL INFO ncclCommInitRank comm 0x1b4e6f80 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x8fa3f4c721825aca - Init COMPLETE +host-c:3985:3985 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.56 (kernels 0.24, alloc 0.06, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.23, rest 0.00) +host-c:3985:3985 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:3985:3985 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:3985:3985 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:3985:3985 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:3985:3985 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:3985:3985 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:3985:3985 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:3985:3985 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9cb18000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9cb59000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9cb9e000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2045 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2046 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2045 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bc9c7 ctsFifoLkey=0x1bc9c7 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2045 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2046 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bc9c7 ctsFifoLkey=0x1bc9c7 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2046 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9cc3d000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9cc85000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9ccc9000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf95a9cd11000) +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf95a9cd55000) +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1789 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1790 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1791 pkey=0 pd=0xf95a9c09bf70 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2173 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2174 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2173 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1dc5c1 ctsFifoLkey=0x1dc5c1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2173 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2174 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1dc5c1 ctsFifoLkey=0x1dc5c1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2174 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1917 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1918 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1919 pkey=0 pd=0xf95a9c2f1b30 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1920 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1921 pkey=0 pd=0xf95a9c2f1b30 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1920 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1920 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1921 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1921 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2175 pkey=0 pd=0xf95a9c283e70 oooRq=0 +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 2: rank 0 ended; leaving +3:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2176 pkey=0 pd=0xf95a9c283e70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2177 pkey=0 pd=0xf95a9c283e70 +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1792 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1793 pkey=0 pd=0xf95a9c09bf70 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1792 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1807c5 ctsFifoLkey=0x1807c5 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1792 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1793 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1807c5 ctsFifoLkey=0x1807c5 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1793 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2045 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2046 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2173 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2174 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1920 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1921 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1792 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1793 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:3985:4000 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2047 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2048 pkey=0 pd=0xf95a9c053eb0 oooRq=0 +host-c:3985:4000 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2049 pkey=0 pd=0xf95a9c053eb0 +host-c:3985:3985 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/client.log new file mode 100644 index 0000000000..a22f9741b1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/client.log @@ -0,0 +1,97 @@ +server up 2026-10-23 01:58:32 +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 48.6, "per_stream_tps": 48.6, "ttft_s_max": 0.11, "timing_valid": true, "end_to_end_tps": 47.79760008016688, "makespan_s": 5.355917442939244, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 59.5, "per_stream_tps": 32.2, "ttft_s_max": 0.27, "timing_valid": true, "end_to_end_tps": 58.94631401444367, "makespan_s": 8.685869652079418, "steady_tps": 61.7, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 78.0, "per_stream_tps": 20.7, "ttft_s_max": 0.68, "timing_valid": true, "end_to_end_tps": 77.6647689908668, "makespan_s": 13.184871510020457, "steady_tps": 81.0, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 96.7, "per_stream_tps": 13.2, "ttft_s_max": 1.73, "timing_valid": true, "end_to_end_tps": 96.53731344547545, "makespan_s": 21.21459492610302, "steady_tps": 103.3, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 35.2, "per_stream_tps": 35.2, "ttft_s_max": 0.16, "timing_valid": true, "end_to_end_tps": 34.53776751939664, "makespan_s": 7.412175667006522, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 52.9, "per_stream_tps": 27.6, "ttft_s_max": 0.38, "timing_valid": true, "end_to_end_tps": 52.27405056794766, "makespan_s": 9.794534657965414, "steady_tps": 55.1, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 69.3, "per_stream_tps": 18.2, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 68.85426849805982, "makespan_s": 14.871990108047612, "steady_tps": 74.2, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 85.7, "per_stream_tps": 11.4, "ttft_s_max": 2.13, "timing_valid": true, "end_to_end_tps": 85.48382361196853, "makespan_s": 23.95774912100751, "steady_tps": 91.2, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 70.5, "per_stream_tps": 70.5, "ttft_s_max": 0.11, "timing_valid": true, "end_to_end_tps": 68.71561576230782, "makespan_s": 3.725499614025466, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 81.8, "per_stream_tps": 41.6, "ttft_s_max": 0.28, "timing_valid": true, "end_to_end_tps": 80.6698486307507, "makespan_s": 6.346857080934569, "steady_tps": 83.5, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 115.2, "per_stream_tps": 30.0, "ttft_s_max": 0.68, "timing_valid": true, "end_to_end_tps": 114.24746459205294, "makespan_s": 8.962999779963866, "steady_tps": 121.3, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 152.0, "per_stream_tps": 20.5, "ttft_s_max": 1.68, "timing_valid": true, "end_to_end_tps": 151.34928133354623, "makespan_s": 13.531613642000593, "steady_tps": 167.0, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 35.0, "per_stream_tps": 35.0, "ttft_s_max": 0.16, "timing_valid": true, "end_to_end_tps": 34.365655118950734, "makespan_s": 7.449297826969996, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 50.6, "per_stream_tps": 25.6, "ttft_s_max": 0.38, "timing_valid": true, "end_to_end_tps": 49.95665958263789, "makespan_s": 10.248883818043396, "steady_tps": 51.6, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 69.6, "per_stream_tps": 17.9, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 69.14265796487372, "makespan_s": 14.809960018028505, "steady_tps": 72.4, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 88.4, "per_stream_tps": 11.6, "ttft_s_max": 2.09, "timing_valid": true, "end_to_end_tps": 88.11060503909225, "makespan_s": 23.243513071909547, "steady_tps": 93.2, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +logs/r6auto-conc.json: 16 cells; alone 60/60 equal (0 unequal, 0 failed); serial 72/72 equal (0 unequal, 0 failed) + 0 S1 t1 drafted prompt 7759 history 7757 cached 0 (allowed [0]) ttft 5.12s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok + 1 S1 t1 plain prompt 7759 history 7757 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 1.01s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok +system cuts [4563, 6099, 6611] + 2 S1 t2 drafted prompt 15550 history 15548 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 6.13s sha 539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b tool_calls ok + 3 S1 t2 plain prompt 15550 history 15548 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 0.83s sha 539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b tool_calls ok + 4 S1 t3 drafted prompt 19462 history 19460 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 3.43s sha 8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa tool_calls ok + 5 S1 t3 plain prompt 19462 history 19460 cached 18432 (allowed {'min': 11268, 'max': 19460}) ttft 0.74s sha 8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa tool_calls ok + 6 S1 t4 drafted prompt 21892 history 21890 cached 18432 (allowed {'min': 11268, 'max': 19460}) ttft 2.31s sha dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af tool_calls ok + 7 S1 t4 plain prompt 21892 history 21890 cached 20480 (allowed {'min': 13698, 'max': 21890}) ttft 0.92s sha dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af tool_calls ok + 8 S1 t5 drafted prompt 24388 history 24386 cached 20480 (allowed {'min': 13698, 'max': 21890}) ttft 2.53s sha 8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa length ok + 9 S1 t5 plain prompt 24388 history 24386 cached 22528 (allowed {'min': 16194, 'max': 24386}) ttft 1.13s sha 8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa length ok + 10 S1 t6 drafted prompt 26895 history 26893 cached 22528 (allowed {'min': 16194, 'max': 24386}) ttft 3.09s sha c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1 tool_calls ok + 11 S1 t6 plain prompt 26895 history 26893 cached 26624 (allowed {'min': 18701, 'max': 26893}) ttft 0.39s sha c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1 tool_calls ok + 12 S1 t7 drafted prompt 29798 history 29796 cached 26624 (allowed {'min': 18701, 'max': 26893}) ttft 2.21s sha 7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb tool_calls ok + 13 S1 t7 plain prompt 29798 history 29796 cached 28672 (allowed {'min': 21604, 'max': 29796}) ttft 0.81s sha 7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb tool_calls ok + 14 S1 t8 drafted prompt 35434 history 35432 cached 28672 (allowed {'min': 21604, 'max': 29796}) ttft 4.63s sha 1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e length ok + 15 S1 t8 plain prompt 35434 history 35432 cached 34816 (allowed {'min': 27240, 'max': 35432}) ttft 0.58s sha 1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e length ok + 16 S1 t9 drafted prompt 38152 history 38150 cached 34816 (allowed {'min': 27240, 'max': 35432}) ttft 2.29s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 length ok + 17 S1 t9 plain prompt 38152 history 38150 cached 36864 (allowed {'min': 29958, 'max': 38150}) ttft 0.89s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 length ok + 18 X t1 drafted prompt 2115 history 2113 cached 0 (allowed [0]) ttft 1.48s sha 7b1461bf71da95a2b74977aec25ecf7215b9f0fa044c82d2e1f4905ed66b798a length ok + 19 A t1 drafted prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 0.90s sha d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666 tool_calls ok + 20 A t1 plain prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 0.86s sha d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666 tool_calls ok + 21 B t1 drafted prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 0.92s sha 9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb tool_calls ok + 22 B t1 plain prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 0.88s sha 9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb tool_calls ok + 23 A t2 drafted prompt 12632 history 12630 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 4.39s sha e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe length ok + 24 A t2 plain prompt 12632 history 12630 cached 12288 (allowed {'min': 4438, 'max': 12630}) ttft 0.43s sha e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe length ok + 25 B t2 drafted prompt 12721 history 12719 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 4.43s sha f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2 tool_calls ok + 26 B t2 plain prompt 12721 history 12719 cached 12288 (allowed {'min': 4527, 'max': 12719}) ttft 0.47s sha f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2 tool_calls ok + 27 A t3 drafted prompt 15383 history 15381 cached 12288 (allowed {'min': 4438, 'max': 12630}) ttft 2.14s sha 7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8 tool_calls ok + 28 A t3 plain prompt 15383 history 15381 cached 14336 (allowed {'min': 7189, 'max': 15381}) ttft 0.77s sha 7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8 tool_calls ok + 29 B t3 drafted prompt 17330 history 17328 cached 12288 (allowed {'min': 4527, 'max': 12719}) ttft 3.39s sha 8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49 length ok + 30 B t3 plain prompt 17330 history 17328 cached 16384 (allowed {'min': 9136, 'max': 17328}) ttft 0.73s sha 8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49 length ok + 31 E t1 drafted prompt 40901 history 40899 cached 6144 (allowed {'min': 0, 'max': 6611}) ttft 22.72s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 length ok + 32 E t1 plain prompt 40901 history 40899 cached 38912 (allowed {'min': 32707, 'max': 40899}) ttft 1.27s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 length ok + 33 T t1 drafted prompt 1520 history 1517 cached 0 (allowed [0]) ttft 0.94s sha 033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89 stop ok + 34 T t1 plain prompt 1520 history 1517 cached 0 (allowed [0]) ttft 0.91s sha 033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89 stop ok + 35 T t2 drafted prompt 4202 history 4199 cached 0 (allowed [0]) ttft 3.03s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 36 T t2 plain prompt 4202 history 4199 cached 4096 (allowed {'min': 1, 'max': 4199}) ttft 0.27s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 37 C t1 cancel prompt None history 0 cached None (allowed None) ttft 0.00s sha None None ok + 38 C t1 drafted prompt 9702 history 9700 cached 6144 (allowed {'min': 0, 'max': 9688}) ttft 2.37s sha 88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c tool_calls ok + 39 C t1 plain prompt 9702 history 9700 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 0.97s sha 88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c tool_calls ok + 40 C t2 drafted prompt 15180 history 15178 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 4.67s sha 5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8 length ok + 41 C t2 plain prompt 15180 history 15178 cached 14336 (allowed {'min': 6986, 'max': 15178}) ttft 0.68s sha 5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8 length ok + 42 A t4 drafted prompt 18246 history 18244 cached 14336 (allowed {'min': 7189, 'max': 15381}) ttft 2.52s sha 9599ff0fa9b19bbb5cb5218a5b6f538de1146ce12dad36c729686f12f666389e length ok + 43 B t4 drafted prompt 19808 history 19806 cached 16384 (allowed {'min': 9136, 'max': 17328}) ttft 4.93s sha 025c4856c697674513547e9be3c33a84537e8a4fcb74e731ea09263af3e6d80d tool_calls ok +record: 44 requests, 0 not cached as planned +{"length": 1024, "rep": 0, "ttft": 0.7261461779708043, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.239251697086729, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.2371755649801344, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.235684675979428, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 5.378336450899951, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 5.375394148984924, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 5.376257557072677, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 10.638155551045202, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 10.633202360011637, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 10.864173055044375, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 21.435393912019208, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 21.35277433693409, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 21.30219480697997, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 43.26865292596631, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 43.16799386893399, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 43.215121807996184, "prompt": 65536, "cached": 0} +alone p2048-0-0: prompt 2048 ttft 1.2527443950530142 sha 7d62171119f86883ec9ea4e3542846c00509f1ad0baba0a0676ccd3b8e14cd56 +alone p2048-1-0: prompt 2048 ttft 1.2454646700061858 sha c948b4bcbd5833ccd9c3bbd0f64e960f0ed36def137ba411ed3ade9437416854 +alone p2048-2-0: prompt 2047 ttft 1.235843372065574 sha 6a2fb209181c72c924d2f9c33c76a176bd29daed59cf2e524f9797dc123b2b19 +alone p2048-0-1: prompt 2048 ttft 1.2485667179571465 sha f8aaeecc59d0c280cda17bf00ec2c94ab68bf77c8f76edf8f6198403d239a62b +alone p2048-1-1: prompt 2048 ttft 1.2499047779710963 sha 854ffaa5d75926074fa12f5f072ace31fab67f02428f5cc5cde0f8536b956273 +alone p2048-2-1: prompt 2047 ttft 1.2417146119987592 sha 0effe3e2e261cfefd427f381703c20e16f63a81a352520ec51fc2d517053699c +alone p2048-0-2: prompt 2048 ttft 1.2480726790381595 sha 2d30fcc66f9925bb2dbab2bfa2c3b819cfbe1e820d089d4774f26089df2b46b3 + p2048-0-0: prompt 2048 first 7.31 s last 21.37 s equal_alone True started after - + p2048-1-0: prompt 2048 first 5.94 s last 22.09 s equal_alone True started after - + p2048-2-0: prompt 2047 first 3.26 s last 21.50 s equal_alone True started after - + p2048-0-1: prompt 2048 first 11.57 s last 21.50 s equal_alone True started after - + p2048-1-1: prompt 2048 first 8.71 s last 22.74 s equal_alone True started after - + p2048-2-1: prompt 2047 first 4.57 s last 20.30 s equal_alone True started after - + p2048-0-2: prompt 2048 first 10.13 s last 21.06 s equal_alone True started after - + long65536: prompt 65536 first 1.44 s last 19.43 s equal_alone None started after - +together == alone: 7/7; 0 request(s) started only after another ended +=== kvp-client r6auto rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/cold.json b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/cold.json new file mode 100644 index 0000000000..5ccc8a70a6 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.7261461779708043, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.239251697086729, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.2371755649801344, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.235684675979428, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 5.378336450899951, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 5.375394148984924, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 5.376257557072677, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 10.638155551045202, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 10.633202360011637, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 10.864173055044375, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 21.435393912019208, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 21.35277433693409, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 21.30219480697997, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 43.26865292596631, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 43.16799386893399, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 43.215121807996184, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/conc.json b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/conc.json new file mode 100644 index 0000000000..02f17dee8b --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/conc.json @@ -0,0 +1,9511 @@ +{ + "label": "", + "tokens": 256, + "mixed": false, + "cells": [ + { + "prompt": "code", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.6, + "per_stream_tps": 48.6, + "ttft_s_max": 0.11, + "timing_valid": true, + "end_to_end_tps": 47.79760008016688, + "makespan_s": 5.355917442939244, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.613658914189806, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 5.355917442939244, + "end_to_end_tps": 47.79760008016688, + "timing_valid": true, + "per_stream_tps": [ + 48.6 + ], + "ttft_s": [ + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + } + ], + "client_intervals": [ + { + "start": 721965.978951871, + "first": 721966.089316713, + "last": 721971.334756075, + "end": 721971.334869314, + "elapsed_s": 5.355917442939244, + "chunk_offsets_s": [ + 0.1103648419957608, + 0.1690374210011214, + 0.2249681029934436, + 0.2857015599729493, + 0.3503274219110608, + 0.416903082979843, + 0.4800361020024866, + 0.5435489189112559, + 0.6079701439011842, + 0.6759794129757211, + 0.7425863239914179, + 0.8097116509452462, + 0.8770395659375936, + 0.9431521910009906, + 1.0085484359879047, + 1.0700170799391344, + 1.1278117559850216, + 1.1815989539027214, + 1.2318789609707892, + 1.285827591898851, + 1.344758759951219, + 1.40523629297968, + 1.464194597909227, + 1.523555391933769, + 1.5842017359100282, + 1.652236818918027, + 1.7213797139702365, + 1.7883260109229013, + 1.8518017759779468, + 1.918222184991464, + 1.982574658934027, + 2.0481289189774543, + 2.11342721094843, + 2.1800250539090484, + 2.2493686659727246, + 2.3170974529348314, + 2.384483653935604, + 2.4515022319974378, + 2.521436251932755, + 2.5895337439142168, + 2.6632689879043028, + 2.7294629079988226, + 2.7968916709069163, + 2.8660692129051313, + 2.9339188339654356, + 3.00337440893054, + 3.0718656689859927, + 3.140444357995875, + 3.2092957359272987, + 3.2780605329899117, + 3.3459873169194907, + 3.414182659937069, + 3.481636709999293, + 3.5480648579541594, + 3.6143545180093497, + 3.6822536669205874, + 3.7468081009574234, + 3.804710002965294, + 3.8640424389159307, + 3.926869247923605, + 3.99172422895208, + 4.056020198971964, + 4.145202896906994, + 4.214887931011617, + 4.282057003933005, + 4.35161385894753, + 4.4150890819728374, + 4.479081614990719, + 4.568513962905854, + 4.63487617589999, + 4.701372126000933, + 4.768947324948385, + 4.83653017191682, + 4.902226776932366, + 4.971744557959028, + 5.03814896591939, + 5.106914572999813, + 5.176130943000317, + 5.243834677967243, + 5.307841097004712, + 5.355804204009473 + ], + "chunk_gaps_ms": { + "count": 80, + "p50": 66.46204902790487, + "p95": 69.6850341046229, + "max": 89.43234791513532 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.5, + "per_stream_tps": 32.2, + "ttft_s_max": 0.27, + "timing_valid": true, + "end_to_end_tps": 58.94631401444367, + "makespan_s": 8.685869652079418, + "steady_tps": 61.7, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.46520260415213, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 8.685869652079418, + "end_to_end_tps": 58.94631401444367, + "timing_valid": true, + "steady_tps": 61.656002370534324, + "per_stream_tps": [ + 34.8, + 29.7 + ], + "ttft_s": [ + 0.27, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + } + ], + "client_intervals": [ + { + "start": 721971.336820856, + "first": 721971.611560662, + "last": 721978.943038738, + "end": 721978.943223117, + "elapsed_s": 7.6064022610662505, + "chunk_offsets_s": [ + 0.2747398060746491, + 0.3547311050351709, + 0.44582837098278105, + 0.5385212859837338, + 0.6321583860553801, + 0.7240302109858021, + 0.8131561580812559, + 0.9048851740080863, + 0.9951987180393189, + 1.089983233017847, + 1.1858550290344283, + 1.2812808310845867, + 1.3775347620248795, + 1.473929230007343, + 1.5693219089880586, + 1.660418536979705, + 1.7520959050161764, + 1.8427417380735278, + 1.9348638510564342, + 2.0293747190153226, + 2.1251612320775166, + 2.2209189910208806, + 2.315444307983853, + 2.4104418290080503, + 2.507595528033562, + 2.603292979998514, + 2.697409456013702, + 2.7932534280698746, + 2.887307996978052, + 2.979935207986273, + 3.0739469439722598, + 3.170644377009012, + 3.270908409031108, + 3.368364943074994, + 3.4642284869914874, + 3.558259007986635, + 3.6540637320140377, + 3.7513071660650894, + 3.8466888080583885, + 3.941894198069349, + 4.039655921049416, + 4.134571026079357, + 4.230720636085607, + 4.329346282989718, + 4.428454854059964, + 4.528147109085694, + 4.627475326065905, + 4.7282576450379565, + 4.823957756976597, + 4.919274470070377, + 5.021798224071972, + 5.118989082984626, + 5.2146161530399695, + 5.310362349031493, + 5.40639994002413, + 5.501632138038985, + 5.599726603017189, + 5.693199303001165, + 5.790122674079612, + 5.887271114974283, + 5.986925185075961, + 6.08538463304285, + 6.182902634027414, + 6.276923899073154, + 6.370925466064364, + 6.47055047203321, + 6.565706412075087, + 6.66016784706153, + 6.755735732032917, + 6.851473595015705, + 6.947349879075773, + 7.043861214071512, + 7.144739545066841, + 7.2458926130784675, + 7.342275589006022, + 7.437331430031918, + 7.526441685040481, + 7.60621788201388 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 95.69745196495205, + "p95": 100.78231897205114, + "max": 102.52375400159508 + } + }, + { + "start": 721971.336719218, + "first": 721971.44574968, + "last": 721980.022194015, + "end": 721980.02258887, + "elapsed_s": 8.685869652079418, + "chunk_offsets_s": [ + 0.10903046210296452, + 0.16713138204067945, + 0.35469372605439276, + 0.4461315250955522, + 0.5384842751082033, + 0.6320982220349833, + 0.7239967361092567, + 0.8130596081027761, + 0.9051504220115021, + 0.995500224060379, + 1.090287491097115, + 1.1857556470204145, + 1.28118567308411, + 1.3775858690496534, + 1.4739381260005757, + 1.5695066240150481, + 1.6605857471004128, + 1.7521544040646404, + 1.8428843870060518, + 1.935028117033653, + 2.0295424250653014, + 2.1252006420399994, + 2.22097509005107, + 2.315494791022502, + 2.4104883280815557, + 2.5077546579996124, + 2.6033382141031325, + 2.697445554076694, + 2.7934174860129133, + 2.8874593341024593, + 2.9800836000358686, + 3.0741046010516584, + 3.170803747023456, + 3.271056609111838, + 3.368524953024462, + 3.464272633078508, + 3.558313427027315, + 3.6542305420152843, + 3.7514522300334647, + 3.8468399370322004, + 3.942062896094285, + 4.03981626604218, + 4.1345541280461475, + 4.2308605801081285, + 4.329495668062009, + 4.42861793609336, + 4.528369890060276, + 4.627636327059008, + 4.728415175108239, + 4.824115606024861, + 4.919330505072139, + 5.021853027050383, + 5.119049999048002, + 5.214776178007014, + 5.310532984090969, + 5.406557501060888, + 5.5018705680267885, + 5.599503341014497, + 5.693054671050049, + 5.78983614500612, + 5.886943543097004, + 5.986857260111719, + 6.085216351086274, + 6.183091285056435, + 6.2767687540035695, + 6.370630488032475, + 6.470230917097069, + 6.56550337607041, + 6.660043808049522, + 6.755346061079763, + 6.851315345033072, + 6.946977585088462, + 7.043640945106745, + 7.144402405014262, + 7.245907750097103, + 7.3419541930779815, + 7.437027219100855, + 7.526208679075353, + 7.606798141030595, + 7.671702894032933, + 7.727794707054272, + 7.779443329083733, + 7.832357657025568, + 7.8911473751068115, + 7.948391062091105, + 8.006716072093695, + 8.057655267999507, + 8.110899849096313, + 8.169154983013868, + 8.227272701100446, + 8.28911827201955, + 8.348773249075748, + 8.404569774051197, + 8.468937702011317, + 8.553487307042815, + 8.627529889112338, + 8.685474797035567 + ], + "chunk_gaps_ms": { + "count": 96, + "p50": 95.21892905468121, + "p95": 100.76145990751684, + "max": 187.5623440137133 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 78.0, + "per_stream_tps": 20.7, + "ttft_s_max": 0.68, + "timing_valid": true, + "end_to_end_tps": 77.6647689908668, + "makespan_s": 13.184871510020457, + "steady_tps": 81.0, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 78.02165768438135, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 13.184871510020457, + "end_to_end_tps": 77.6647689908668, + "timing_valid": true, + "steady_tps": 80.96613675280626, + "per_stream_tps": [ + 23.1, + 19.5, + 20.6, + 20.8 + ], + "ttft_s": [ + 0.28, + 0.11, + 0.46, + 0.68 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0b4d014c8872", + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b82191a9a6ec", + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a" + } + ], + "client_intervals": [ + { + "start": 721980.026556978, + "first": 721980.302281033, + "last": 721991.348383873, + "end": 721991.348700903, + "elapsed_s": 11.322143925004639, + "chunk_offsets_s": [ + 0.27572405501268804, + 0.3557230429723859, + 0.5706963209668174, + 0.8065064819529653, + 0.934989374014549, + 1.063963304972276, + 1.1927871339721605, + 1.317923684953712, + 1.447653983021155, + 1.5822634160285816, + 1.7222260220441967, + 1.8545725300209597, + 1.9921383670298383, + 2.1325110950274393, + 2.2709855609573424, + 2.404621221940033, + 2.5331133720465004, + 2.6686104179825634, + 2.8040230449987575, + 2.943199616041966, + 3.081027835025452, + 3.2225127490237355, + 3.364957949030213, + 3.501617201953195, + 3.645060310023837, + 3.783946773968637, + 3.9233454290078953, + 4.066181235015392, + 4.210603996994905, + 4.352116244030185, + 4.4909251580247656, + 4.629292543046176, + 4.77525493002031, + 4.919531392981298, + 5.063240885036066, + 5.204821396037005, + 5.3422927260398865, + 5.485698817996308, + 5.6281073819845915, + 5.771684919949621, + 5.913614562014118, + 6.058636095025577, + 6.205102483043447, + 6.347746463958174, + 6.496379549964331, + 6.644668451976031, + 6.7948918079491705, + 6.9448542080353945, + 7.088660222943872, + 7.232169733964838, + 7.378474980010651, + 7.518315052962862, + 7.659393301000819, + 7.802494772011414, + 7.940090226940811, + 8.082278380985372, + 8.228432320989668, + 8.369498915039003, + 8.51356588199269, + 8.656064937007613, + 8.809367403038777, + 8.9817755400436, + 9.164038643008098, + 9.307816258980893, + 9.44943105103448, + 9.598040897981264, + 9.741799116018228, + 9.88432814204134, + 10.029783132020384, + 10.176376024028286, + 10.322309550945647, + 10.46475190401543, + 10.613684677984565, + 10.76306264998857, + 10.906214499031194, + 11.048414135002531, + 11.189991308026947, + 11.321826895000413 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 142.44235306978226, + "p95": 172.4081370048225, + "max": 235.81016098614782 + } + }, + { + "start": 721980.026701354, + "first": 721980.137943022, + "last": 721993.211236134, + "end": 721993.21132349, + "elapsed_s": 13.18462213606108, + "chunk_offsets_s": [ + 0.11124166799709201, + 0.16745839302893728, + 0.3557528860401362, + 0.5701776990899816, + 0.8061998820630834, + 0.9350515210535377, + 1.0636975810630247, + 1.1921599870547652, + 1.3181603529956192, + 1.4470216979971156, + 1.5821893820539117, + 1.721888150088489, + 1.8541911350330338, + 1.9920520050218329, + 2.1320608220994473, + 2.2706033950671554, + 2.404246614081785, + 2.53273800003808, + 2.6680897240294144, + 2.80355222907383, + 2.942800570046529, + 3.080730422050692, + 3.2220234140986577, + 3.364385145017877, + 3.50089507503435, + 3.6442583330208436, + 3.783410457079299, + 3.9228331240592524, + 4.065773523063399, + 4.210160099086352, + 4.35142588801682, + 4.490538142039441, + 4.62887934606988, + 4.7750078700482845, + 4.919018044020049, + 5.062889183987863, + 5.204423955990933, + 5.341798995039426, + 5.485294479061849, + 5.627656803000718, + 5.771183947101235, + 5.913350598071702, + 6.058253595023416, + 6.204517433070578, + 6.347238800022751, + 6.495895097032189, + 6.644032092066482, + 6.794426856096834, + 6.944200098048896, + 7.08823414798826, + 7.232201860053465, + 7.378014057991095, + 7.517934618052095, + 7.658851035055704, + 7.802090820041485, + 7.939723357092589, + 8.081984434043989, + 8.228154248092324, + 8.369134354987182, + 8.513586719054729, + 8.65611585509032, + 8.80955977505073, + 8.981898563099094, + 9.163655691081658, + 9.307370764086954, + 9.449195227003656, + 9.597609817050397, + 9.74142192909494, + 9.883802267024294, + 10.02920216799248, + 10.175962605047971, + 10.321879516006447, + 10.464317520032637, + 10.613242537016049, + 10.762808080995455, + 10.905513331061229, + 11.04811218008399, + 11.189551509101875, + 11.32192469201982, + 11.44078232604079, + 11.559911731048487, + 11.681290772045031, + 11.796400825027376, + 11.910995212034322, + 12.031861327006482, + 12.147224211017601, + 12.267706459038891, + 12.388250322081149, + 12.507016621064395, + 12.6316459790105, + 12.750808785087429, + 12.850846320041455, + 12.929861785029061, + 12.996665341081098, + 13.079630797030404, + 13.135424629086629, + 13.184534780099057 + ], + "chunk_gaps_ms": { + "count": 96, + "p50": 141.2793904892169, + "p95": 172.33878804836422, + "max": 236.02218297310174 + } + }, + { + "start": 721980.027207944, + "first": 721980.488851718, + "last": 721992.877263363, + "end": 721992.877606655, + "elapsed_s": 12.850398711045273, + "chunk_offsets_s": [ + 0.4616437740623951, + 0.5696260340046138, + 0.8054461650317535, + 0.9341822930146009, + 1.0634047260973603, + 1.1917040760163218, + 1.3169321280438453, + 1.4463125930633396, + 1.5812943840865046, + 1.7211438410449773, + 1.853429742041044, + 1.9911237120395526, + 2.131628058035858, + 2.2701732370769605, + 2.4036463780794293, + 2.532095905044116, + 2.6675424680579454, + 2.8032493180362508, + 2.942257640999742, + 3.0805719090858474, + 3.221588244079612, + 3.3638155889930204, + 3.50033855100628, + 3.643698133993894, + 3.782856988022104, + 3.922208495088853, + 4.065051048062742, + 4.209442684077658, + 4.351173285045661, + 4.489818896050565, + 4.6281629810109735, + 4.774343180004507, + 4.918465364025906, + 5.062327035004273, + 5.204074684996158, + 5.341237713000737, + 5.484735107049346, + 5.627083089086227, + 5.770632851985283, + 5.912798485020176, + 6.0577029610285535, + 6.203944388078526, + 6.346955739078112, + 6.495598218054511, + 6.643459388054907, + 6.7937756350729614, + 6.943655873066746, + 7.087673351052217, + 7.23174417600967, + 7.377463556011207, + 7.517361888079904, + 7.6582971080206335, + 7.801543676992878, + 7.9391715010860935, + 8.08151936903596, + 8.227684963028878, + 8.36867674801033, + 8.513036846998148, + 8.655657133087516, + 8.809012993006036, + 8.981462103081867, + 9.163191077997908, + 9.30690242908895, + 9.448874879046343, + 9.597143902094103, + 9.740809318027459, + 9.883448378997855, + 10.028651733067818, + 10.175302295014262, + 10.321563572040759, + 10.463735509081744, + 10.612771196989343, + 10.762172010028735, + 10.90504286705982, + 11.047540395986289, + 11.189081825083122, + 11.321368177072145, + 11.440318493056111, + 11.55934486107435, + 11.680851996992715, + 11.795974284992553, + 11.910551818087697, + 12.031419415026903, + 12.146804345073178, + 12.267256158054806, + 12.387801518081687, + 12.506573314080015, + 12.631073605036363, + 12.750189108075574, + 12.850055419025011 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 141.73060096800327, + "p95": 150.31624701805413, + "max": 235.82013102713972 + } + }, + { + "start": 721980.02645198, + "first": 721980.706207959, + "last": 721992.956931333, + "end": 721992.957064208, + "elapsed_s": 12.93061222799588, + "chunk_offsets_s": [ + 0.6797559789847583, + 0.8067983000073582, + 0.935013244044967, + 1.0637264549732208, + 1.1926210490055382, + 1.3181997500360012, + 1.4468545720446855, + 1.5822851080447435, + 1.7216829250101, + 1.8542313280049711, + 1.992097335983999, + 2.1321860080352053, + 2.270893641980365, + 2.4045879630139098, + 2.5329110380262136, + 2.6684058400569484, + 2.8040394090348855, + 2.9428452310385182, + 3.081089462968521, + 3.222771564964205, + 3.365302860038355, + 3.5015218780608848, + 3.6449230869766325, + 3.7839045010041445, + 3.9232312319800258, + 4.066532972967252, + 4.210963101009838, + 4.352317622979172, + 4.490992548991926, + 4.629448038991541, + 4.775392170995474, + 4.919680475955829, + 5.0633786680409685, + 5.20488854800351, + 5.342352804960683, + 5.485842399997637, + 5.628176899976097, + 5.771747021004558, + 5.913775430060923, + 6.058706440962851, + 6.205155350966379, + 6.347885753028095, + 6.496450018021278, + 6.644739222014323, + 6.795082501950674, + 6.944923912058584, + 7.088730341056362, + 7.232357928995043, + 7.378628253005445, + 7.518456656020135, + 7.659369235043414, + 7.802636343054473, + 7.940233182976954, + 8.082597514032386, + 8.22876307705883, + 8.369638102012686, + 8.513603106024675, + 8.656248409999534, + 8.80970428895671, + 8.98207495699171, + 9.164102923008613, + 9.30787631298881, + 9.449696363997646, + 9.598205078043975, + 9.742123928968795, + 9.884633715962991, + 10.030073066009209, + 10.176662566023879, + 10.322529472061433, + 10.465053330059163, + 10.613953499007039, + 10.76344822498504, + 10.906630482058972, + 11.048714144038968, + 11.190297640045173, + 11.322006989037618, + 11.441339156008326, + 11.560657590976916, + 11.681775628007017, + 11.797206757008098, + 11.911758259986527, + 12.032627832028084, + 12.147760665044188, + 12.268307410995476, + 12.38898802001495, + 12.507615054957569, + 12.632295556017198, + 12.751376198953949, + 12.851661831024103, + 12.930479353060946 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 141.50987996254116, + "p95": 149.84141010791063, + "max": 182.02796601690352 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 96.7, + "per_stream_tps": 13.2, + "ttft_s_max": 1.73, + "timing_valid": true, + "end_to_end_tps": 96.53731344547545, + "makespan_s": 21.21459492610302, + "steady_tps": 103.3, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 96.66070094981092, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 21.21459492610302, + "end_to_end_tps": 96.53731344547545, + "timing_valid": true, + "steady_tps": 103.2569216801111, + "per_stream_tps": [ + 14.8, + 12.6, + 13.5, + 13.1, + 14.5, + 13.2, + 13.2, + 12.2 + ], + "ttft_s": [ + 0.46, + 0.68, + 1.73, + 0.11, + 1.16, + 0.9, + 1.44, + 0.28 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0b4d014c8872", + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b82191a9a6ec", + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "3f4413b23169", + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "51fe79c071ab", + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e9b7a497dd30", + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "097c9fe5896b", + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05" + } + ], + "client_intervals": [ + { + "start": 721993.217837032, + "first": 721993.682530535, + "last": 722010.909802213, + "end": 722010.910093803, + "elapsed_s": 17.692256770911627, + "chunk_offsets_s": [ + 0.4646935029886663, + 0.5719976499676704, + 0.7992334109731019, + 1.055307060945779, + 1.331525516929105, + 1.626395576982759, + 1.9311582410009578, + 2.1444615270011127, + 2.3534456229535863, + 2.5700245569460094, + 2.782207188894972, + 2.9940917069325224, + 3.214274376980029, + 3.431231708964333, + 3.646232926985249, + 3.8528622499434277, + 4.057663082960062, + 4.2724337939871475, + 4.4913076009834185, + 4.709921441972256, + 4.925175628974102, + 5.147941169911064, + 5.371172967948951, + 5.597156829899177, + 5.821021661977284, + 6.044663854991086, + 6.270677429973148, + 6.490274657960981, + 6.711482341983356, + 6.931513696908951, + 7.1490040679927915, + 7.373397001996636, + 7.590906414901838, + 7.808137607993558, + 8.034745028009638, + 8.257304101949558, + 8.479971201973967, + 8.703030266915448, + 8.927211258909665, + 9.148642691900022, + 9.370493341004476, + 9.598108309903182, + 9.826776008936577, + 10.05046318192035, + 10.276590391993523, + 10.50134915090166, + 10.725306859938428, + 10.946285035926849, + 11.167551063001156, + 11.389719218946993, + 11.611104651005007, + 11.830811777967028, + 12.053258961997926, + 12.272866848972626, + 12.496565874898806, + 12.720289163989946, + 12.945723993936554, + 13.167971697985195, + 13.389591570943594, + 13.620488954009488, + 13.847903833957389, + 14.073684450006112, + 14.299823660985567, + 14.519224685966037, + 14.744435640983284, + 14.974192305002362, + 15.199024389963597, + 15.427228599903174, + 15.6548487219261, + 15.88599861098919, + 16.116346820956096, + 16.348179160966538, + 16.577232765965164, + 16.808681942988187, + 17.03387154894881, + 17.252936188946478, + 17.475774568971246, + 17.69196518091485 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 222.8383800247684, + "p95": 256.07364997267723, + "max": 304.76266401819885 + } + }, + { + "start": 721993.216480033, + "first": 721993.895218526, + "last": 722014.092192952, + "end": 722014.092420226, + "elapsed_s": 20.875940192956477, + "chunk_offsets_s": [ + 0.6787384930066764, + 0.8010149269830436, + 1.0569965629838407, + 1.3331254760269076, + 1.6282152759376913, + 1.933435300947167, + 2.1466372290160507, + 2.355595043976791, + 2.572217328939587, + 2.7841083329403773, + 2.9960659409407526, + 3.2160570380510762, + 3.433159466017969, + 3.6483677360229194, + 3.8549485550029203, + 4.059577667037956, + 4.2743780290475115, + 4.492464683949947, + 4.711442505940795, + 4.926852994947694, + 5.149677379988134, + 5.372588124941103, + 5.59893788699992, + 5.82261124800425, + 6.04619596793782, + 6.271307870047167, + 6.491798209026456, + 6.7131140909623355, + 6.933079947018996, + 7.150852688006125, + 7.374918257002719, + 7.592346059973352, + 7.809632441028953, + 8.03618757496588, + 8.258951066993177, + 8.482078002998605, + 8.703504896024242, + 8.92910333094187, + 9.150869076955132, + 9.37225961103104, + 9.599875823943876, + 9.82912109803874, + 10.05219307297375, + 10.278367909952067, + 10.503149030962959, + 10.727215006947517, + 10.947757638990879, + 11.169120977981947, + 11.391364153008908, + 11.612707313965075, + 11.832411314942874, + 12.054106757976115, + 12.27448258199729, + 12.498220036970451, + 12.72184938204009, + 12.947314632940106, + 13.169757804018445, + 13.3916308300104, + 13.622146695968695, + 13.849791900021955, + 14.075168146984652, + 14.301408304017968, + 14.520491014001891, + 14.745461239945143, + 14.97645520803053, + 15.201115558971651, + 15.429012152017094, + 15.657399600953795, + 15.88777252600994, + 16.117920294986106, + 16.34967170096934, + 16.57892619096674, + 16.810198753955774, + 17.036046145949513, + 17.25501834694296, + 17.47792356100399, + 17.693925976986066, + 17.894441419048235, + 18.10258646204602, + 18.30401423200965, + 18.504964850959368, + 18.710230487049557, + 18.89241247694008, + 19.075686466996558, + 19.263187741977163, + 19.443586503970437, + 19.616481062956154, + 19.774261479033157, + 19.94345639704261, + 20.106176884961314, + 20.264476484968327, + 20.407778959022835, + 20.542854530038312, + 20.66745126096066, + 20.78278481704183, + 20.875712919048965 + ], + "chunk_gaps_ms": { + "count": 95, + "p50": 221.3633389910683, + "p95": 231.75140598323196, + "max": 305.22002500947565 + } + }, + { + "start": 721993.219341293, + "first": 721994.950466054, + "last": 722013.884067237, + "end": 722013.884208128, + "elapsed_s": 20.664866834995337, + "chunk_offsets_s": [ + 1.7311247610487044, + 1.9301061619771644, + 2.143494255025871, + 2.352345988037996, + 2.5688762640347704, + 2.7811467610299587, + 2.9929851869819686, + 3.213148183072917, + 3.430121661047451, + 3.6452384870499372, + 3.851798568968661, + 4.056567327002995, + 4.2714816350489855, + 4.490023614023812, + 4.708986929035746, + 4.923819556017406, + 5.146928739035502, + 5.370133307063952, + 5.596257496043108, + 5.819913989980705, + 6.043473583995365, + 6.268816451076418, + 6.4892354720504954, + 6.710503217997029, + 6.93063538800925, + 7.148295814986341, + 7.3719859750708565, + 7.589739006012678, + 7.806949262041599, + 8.03356229502242, + 8.256537746055983, + 8.479157802066766, + 8.70071334100794, + 8.925905092968605, + 9.147915921988897, + 9.369547166046686, + 9.597166600986384, + 9.825967628974468, + 10.049591796007007, + 10.275739963981323, + 10.50055716407951, + 10.723840879043564, + 10.944384213071316, + 11.165733201080002, + 11.387881339062005, + 11.609409086988308, + 11.830017999047413, + 12.051338533055969, + 12.27153103100136, + 12.49513651104644, + 12.7186139250407, + 12.944072731072083, + 13.16650958696846, + 13.388146194978617, + 13.61902981007006, + 13.84635906596668, + 14.071734601049684, + 14.29821015696507, + 14.517911150003783, + 14.742686328012496, + 14.973546114983037, + 15.19799579505343, + 15.426037332974374, + 15.653864340973087, + 15.885120099061169, + 16.11510262999218, + 16.346954709966667, + 16.5762601030292, + 16.80728931899648, + 17.032798130996525, + 17.25209177006036, + 17.47501611802727, + 17.690731827984564, + 17.891472425078973, + 18.09950014401693, + 18.30129030207172, + 18.502012718003243, + 18.707138376077637, + 18.890071411966346, + 19.072749922052026, + 19.260581417009234, + 19.4403808680363, + 19.61371741699986, + 19.77156438701786, + 19.9407287259819, + 20.10296669404488, + 20.262163161998615, + 20.404998767073266, + 20.54012798401527, + 20.664725944050588 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 221.32053400855511, + "p95": 230.85978697054088, + "max": 231.85207997448742 + } + }, + { + "start": 721993.217054455, + "first": 721993.325878349, + "last": 722012.832481536, + "end": 722012.83259112, + "elapsed_s": 19.615536665078253, + "chunk_offsets_s": [ + 0.10882389405742288, + 0.16712324600666761, + 0.3580565539887175, + 0.5728249860694632, + 0.8000719490228221, + 1.0560419569956139, + 1.3322146460413933, + 1.627087526023388, + 1.9319993030512705, + 2.1449081050232053, + 2.354289305047132, + 2.5709270330844447, + 2.7830367820570245, + 2.995210394030437, + 3.2147400630638003, + 3.4323388760676607, + 3.6472756820730865, + 3.853816175018437, + 4.058275649091229, + 4.273539273068309, + 4.492243180051446, + 4.711131334071979, + 4.925735191092826, + 5.14888679201249, + 5.37205027602613, + 5.5981105350656435, + 5.821845481987111, + 6.045568919042125, + 6.270544541068375, + 6.491278619039804, + 6.712211490026675, + 6.932624676031992, + 7.150196800008416, + 7.374396297032945, + 7.591582810040563, + 7.808973029023036, + 8.035367142991163, + 8.258126339060254, + 8.480958508094773, + 8.703552587074228, + 8.927777258097194, + 9.149529093992896, + 9.371443983050995, + 9.599100020015612, + 9.827692250022665, + 10.051420410047285, + 10.277487729093991, + 10.502251320052892, + 10.726248282007873, + 10.947578583029099, + 11.168177540996112, + 11.389918318018317, + 11.611657818080857, + 11.83154262509197, + 12.053804182098247, + 12.273574849008583, + 12.49719615408685, + 12.721122056012973, + 12.946448376053013, + 13.168832831084728, + 13.39028804609552, + 13.621022924082354, + 13.848497726023197, + 14.074075877084397, + 14.300548864062876, + 14.520123574067838, + 14.745303496019915, + 14.975193783990107, + 15.199674324016087, + 15.42779878503643, + 15.655587611021474, + 15.886695711989887, + 16.117036813986488, + 16.34885827207472, + 16.577959350077435, + 16.80902553803753, + 17.034757656045258, + 17.2535431929864, + 17.476620086003095, + 17.69267714302987, + 17.893424522015266, + 18.10097193904221, + 18.303036965080537, + 18.50354452105239, + 18.708693718072027, + 18.89148107706569, + 19.07430228102021, + 19.262206905987114, + 19.442124271998182, + 19.61542708100751 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 221.9148890580982, + "p95": 231.82145808823407, + "max": 304.91177702788264 + } + }, + { + "start": 721993.216830399, + "first": 721994.378631847, + "last": 722011.925665121, + "end": 722011.925760972, + "elapsed_s": 18.708930572960526, + "chunk_offsets_s": [ + 1.1618014479754493, + 1.3329505160218105, + 1.627905466943048, + 1.9329086679499596, + 2.146127423038706, + 2.3550248340470716, + 2.5714244690025225, + 2.783719621016644, + 2.9956639369484037, + 3.215410692966543, + 3.4326812979998067, + 3.6479742700466886, + 3.854489833000116, + 4.059122440987267, + 4.274095779052004, + 4.4928801889764145, + 4.7117563229985535, + 4.9264106650371104, + 5.1495993620483205, + 5.373050577938557, + 5.598737222026102, + 5.822461073054001, + 6.046090718940832, + 6.27153096802067, + 6.492074719979428, + 6.712704875040799, + 6.933423502952792, + 7.150912062032148, + 7.374874830013141, + 7.592284278944135, + 7.809530986938626, + 8.036035859957337, + 8.25877026701346, + 8.481097279000096, + 8.703462564037181, + 8.929019807954319, + 9.150550703983754, + 9.372004728997126, + 9.599605886032805, + 9.82822435197886, + 10.052222250029445, + 10.27811890502926, + 10.502989441039972, + 10.727059148950502, + 10.947989778942429, + 11.168696837965399, + 11.390665811952204, + 11.61287753097713, + 11.83285986399278, + 12.05408059398178, + 12.27441766904667, + 12.497996364952996, + 12.720989319030195, + 12.94653859094251, + 13.168425041018054, + 13.390550008043647, + 13.621395173948258, + 13.848759704036638, + 14.074351578950882, + 14.300678882049397, + 14.520068419980817, + 14.744709193939343, + 14.97506309801247, + 15.199704705970362, + 15.428312315954827, + 15.655761708971113, + 15.88695594004821, + 16.117105814046226, + 16.348909448017366, + 16.577999753993936, + 16.809413745999336, + 17.034715290996246, + 17.253726953989826, + 17.476729050977156, + 17.692760861944407, + 17.89350954897236, + 18.101038477034308, + 18.303219813969918, + 18.50362173502799, + 18.70883472193964 + ], + "chunk_gaps_ms": { + "count": 79, + "p50": 222.21171902492642, + "p95": 231.41399200540036, + "max": 305.0032010069117 + } + }, + { + "start": 721993.218443287, + "first": 721994.123359585, + "last": 722013.481103139, + "end": 722013.481249488, + "elapsed_s": 20.262806201004423, + "chunk_offsets_s": [ + 0.9049162979936227, + 1.0548651210265234, + 1.330880681052804, + 1.6258454960770905, + 1.9309203290613368, + 2.14444786706008, + 2.35295351408422, + 2.569475040072575, + 2.7816984250675887, + 2.9935431140474975, + 3.2134107970632613, + 3.4307012140052393, + 3.645811712020077, + 3.8524859840981662, + 4.057149222004227, + 4.272245322004892, + 4.490769080002792, + 4.710078730015084, + 4.925070736091584, + 5.147888475097716, + 5.370935912011191, + 5.596940548042767, + 5.820582980057225, + 6.044132100068964, + 6.2692868179874495, + 6.489784236997366, + 6.711328471079469, + 6.931416459032334, + 7.148852089070715, + 7.37308196304366, + 7.590567967039533, + 7.807740097050555, + 8.034289029077627, + 8.256921473075636, + 8.479736339999363, + 8.701680047088303, + 8.926456058048643, + 9.148089737049304, + 9.370096808997914, + 9.597650886047632, + 9.826357498997822, + 10.049901347025298, + 10.276059229043312, + 10.500798785011284, + 10.72479182202369, + 10.945639959070832, + 11.166862674988806, + 11.38971823209431, + 11.610852129990235, + 11.830020521068946, + 12.052053373074159, + 12.27306665701326, + 12.496769941993989, + 12.719846087042242, + 12.945450842031278, + 13.167700806050561, + 13.389342567999847, + 13.620035439031199, + 13.847786975093186, + 14.072889066068456, + 14.29948216106277, + 14.518661556998268, + 14.743206473998725, + 14.973876972100697, + 15.19853127899114, + 15.426975676091388, + 15.654654516023584, + 15.885846944060177, + 16.115926642087288, + 16.347673245007172, + 16.576846242067404, + 16.80843889003154, + 17.034202283015475, + 17.25259658007417, + 17.47557106905151, + 17.691587401088327, + 17.89232530107256, + 18.099919434054755, + 18.302230408997275, + 18.502864139038138, + 18.70765538502019, + 18.89054161205422, + 19.073239407036453, + 19.261381235090084, + 19.44109354307875, + 19.614463417092338, + 19.77233338507358, + 19.94132972601801, + 20.10412597109098, + 20.262659852043726 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 221.64176194928586, + "p95": 231.5926479641348, + "max": 305.0748329842463 + } + }, + { + "start": 721993.219273804, + "first": 721994.655508857, + "last": 722013.999175897, + "end": 722013.999410944, + "elapsed_s": 20.780137139954604, + "chunk_offsets_s": [ + 1.436235052999109, + 1.6252707350067794, + 1.9306991710327566, + 2.143436840036884, + 2.3522528409957886, + 2.569249788997695, + 2.781396529986523, + 2.993317735963501, + 3.2130992820020765, + 3.430291736032814, + 3.6456060190685093, + 3.8519784150412306, + 4.0565192750655115, + 4.271614848985337, + 4.490128134028055, + 4.709016112028621, + 4.924296617973596, + 5.147118316963315, + 5.37052750797011, + 5.596214646007866, + 5.820054447976872, + 6.0436138190561906, + 6.268920707050711, + 6.489410871057771, + 6.710607505985536, + 6.930939679965377, + 7.148428059066646, + 7.372387718060054, + 7.5898795740213245, + 7.807137412019074, + 8.033666713978164, + 8.25645796302706, + 8.479190048063174, + 8.701139240991324, + 8.925928949960507, + 9.147643595002592, + 9.369523414061405, + 9.597132003051229, + 9.825739752035588, + 10.04944263806101, + 10.27586415503174, + 10.5004668449983, + 10.724583429982886, + 10.945303617045283, + 11.166204718057998, + 11.388709684018977, + 11.610302338027395, + 11.83015150995925, + 12.051687152008526, + 12.271759241004474, + 12.495730719063431, + 12.719136186991818, + 12.944798785960302, + 13.166675047017634, + 13.38879412796814, + 13.619394866982475, + 13.84692276304122, + 14.072306727059186, + 14.298745442996733, + 14.518057456007227, + 14.742621902027167, + 14.973490001051687, + 15.19801702699624, + 15.426174661028199, + 15.654390120063908, + 15.885128758032806, + 16.115204985020682, + 16.346983470954, + 16.576390256057493, + 16.806881507043727, + 17.032994267065078, + 17.251900225062855, + 17.475159998983145, + 17.691095534013584, + 17.8915788839804, + 18.099256313988008, + 18.301315770018846, + 18.5017585150199, + 18.70690185506828, + 18.890029147965834, + 19.07249498006422, + 19.26060951605905, + 19.440506660030223, + 19.613721671048552, + 19.771557437023148, + 19.94076161296107, + 20.102989522973076, + 20.261955125955865, + 20.405100189032964, + 20.54012064600829, + 20.664750257972628, + 20.779902092996053 + ], + "chunk_gaps_ms": { + "count": 91, + "p50": 221.19663492776453, + "p95": 230.60073901433498, + "max": 305.42843602597713 + } + }, + { + "start": 721993.21637539, + "first": 721993.492601348, + "last": 722014.43062909, + "end": 722014.430970316, + "elapsed_s": 21.21459492610302, + "chunk_offsets_s": [ + 0.276225958019495, + 0.35909568204078823, + 0.5737967290915549, + 0.8011743840761483, + 1.057143675046973, + 1.333195059094578, + 1.6282651320798323, + 1.9332574320724234, + 2.1467810469912365, + 2.3552552650216967, + 2.571939069079235, + 2.7842538880649954, + 2.9960311769973487, + 3.216198708047159, + 3.4332296800566837, + 3.6483850470976904, + 3.855014880071394, + 4.059645710047334, + 4.274578303098679, + 4.493182052043267, + 4.712078734999523, + 4.927225416060537, + 5.14984705299139, + 5.37345929001458, + 5.599160091020167, + 5.822996943024918, + 6.046578650013544, + 6.271617284044623, + 6.492351872031577, + 6.713548332103528, + 6.933806461049244, + 7.151406616088934, + 7.375361889018677, + 7.592807535082102, + 7.810067909071222, + 8.036436176043935, + 8.259429048048332, + 8.481716894079, + 8.704327223007567, + 8.92914023110643, + 9.15065275609959, + 9.372549974010326, + 9.600091468077153, + 9.828982622013427, + 10.05260521301534, + 10.278789923060685, + 10.503585287020542, + 10.727270014001988, + 10.948108643991873, + 11.169184930040501, + 11.391259261057712, + 11.613157444051467, + 11.832660268060863, + 12.054428784991615, + 12.274922226089984, + 12.498408733052202, + 12.722003924078308, + 12.94761043402832, + 13.1698056680616, + 13.391663091024384, + 13.62206884403713, + 13.84978349506855, + 14.075238104094751, + 14.301596614997834, + 14.52092330600135, + 14.745701659005135, + 14.976141178049147, + 15.200685164076276, + 15.428592574084178, + 15.657222300069407, + 15.887983277090825, + 16.117949604056776, + 16.349846412078477, + 16.579113428015262, + 16.810206505004317, + 17.036221655085683, + 17.255011618020944, + 17.477938503026962, + 17.693897671997547, + 17.89459178200923, + 18.1024037420284, + 18.30415353900753, + 18.50447137001902, + 18.710057454067282, + 18.892726188059896, + 19.075597479008138, + 19.263405097066425, + 19.443439309019595, + 19.6167332760524, + 19.774489894043654, + 19.94360801205039, + 20.106233820086345, + 20.264821777003817, + 20.40793044900056, + 20.543054510024376, + 20.667598793050274, + 20.782924846047536, + 20.875870282994583, + 20.943714414024726, + 21.00049869704526, + 21.05024382204283, + 21.10188619699329, + 21.16093654499855, + 21.214253700105473 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 220.8386299898848, + "p95": 231.09307698905468, + "max": 304.99229999259114 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 35.2, + "per_stream_tps": 35.2, + "ttft_s_max": 0.16, + "timing_valid": true, + "end_to_end_tps": 34.53776751939664, + "makespan_s": 7.412175667006522, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 35.17402127971944, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 7.412175667006522, + "end_to_end_tps": 34.53776751939664, + "timing_valid": true, + "per_stream_tps": [ + 35.2 + ], + "ttft_s": [ + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + } + ], + "client_intervals": [ + { + "start": 722164.061137688, + "first": 722164.223340618, + "last": 722171.473009286, + "end": 722171.473313355, + "elapsed_s": 7.412175667006522, + "chunk_offsets_s": [ + 0.1622029299614951, + 0.22403310297522694, + 0.2834871959639713, + 0.3358268029987812, + 0.3901983139803633, + 0.44795292301569134, + 0.49922376999165863, + 0.5506385449552909, + 0.6021007880335674, + 0.6534089719643816, + 0.703431032015942, + 0.7545203289482743, + 0.8062945520505309, + 0.8579266909509897, + 0.9099837499670684, + 0.9602857769932598, + 1.0108174809720367, + 1.061495981994085, + 1.1124910310609266, + 1.1631920549552888, + 1.2135370429605246, + 1.2647547960514203, + 1.3171755250077695, + 1.3703487690072507, + 1.4299722700379789, + 1.485439946060069, + 1.5368838020367548, + 1.5905948389554396, + 1.6473928169580176, + 1.704368200036697, + 1.763897237018682, + 1.8223242700332776, + 1.882329848012887, + 1.9462785490322858, + 1.9988031210377812, + 2.056512295966968, + 2.1154184360057116, + 2.173393805976957, + 2.231657322961837, + 2.2849398349644616, + 2.336741546052508, + 2.3879425190389156, + 2.4396276669576764, + 2.4896848029457033, + 2.542254948988557, + 2.5926733169471845, + 2.646989817963913, + 2.7062098450260237, + 2.7647054539993405, + 2.820493923034519, + 2.8722901040455326, + 2.9257516290526837, + 2.9847755240043625, + 3.0434596829582006, + 3.0962935240240768, + 3.1478037140332162, + 3.200432767975144, + 3.2588846819708124, + 3.317042652051896, + 3.369143047952093, + 3.4210280580446124, + 3.47320921998471, + 3.5235723230289295, + 3.574112761998549, + 3.6251837969757617, + 3.6759772249497473, + 3.7267990220570937, + 3.7778492840006948, + 3.828624424058944, + 3.879852904006839, + 3.9311477560549974, + 3.982352497987449, + 4.033993399003521, + 4.084703271975741, + 4.136355569004081, + 4.188875727006234, + 4.246761395945214, + 4.305503471987322, + 4.365810306044295, + 4.425902599003166, + 4.48708013095893, + 4.539081266964786, + 4.590646974975243, + 4.6414088510209695, + 4.6925168200396, + 4.743683625012636, + 4.7951924999943, + 4.845511013059877, + 4.8962646999862045, + 4.94945784797892, + 5.0012277429923415, + 5.052792840986513, + 5.104797199019231, + 5.154848907026462, + 5.205901841982268, + 5.256533905980177, + 5.308290187967941, + 5.359031087020412, + 5.412331399973482, + 5.472447257023305, + 5.53339350502938, + 5.595014752005227, + 5.6560917750466615, + 5.721317935036495, + 5.785184656037018, + 5.837762066046707, + 5.89756011904683, + 5.955995031981729, + 6.0142576929647475, + 6.06446721998509, + 6.115830376045778, + 6.166720399982296, + 6.2177348310360685, + 6.2707292980048805, + 6.323112262994982, + 6.3747627639677376, + 6.426006551948376, + 6.478825267055072, + 6.536429080995731, + 6.587486620992422, + 6.6384746100520715, + 6.689978173002601, + 6.741751556983218, + 6.7938547800295055, + 6.845442075980827, + 6.896682868013158, + 6.948908093036152, + 6.9987029960611835, + 7.051014294964261, + 7.1023096380522475, + 7.153798616956919, + 7.211626147036441, + 7.2621399640338495, + 7.314685698947869, + 7.3650410959962755, + 7.411871597985737 + ], + "chunk_gaps_ms": { + "count": 135, + "p50": 51.80171108804643, + "p95": 61.077023041434586, + "max": 65.22615998983383 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 52.9, + "per_stream_tps": 27.6, + "ttft_s_max": 0.38, + "timing_valid": true, + "end_to_end_tps": 52.27405056794766, + "makespan_s": 9.794534657965414, + "steady_tps": 55.1, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 52.946368114630914, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 9.794534657965414, + "end_to_end_tps": 52.27405056794766, + "timing_valid": true, + "steady_tps": 55.059713549164606, + "per_stream_tps": [ + 27.1, + 28.1 + ], + "ttft_s": [ + 0.38, + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + } + ], + "client_intervals": [ + { + "start": 722171.474992025, + "first": 722171.856161952, + "last": 722181.269213915, + "end": 722181.269405648, + "elapsed_s": 9.794413623050787, + "chunk_offsets_s": [ + 0.3811699270736426, + 0.4654034440172836, + 0.5603622751077637, + 0.6553167150123045, + 0.7489976061042398, + 0.8419131250120699, + 0.9343844790710136, + 1.0285854870453477, + 1.1232231960166246, + 1.2164173410274088, + 1.3118845440912992, + 1.405065869097598, + 1.496134392102249, + 1.5907816670369357, + 1.686927852104418, + 1.7824255489977077, + 1.8831899940269068, + 1.9800468280445784, + 2.078088831040077, + 2.174156189081259, + 2.2718000480672345, + 2.3672663930337876, + 2.4646730300737545, + 2.5599137850804254, + 2.6548856230219826, + 2.752316743019037, + 2.845800831099041, + 2.9408634490100667, + 3.033651558100246, + 3.1282511280151084, + 3.2212010960793123, + 3.3201139310840517, + 3.417364057037048, + 3.515123679069802, + 3.6099866420263425, + 3.70679280301556, + 3.8016883600503206, + 3.8989331980701536, + 3.9949163290439174, + 4.094098991015926, + 4.192294924054295, + 4.286290677031502, + 4.384658892056905, + 4.4813203810481355, + 4.579590831999667, + 4.675913030048832, + 4.772139049018733, + 4.867434837040491, + 4.960832998040132, + 5.05518862104509, + 5.1484298560535535, + 5.238075326080434, + 5.328924532048404, + 5.424802437075414, + 5.519407891086303, + 5.613792736083269, + 5.710953618050553, + 5.8062739400193095, + 5.899713257094845, + 5.993817721027881, + 6.092097894055769, + 6.187665861099958, + 6.281871990067884, + 6.372857301030308, + 6.464474596083164, + 6.559909483068623, + 6.655689174076542, + 6.752487165038474, + 6.849806723068468, + 6.946956312051043, + 7.040064993081614, + 7.134246943052858, + 7.229801376000978, + 7.3272278000367805, + 7.426120440009981, + 7.522553438087925, + 7.616952694021165, + 7.711320487083867, + 7.80673607008066, + 7.9036753920372576, + 7.9974566450109705, + 8.092669838108122, + 8.185625183046795, + 8.280169854057021, + 8.374602616997436, + 8.471794702112675, + 8.570086781051941, + 8.66443462704774, + 8.75433943909593, + 8.849472909001634, + 8.944045094074681, + 9.039572327048518, + 9.132750319084153, + 9.222741147037596, + 9.284341117017902, + 9.335642989026383, + 9.387206490035169, + 9.438460723031312, + 9.489967758068815, + 9.54239071009215, + 9.594719713088125, + 9.645769520080648, + 9.697203592048027, + 9.74762866506353, + 9.794221890042536 + ], + "chunk_gaps_ms": { + "count": 104, + "p50": 94.8792599956505, + "p95": 98.2920789392665, + "max": 100.76444502919912 + } + }, + { + "start": 722171.47487099, + "first": 722171.636825176, + "last": 722180.697370436, + "end": 722180.697483077, + "elapsed_s": 9.222612086916342, + "chunk_offsets_s": [ + 0.1619541859254241, + 0.2229415850015357, + 0.46508367091882974, + 0.5601490659173578, + 0.6550840679556131, + 0.7488296999363229, + 0.8416508999653161, + 0.9342036709422246, + 1.0284338429337367, + 1.123026364017278, + 1.2162409229204059, + 1.311712099937722, + 1.4047987889498472, + 1.4958627129672095, + 1.5906397009966895, + 1.6866189009742811, + 1.7822358639677987, + 1.8835071519715711, + 1.9799390529515222, + 2.0776987599674612, + 2.173843269003555, + 2.271570928976871, + 2.367068743915297, + 2.4645732659846544, + 2.5597799719544128, + 2.6545974910259247, + 2.751992894918658, + 2.84560659294948, + 2.9405556439887732, + 3.033576974994503, + 3.1279188089538366, + 3.221165783936158, + 3.319753171992488, + 3.4169832699699327, + 3.514937932952307, + 3.6096785329282284, + 3.7065600469941273, + 3.8013601469574496, + 3.8992027540225536, + 3.994867565925233, + 4.0939820549683645, + 4.192107637994923, + 4.286007196991704, + 4.384411139995791, + 4.481212963000871, + 4.579486185917631, + 4.675844864919782, + 4.771826633950695, + 4.867229892988689, + 4.96072622993961, + 5.0551135269925, + 5.148278838954866, + 5.237756295944564, + 5.328870169003494, + 5.424439797992818, + 5.519147738930769, + 5.613571258960292, + 5.710707066929899, + 5.806044754921459, + 5.8995016779517755, + 5.994204404996708, + 6.09188943891786, + 6.187561137950979, + 6.281650318996981, + 6.372578421025537, + 6.464448660961352, + 6.559735631919466, + 6.655374104971997, + 6.7521783750271425, + 6.849665708024986, + 6.946815922972746, + 7.039707200950943, + 7.13385606999509, + 7.229696693946607, + 7.327028954983689, + 7.4257543969433755, + 7.522289039916359, + 7.616697171004489, + 7.711065941955894, + 7.806624284945428, + 7.903622973011807, + 7.997436327976175, + 8.093044757959433, + 8.185522880987264, + 8.280017267912626, + 8.37449192092754, + 8.471775538986549, + 8.569897383917123, + 8.664345027995296, + 8.754052806994878, + 8.84910544694867, + 8.943810888915323, + 9.03942957194522, + 9.132644181954674, + 9.22249944601208 + ], + "chunk_gaps_ms": { + "count": 94, + "p50": 95.1360504841432, + "p95": 98.58738805633038, + "max": 242.14208591729403 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.3, + "per_stream_tps": 18.2, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 68.85426849805982, + "makespan_s": 14.871990108047612, + "steady_tps": 74.2, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.34813259317619, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 14.871990108047612, + "end_to_end_tps": 68.85426849805982, + "timing_valid": true, + "steady_tps": 74.1967611540567, + "per_stream_tps": [ + 17.7, + 19.3, + 17.9, + 18.6 + ], + "ttft_s": [ + 0.16, + 0.39, + 0.63, + 0.89 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0a24c104609f", + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "28cedcb814fc", + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767" + } + ], + "client_intervals": [ + { + "start": 722181.271226071, + "first": 722181.434203693, + "last": 722195.824987997, + "end": 722195.825107408, + "elapsed_s": 14.553881336934865, + "chunk_offsets_s": [ + 0.1629776218906045, + 0.22624916792847216, + 0.4681934709660709, + 0.7355282559292391, + 1.0175019289599732, + 1.1440484719350934, + 1.2720865699229762, + 1.3915240779751912, + 1.5135695909848437, + 1.640075993956998, + 1.7686232149135321, + 1.9000980309210718, + 2.0307955750031397, + 2.1685618369374424, + 2.302329445956275, + 2.4418198689818382, + 2.5840971749275923, + 2.726908571901731, + 2.865471887984313, + 3.005800515995361, + 3.1447003529174253, + 3.2864416879601777, + 3.426377174910158, + 3.569148315000348, + 3.709251089952886, + 3.856517071952112, + 3.9979701539268717, + 4.137562950956635, + 4.281485402956605, + 4.423287870944478, + 4.5640025599859655, + 4.714521014946513, + 4.855558241950348, + 4.994524277979508, + 5.135596714913845, + 5.278829312999733, + 5.4171665189787745, + 5.560079680988565, + 5.702650183928199, + 5.84575298090931, + 5.989558534929529, + 6.126551936962642, + 6.269538455992006, + 6.413638549973257, + 6.56103236193303, + 6.7046794350026175, + 6.8512543889228255, + 6.995519208954647, + 7.13877950690221, + 7.2818261929787695, + 7.424221504945308, + 7.562158586923033, + 7.701578159001656, + 7.842178450897336, + 7.982243726961315, + 8.123372884932905, + 8.266166432993487, + 8.415095493895933, + 8.554388155927882, + 8.69903038290795, + 8.847535675973631, + 8.988638598937541, + 9.125696195987985, + 9.266659112996422, + 9.405325337895192, + 9.54476535296999, + 9.688657883903943, + 9.836187217966653, + 9.97468524100259, + 10.115462913992815, + 10.261137259891257, + 10.4050119969761, + 10.544304875889793, + 10.684336551930755, + 10.827245495980605, + 10.96729132591281, + 11.107984807924367, + 11.24680894089397, + 11.387765933992341, + 11.53024159988854, + 11.66816050792113, + 11.80536877294071, + 11.946283589000814, + 12.0854694849113, + 12.222886247909628, + 12.365473885904066, + 12.505120709887706, + 12.645226275897585, + 12.783941509900615, + 12.92506144696381, + 13.066264565917663, + 13.207449529902078, + 13.344830585992895, + 13.479892275994644, + 13.612189943902194, + 13.728543841978535, + 13.846225720946677, + 13.963062158902176, + 14.080157860997133, + 14.202664555981755, + 14.32792739290744, + 14.446375645929947, + 14.553761925897561 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 140.46445995336398, + "p95": 148.50529306568205, + "max": 281.97367303073406 + } + }, + { + "start": 722181.271525844, + "first": 722181.656570868, + "last": 722194.882902349, + "end": 722194.883448518, + "elapsed_s": 13.61192267399747, + "chunk_offsets_s": [ + 0.38504502398427576, + 0.46776720602065325, + 0.7347670750459656, + 1.0171391130425036, + 1.1434569290140644, + 1.2716487919678912, + 1.3911095260409638, + 1.5130983690032735, + 1.639538750052452, + 1.7682482090312988, + 1.8997367840493098, + 2.030900895013474, + 2.1681566749466583, + 2.301929712994024, + 2.4412764509906992, + 2.584208645974286, + 2.726426161942072, + 2.864979546982795, + 3.005332494038157, + 3.143903456046246, + 3.28599911602214, + 3.425859984010458, + 3.569007068988867, + 3.708751467987895, + 3.8558557990472764, + 3.997463844018057, + 4.137059428030625, + 4.280945796053857, + 4.423144857981242, + 4.563203005003743, + 4.713834368973039, + 4.855125474976376, + 4.994053559959866, + 5.134895636001602, + 5.278157541994005, + 5.4170450899982825, + 5.5595302489819005, + 5.702538780984469, + 5.845234088948928, + 5.9894044829998165, + 6.126002992969006, + 6.268914308049716, + 6.412922909948975, + 6.560587689047679, + 6.703999363002367, + 6.850629408028908, + 6.994939828990027, + 7.138183793053031, + 7.281287841964513, + 7.424019898986444, + 7.561708937981166, + 7.701171891996637, + 7.841670542024076, + 7.981746467994526, + 8.12275514495559, + 8.2656492900569, + 8.414527141023427, + 8.55469695001375, + 8.698427935945801, + 8.846961717004888, + 8.9881376309786, + 9.125298806000501, + 9.266050574020483, + 9.404818819020875, + 9.544255406945013, + 9.687917363946326, + 9.83546147798188, + 9.97420888999477, + 10.114995631971397, + 10.2604522210313, + 10.404572337050922, + 10.543829923961312, + 10.684199323994108, + 10.827285017003305, + 10.966653140028939, + 11.108070852002129, + 11.246386529994197, + 11.38711819704622, + 11.529664220986888, + 11.667748296982609, + 11.804906810983084, + 11.945751316961832, + 12.084994124947116, + 12.222815510001965, + 12.365072148968466, + 12.50471386499703, + 12.644743966986425, + 12.783936116029508, + 12.924663232057355, + 13.065892260987312, + 13.20695430401247, + 13.344346937024966, + 13.479532571975142, + 13.61137650499586 + ], + "chunk_gaps_ms": { + "count": 93, + "p50": 140.7316670520231, + "p95": 148.53378105908632, + "max": 282.372037996538 + } + }, + { + "start": 722181.271405214, + "first": 722181.897411112, + "last": 722196.142602634, + "end": 722196.143029107, + "elapsed_s": 14.871623893035576, + "chunk_offsets_s": [ + 0.6260058979969472, + 0.7351545609999448, + 1.0174921520520002, + 1.144121248042211, + 1.271948351059109, + 1.3913859870517626, + 1.513484661001712, + 1.6399423399707302, + 1.7684867900097743, + 1.8999600689858198, + 2.030895826057531, + 2.16845568199642, + 2.30219734297134, + 2.4414554940303788, + 2.584004473988898, + 2.7269208930665627, + 2.865387515979819, + 3.0057383390376344, + 3.1442834669724107, + 3.286189767997712, + 3.4261545330518857, + 3.569070842000656, + 3.709027133998461, + 3.8563733560731634, + 3.997843845980242, + 4.137336659012362, + 4.281415070057847, + 4.422863848973066, + 4.563439266989008, + 4.714080305071548, + 4.855417568003759, + 4.993872706079856, + 5.135215654037893, + 5.278424262069166, + 5.4173120660707355, + 5.559940448030829, + 5.702516091987491, + 5.84514389699325, + 5.989161971025169, + 6.126549722976051, + 6.269287072005682, + 6.4133710380410776, + 6.560891653993167, + 6.704368027043529, + 6.850920321070589, + 6.995273735024966, + 7.138818236067891, + 7.281592954066582, + 7.4242246829671785, + 7.561932491022162, + 7.701461407006718, + 7.841947230044752, + 7.982030394021422, + 8.123283418011852, + 8.26595334103331, + 8.414998517022468, + 8.554291083011776, + 8.698790020076558, + 8.847385918023065, + 8.988680316018872, + 9.12555527605582, + 9.266395227983594, + 9.405038587050512, + 9.544534472050145, + 9.688313549035229, + 9.835916127078235, + 9.97457103105262, + 10.115248639020137, + 10.260857605026104, + 10.404884607996792, + 10.544081234023906, + 10.684089194983244, + 10.827260027057491, + 10.967074249056168, + 11.107755962992087, + 11.246682350989431, + 11.387547063990496, + 11.530025709071197, + 11.667941958992742, + 11.805413210066035, + 11.946138952043839, + 12.085253225988708, + 12.223006208078004, + 12.365255450014956, + 12.504988114000298, + 12.645002298988402, + 12.78415246505756, + 12.924976518028416, + 13.066122857970186, + 13.207236682996154, + 13.344617114053108, + 13.479748230078258, + 13.611963102011941, + 13.728329329984263, + 13.84600699704606, + 13.962845357018523, + 14.080063785077073, + 14.202447002055123, + 14.327791191986762, + 14.446158732986078, + 14.553542227018625, + 14.635561870993115, + 14.702002414036542, + 14.762197601026855, + 14.82109746907372, + 14.871197420055978 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 140.00796095933765, + "p95": 147.52061595208943, + "max": 282.33759105205536 + } + }, + { + "start": 722181.271038999, + "first": 722182.165514944, + "last": 722195.906655396, + "end": 722195.906727671, + "elapsed_s": 14.635688672075048, + "chunk_offsets_s": [ + 0.894475945038721, + 1.0177587560610846, + 1.144032348995097, + 1.2724809040082619, + 1.3917935910867527, + 1.5138958360766992, + 1.640349559020251, + 1.768894073087722, + 1.9003613130189478, + 2.031036750995554, + 2.168863379978575, + 2.3026011820184067, + 2.4416621190030128, + 2.5845850399928167, + 2.7272132070502266, + 2.8658276150235906, + 3.0061887659830973, + 3.1446937100263312, + 3.286588191986084, + 3.426603375002742, + 3.5694050890160725, + 3.709513709996827, + 3.8566061890451238, + 3.998282772023231, + 4.137826321995817, + 4.281849863007665, + 4.423271832987666, + 4.563979219063185, + 4.714410437038168, + 4.855973344063386, + 4.994395517045632, + 5.135550816077739, + 5.278822795022279, + 5.417647116002627, + 5.560099684051238, + 5.7027655250858516, + 5.8455730030545965, + 5.9901290400885046, + 6.1266344339819625, + 6.269689014996402, + 6.413887678063475, + 6.561171342036687, + 6.704773334087804, + 6.851349361008033, + 6.995675261016004, + 7.139236657065339, + 7.282055510091595, + 7.424157973029651, + 7.562460913090035, + 7.701858161017299, + 7.842251566005871, + 7.982460329076275, + 8.123679867014289, + 8.26638292206917, + 8.415428915061057, + 8.554972222074866, + 8.6994293309981, + 8.847666107001714, + 8.989101201994345, + 9.125973982037976, + 9.26680970704183, + 9.405457677086815, + 9.544863839983009, + 9.688792176079005, + 9.836197872995399, + 9.97483830107376, + 10.115579898003489, + 10.261394805042073, + 10.40515684406273, + 10.544532765052281, + 10.684600117034279, + 10.827361550997011, + 10.967196093988605, + 11.108431808068417, + 11.24667073902674, + 11.387768912012689, + 11.530297819059342, + 11.668045860016719, + 11.805727225029841, + 11.946415401995182, + 12.085709540056996, + 12.223014252027497, + 12.365404095035046, + 12.505037388997152, + 12.64508108398877, + 12.784605305059813, + 12.924993369029835, + 13.066180996014737, + 13.207509086001664, + 13.344919939991087, + 13.479833811055869, + 13.61207002599258, + 13.728538279072382, + 13.846305835992098, + 13.963068925077096, + 14.080177281051874, + 14.202641642070375, + 14.327831931994297, + 14.446362067013979, + 14.553766007069498, + 14.635616397019476 + ], + "chunk_gaps_ms": { + "count": 100, + "p50": 140.1586920255795, + "p95": 147.0924790482968, + "max": 150.43121797498316 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 85.7, + "per_stream_tps": 11.4, + "ttft_s_max": 2.13, + "timing_valid": true, + "end_to_end_tps": 85.48382361196853, + "makespan_s": 23.95774912100751, + "steady_tps": 91.2, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 85.72809574698789, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 23.95774912100751, + "end_to_end_tps": 85.48382361196853, + "timing_valid": true, + "steady_tps": 91.23306120595402, + "per_stream_tps": [ + 11.8, + 12.5, + 11.3, + 11.8, + 10.7, + 11.3, + 11.0, + 11.4 + ], + "ttft_s": [ + 2.13, + 1.48, + 1.17, + 1.79, + 0.16, + 0.89, + 0.62, + 0.38 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0a24c104609f", + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "28cedcb814fc", + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "22e8f3f09dfa", + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c28174794bf0", + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c6d6ec2a35a8", + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "1a4e0d70467b", + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66" + } + ], + "client_intervals": [ + { + "start": 722196.147944465, + "first": 722198.275042011, + "last": 722219.935091187, + "end": 722219.93527345, + "elapsed_s": 23.78732898493763, + "chunk_offsets_s": [ + 2.127097546006553, + 2.325976168969646, + 2.526692971936427, + 2.72641116194427, + 2.930872383993119, + 3.134699415997602, + 3.344534298987128, + 3.554374143946916, + 3.770921633928083, + 3.9828945519402623, + 4.195462316973135, + 4.402082131942734, + 4.617887932923622, + 4.837579584913328, + 5.058076989953406, + 5.26632943097502, + 5.4898341979132965, + 5.704804168897681, + 5.923344837967306, + 6.1432333489647135, + 6.360005100956187, + 6.576911342912354, + 6.794707976980135, + 7.011163939954713, + 7.23010028200224, + 7.445560274994932, + 7.66754894296173, + 7.879807912977412, + 8.09918977599591, + 8.318619715981185, + 8.53472526895348, + 8.756234899978153, + 8.973236431949772, + 9.193278846913017, + 9.417559732915834, + 9.640853778924793, + 9.866928256000392, + 10.093121832003817, + 10.317732211900875, + 10.540308755007572, + 10.759712150902487, + 10.982003357959911, + 11.20306389790494, + 11.426942457910627, + 11.647678992943838, + 11.870008118916303, + 12.089395118993707, + 12.306949082994834, + 12.529346668976359, + 12.744038775912486, + 12.962435091962107, + 13.180763488984667, + 13.396964101935737, + 13.621085256920196, + 13.841158911003731, + 14.05857966397889, + 14.274050158914179, + 14.495772057911381, + 14.715698410989717, + 14.931206689914688, + 15.14533488790039, + 15.358901316998526, + 15.580114234006032, + 15.800891088903882, + 16.021287102950737, + 16.244997303932905, + 16.46953125100117, + 16.686579838977195, + 16.907301510917023, + 17.13138648797758, + 17.352632828988135, + 17.576182477991097, + 17.790733807953075, + 18.010339792002924, + 18.22638140199706, + 18.44193341897335, + 18.654836768982932, + 18.869481066940352, + 19.091156040900387, + 19.308941026916727, + 19.52248018491082, + 19.741602531983517, + 19.955831626895815, + 20.171106347930618, + 20.392312011914328, + 20.611145552946255, + 20.835298188962042, + 21.05292073194869, + 21.271123558981344, + 21.488229519920424, + 21.710990348947234, + 21.933252040995285, + 22.13345320499502, + 22.329743340960704, + 22.52395212301053, + 22.720466441009194, + 22.908444080967456, + 23.09658002690412, + 23.27036413096357, + 23.43546434689779, + 23.57186430389993, + 23.696418172912672, + 23.787146721966565 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 217.70376450149342, + "p95": 224.15263601578772, + "max": 226.19357600342482 + } + }, + { + "start": 722196.145354021, + "first": 722197.62557059, + "last": 722218.079973247, + "end": 722218.080559532, + "elapsed_s": 21.9352055110503, + "chunk_offsets_s": [ + 1.4802165690343827, + 1.6382225439883769, + 1.9720027570147067, + 2.3278819740517065, + 2.529138316051103, + 2.728631726000458, + 2.9341512930113822, + 3.1380858160555363, + 3.3461577870184556, + 3.556860813056119, + 3.773232692037709, + 3.9857152639888227, + 4.19721950404346, + 4.405156475957483, + 4.620593277039006, + 4.839592154021375, + 5.060268752044067, + 5.2687220829539, + 5.492327113985084, + 5.707340453984216, + 5.925839436007664, + 6.146171023021452, + 6.3629477539798245, + 6.580314024002291, + 6.7974179739831015, + 7.013803915004246, + 7.233031642041169, + 7.447732546017505, + 7.6697732410393655, + 7.882346600992605, + 8.1017413190566, + 8.321404146961868, + 8.537046486977488, + 8.758783322991803, + 8.975871923030354, + 9.195979216950946, + 9.420098470989615, + 9.643233831040561, + 9.869594032992609, + 10.095819688984193, + 10.321028416976333, + 10.543179061030969, + 10.761989710037597, + 10.984313635970466, + 11.205383336986415, + 11.429333545966074, + 11.649542893981561, + 11.872179585974663, + 12.092329401988536, + 12.309084156062454, + 12.532414398039691, + 12.74614737299271, + 12.96524099202361, + 13.183440434979275, + 13.399688092060387, + 13.62360275699757, + 13.844051501015201, + 14.060624417033978, + 14.27678549499251, + 14.498681194032542, + 14.719028649968095, + 14.933609536034055, + 15.147835872019641, + 15.36155489005614, + 15.583050705026835, + 15.803825190057978, + 16.02346466097515, + 16.247518121032044, + 16.471962337032892, + 16.688966699992307, + 16.91035927203484, + 17.13380640395917, + 17.355170661001466, + 17.5789798629703, + 17.79329251195304, + 18.013495167018846, + 18.22904601600021, + 18.444557458977215, + 18.657943575060926, + 18.872350619989447, + 19.093839799053967, + 19.312031036010012, + 19.525712099974044, + 19.744470993056893, + 19.957742250990123, + 20.17418287496548, + 20.39430619205814, + 20.61443485505879, + 20.838210357003845, + 21.055797992041335, + 21.273862499045208, + 21.491580946021713, + 21.713404032052495, + 21.93461922602728 + ], + "chunk_gaps_ms": { + "count": 93, + "p50": 218.8106490066275, + "p95": 225.20872799213976, + "max": 355.87921703699976 + } + }, + { + "start": 722196.145277168, + "first": 722197.315113486, + "last": 722219.93515383, + "end": 722219.935738838, + "elapsed_s": 23.790461670025252, + "chunk_offsets_s": [ + 1.169836318003945, + 1.3206460759975016, + 1.638646662933752, + 1.971995571977459, + 2.328736644005403, + 2.529714106931351, + 2.729821534943767, + 2.9344079069560394, + 3.1379683749983087, + 3.347303000977263, + 3.556863892939873, + 3.7726242810022086, + 3.9860341999446973, + 4.19786481896881, + 4.405174718936905, + 4.620989165036008, + 4.840129676973447, + 5.060256639961153, + 5.268838717951439, + 5.492271892027929, + 5.707288097939454, + 5.925875289016403, + 6.145857085939497, + 6.362919957959093, + 6.579772794037126, + 6.797326894942671, + 7.013735568034463, + 7.232421987922862, + 7.447437569964677, + 7.66954040993005, + 7.882285892963409, + 8.101419304963201, + 8.321024831035174, + 8.537047076970339, + 8.75838815793395, + 8.975796489976346, + 9.195793619961478, + 9.41991410695482, + 9.643256224924698, + 9.869409556966275, + 10.095452114008367, + 10.320854926947504, + 10.542820793925785, + 10.761420973925851, + 10.98425557592418, + 11.205251117935404, + 11.429258057032712, + 11.650016937986948, + 11.872182547929697, + 12.091829892946407, + 12.309109081979841, + 12.531530854990706, + 12.745960157015361, + 12.9647817669902, + 13.183137075975537, + 13.399275021976791, + 13.62339467194397, + 13.843396806973033, + 14.061108949943446, + 14.276159111992456, + 14.498015546938404, + 14.718102094018832, + 14.934022784931585, + 15.147270387969911, + 15.361242259037681, + 15.582681827014312, + 15.803343098959886, + 16.023212485946715, + 16.247112466953695, + 16.47250631498173, + 16.6886931930203, + 16.910030863946304, + 17.133251645951532, + 17.354930868023075, + 17.579318172996864, + 17.793593768030405, + 18.013664470985532, + 18.228886511991732, + 18.444017064990476, + 18.65713719802443, + 18.871802847948857, + 19.093472987995483, + 19.312245920998976, + 19.525253455038182, + 19.74437961098738, + 19.958785422961228, + 20.174310000962578, + 20.39455635100603, + 20.61395214498043, + 20.838160207029432, + 21.055645663989708, + 21.274055549991317, + 21.491035518003628, + 21.712776017026044, + 21.93558278097771, + 22.136268371948972, + 22.332746022031642, + 22.52708964701742, + 22.72348158992827, + 22.911336903925985, + 23.099601353984326, + 23.272410376928747, + 23.4377490329789, + 23.57437653001398, + 23.699149856925942, + 23.789876661961898 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 218.35530898533762, + "p95": 225.40281293913722, + "max": 356.741072027944 + } + }, + { + "start": 722196.147079815, + "first": 722197.940187201, + "last": 722219.584196184, + "end": 722219.584321182, + "elapsed_s": 23.437241366948, + "chunk_offsets_s": [ + 1.7931073859799653, + 1.9705859799869359, + 2.3260817979462445, + 2.5273753949441016, + 2.727384280995466, + 2.932195689994842, + 3.135891209007241, + 3.3444796059047803, + 3.5545325889252126, + 3.7704177879495546, + 3.9841723329154775, + 4.1956022379454225, + 4.403220686945133, + 4.618811690015718, + 4.837821276974864, + 5.0588085380150005, + 5.26736436993815, + 5.491052632918581, + 5.705911563942209, + 5.924431453924626, + 6.143976715975441, + 6.360795117914677, + 6.578303225920536, + 6.796041011926718, + 7.012383919907734, + 7.231349456007592, + 7.446525987004861, + 7.66858285095077, + 7.881078777951188, + 8.100630714907311, + 8.319796253927052, + 8.535936223925091, + 8.757222796906717, + 8.974687605979852, + 9.194421281921677, + 9.418678220012225, + 9.6419712259667, + 9.868120635976084, + 10.094152371981181, + 10.318800800014287, + 10.541819058009423, + 10.760778352967463, + 10.982913821004331, + 11.203859059954993, + 11.42787008092273, + 11.648832303937525, + 11.871034570969641, + 12.090792478993535, + 12.307884948910214, + 12.530534807010554, + 12.744949910906143, + 12.963890308979899, + 13.181780908955261, + 13.398283203947358, + 13.622045403928496, + 13.841518190922216, + 14.059553528903052, + 14.274408678989857, + 14.496327154920436, + 14.716937023913488, + 14.931439317995682, + 15.145656984997913, + 15.359481516992673, + 15.580501844990067, + 15.801438923925161, + 16.021628310903907, + 16.24562999198679, + 16.470574961975217, + 16.686819622991607, + 16.90881852596067, + 17.131864689989015, + 17.353885639924556, + 17.577587088919245, + 17.791087738936767, + 18.0106565579772, + 18.2270185619127, + 18.443079498014413, + 18.655961640994065, + 18.871099893003702, + 19.091509178979322, + 19.31041169492528, + 19.52384004695341, + 19.742841311963275, + 19.956183522939682, + 20.17172894894611, + 20.39309509890154, + 20.612240951973945, + 20.83726920990739, + 21.054730291012675, + 21.27258252899628, + 21.48989492491819, + 21.711035485961474, + 21.934176837909035, + 22.134545093984343, + 22.330404390930198, + 22.524999934015796, + 22.72172628599219, + 22.90976328891702, + 23.09714525495656, + 23.270795434946194, + 23.437116368906572 + ], + "chunk_gaps_ms": { + "count": 100, + "p50": 217.96296897809952, + "p95": 224.64842803310603, + "max": 355.4958179593086 + } + }, + { + "start": 722196.144863136, + "first": 722196.305908401, + "last": 722220.102074433, + "end": 722220.102535131, + "elapsed_s": 23.957671995041892, + "chunk_offsets_s": [ + 0.16104526503477246, + 0.21958679205272347, + 0.4585970710031688, + 0.7294267000397667, + 1.0124916039640084, + 1.3205078759929165, + 1.6381265199743211, + 1.9728483529761434, + 2.328348236042075, + 2.530002524028532, + 2.729800909059122, + 2.9343009550357237, + 3.1379299079999328, + 3.3468574070138857, + 3.5569221889600158, + 3.772859291988425, + 3.986132488003932, + 4.1979012209922075, + 4.4053737439680845, + 4.621315418044105, + 4.840401818044484, + 5.061251076986082, + 5.268985494971275, + 5.493154269992374, + 5.708069329964928, + 5.926593477954157, + 6.146021028980613, + 6.362953893025406, + 6.579899693024345, + 6.7980506010353565, + 7.014505218947306, + 7.233288947958499, + 7.448280996992253, + 7.670160984038375, + 7.883087138994597, + 8.102549412054941, + 8.321956134983338, + 8.537935512023978, + 8.759386776946485, + 8.976616921951063, + 9.19659144198522, + 9.421112495008856, + 9.64386560197454, + 9.870215703034773, + 10.096022190991789, + 10.321327690966427, + 10.543926708982326, + 10.762682163040154, + 10.984900615061633, + 11.205982399987988, + 11.430068703950383, + 11.65032544999849, + 11.872873891959898, + 12.092437710962258, + 12.309653779957443, + 12.532518662977964, + 12.747218196978793, + 12.965694084996358, + 13.18381049996242, + 13.399971976061352, + 13.623891780036502, + 13.844164159032516, + 14.060780278057791, + 14.27700351504609, + 14.498737707035616, + 14.71901980706025, + 14.933859713026322, + 15.14827837003395, + 15.361939604976214, + 15.5829947469756, + 15.803915163036436, + 16.02404161298182, + 16.247640687040985, + 16.472501865006052, + 16.68919146596454, + 16.910804650979117, + 17.13386157504283, + 17.355903011979535, + 17.579040218959562, + 17.79345071304124, + 18.01382111106068, + 18.229496759013273, + 18.44475969602354, + 18.65870678203646, + 18.873071267968044, + 19.094119695946574, + 19.311789667000994, + 19.525512089021504, + 19.744589620037004, + 19.95895287406165, + 20.17413644399494, + 20.39556747896131, + 20.615042916033417, + 20.83942473202478, + 21.05684030998964, + 21.27453809999861, + 21.491686603054404, + 21.713433583965525, + 21.936580042005517, + 22.136591720045544, + 22.33359159703832, + 22.52776433597319, + 22.72328496805858, + 22.912036065012217, + 23.100072316010483, + 23.272760197985917, + 23.439194943057373, + 23.574723582016304, + 23.69968413305469, + 23.79082139802631, + 23.85629413300194, + 23.908119485946372, + 23.957211297005415 + ], + "chunk_gaps_ms": { + "count": 112, + "p50": 217.90710248751566, + "p95": 270.8296290365979, + "max": 355.4998830659315 + } + }, + { + "start": 722196.145079382, + "first": 722197.031815532, + "last": 722219.582927083, + "end": 722219.583646771, + "elapsed_s": 23.438567389035597, + "chunk_offsets_s": [ + 0.8867361500160769, + 1.01245772105176, + 1.320510764955543, + 1.6383943320252001, + 1.9723466569557786, + 2.328600123990327, + 2.530282109975815, + 2.729631621041335, + 2.9345457670278847, + 3.1379856190178543, + 3.3472291020443663, + 3.5570250330492854, + 3.7729646490188316, + 3.986075042048469, + 4.198023478034884, + 4.405337139964104, + 4.621293558040634, + 4.840218038996682, + 5.061206612037495, + 5.268710479023866, + 5.493343406007625, + 5.708175266976468, + 5.926661762059666, + 6.1462476489832625, + 6.362924566026777, + 6.580373088945635, + 6.797635558061302, + 7.013981172000058, + 7.23279018397443, + 7.4477531460579485, + 7.670178220956586, + 7.882563082035631, + 8.101971070980653, + 8.321328031015582, + 8.53802968095988, + 8.758832967025228, + 8.976033800048754, + 9.196208633948117, + 9.420839181053452, + 9.643895413028076, + 9.869745122967288, + 10.096053140005097, + 10.32117637200281, + 10.54340929503087, + 10.761694701039232, + 10.984511277987622, + 11.205581509042531, + 11.42966930102557, + 11.649767877999693, + 11.872502455022186, + 12.091779005015269, + 12.309511166997254, + 12.531815784052014, + 12.746511428034864, + 12.96520157204941, + 13.183484141947702, + 13.399560029036365, + 13.624008508981206, + 13.84450685698539, + 14.06120411597658, + 14.277094805031084, + 14.498692412045784, + 14.718457369948737, + 14.933840621029958, + 15.147969334968366, + 15.361800333950669, + 15.583275516051799, + 15.804137011989951, + 16.024105206946842, + 16.24773795402143, + 16.472343289060518, + 16.68919816997368, + 16.910308809019625, + 17.13411731296219, + 17.355723420041613, + 17.57916528498754, + 17.793534755008295, + 18.013292748015374, + 18.229396010981873, + 18.444869230967015, + 18.65805263502989, + 18.872903599054553, + 19.094069894053973, + 19.311979033984244, + 19.52574715204537, + 19.74498782795854, + 19.958659849013202, + 20.174228652962483, + 20.394869616022334, + 20.61452147900127, + 20.838845420978032, + 21.05636461696122, + 21.274368572980165, + 21.491754384012893, + 21.713269419036806, + 21.93501654395368, + 22.136427361983806, + 22.332999745965935, + 22.526902624987997, + 22.723136279033497, + 22.91193595097866, + 23.100095200003125, + 23.273319817031734, + 23.437847701017745 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 218.28256989829242, + "p95": 225.8497099392116, + "max": 356.2534670345485 + } + }, + { + "start": 722196.145191764, + "first": 722196.761388749, + "last": 722219.844474159, + "end": 722219.84476407, + "elapsed_s": 23.69957230600994, + "chunk_offsets_s": [ + 0.6161969850072637, + 0.7295680689858273, + 1.0125258120242506, + 1.3205626660492271, + 1.6384820489911363, + 1.9720251789549366, + 2.3287796060321853, + 2.530036636046134, + 2.7298262000549585, + 2.934382888954133, + 3.1376726189628243, + 3.3466396459843963, + 3.5566643240163103, + 3.773474838002585, + 3.986157370964065, + 4.197652447968721, + 4.405187151976861, + 4.621122836950235, + 4.840169470058754, + 5.06106424506288, + 5.268799107056111, + 5.493499304982834, + 5.708158491062932, + 5.92651780997403, + 6.145761381951161, + 6.362564831972122, + 6.580056851031259, + 6.798086476977915, + 7.014234942966141, + 7.233123062062077, + 7.448119241045788, + 7.669760204968043, + 7.883031071047299, + 8.10235624795314, + 8.321761160041206, + 8.537858820986003, + 8.75918166898191, + 8.976509383064695, + 9.196356571977958, + 9.420462443958968, + 9.643893965985626, + 9.869974125991575, + 10.096077914000489, + 10.321132117998786, + 10.543409687001258, + 10.762071912060492, + 10.984844565042295, + 11.20585290400777, + 11.429800504003651, + 11.650636172969826, + 11.872819566982798, + 12.09218607796356, + 12.309967534034513, + 12.5322587529663, + 12.74656172702089, + 12.96556330495514, + 13.183280584984459, + 13.39951886201743, + 13.623301599058323, + 13.844530598027632, + 14.060188048984855, + 14.27632598998025, + 14.498178406967781, + 14.718406261061318, + 14.93447943101637, + 15.147512109950185, + 15.361397051019594, + 15.58310295001138, + 15.80350019596517, + 16.023241483955644, + 16.246991678955965, + 16.47253428795375, + 16.688556223060004, + 16.90993072395213, + 17.133591191959567, + 17.355254585039802, + 17.578518046997488, + 17.792911653988995, + 18.012457775999792, + 18.229061115998775, + 18.44421466998756, + 18.657652821042575, + 18.87203976197634, + 19.093604441033676, + 19.311339284060523, + 19.525297836982645, + 19.74396319303196, + 19.957868191064335, + 20.173460867023095, + 20.3945825350238, + 20.613991420948878, + 20.838199164019898, + 21.055819866014645, + 21.27410007698927, + 21.49108230206184, + 21.712756971013732, + 21.935207747970708, + 22.136397934053093, + 22.332594232051633, + 22.526982926996425, + 22.723276871955022, + 22.91126080194954, + 23.099114755983464, + 23.273015554063022, + 23.437878306023777, + 23.574146623024717, + 23.699282394954935 + ], + "chunk_gaps_ms": { + "count": 106, + "p50": 218.15491846064106, + "p95": 226.1037880089134, + "max": 356.7544270772487 + } + }, + { + "start": 722196.14478601, + "first": 722196.521512624, + "last": 722218.868266493, + "end": 722218.868487314, + "elapsed_s": 22.723701303941198, + "chunk_offsets_s": [ + 0.3767266139620915, + 0.4590523549122736, + 0.7298052449477836, + 1.0127025640103966, + 1.3207636299775913, + 1.6392572110053152, + 1.9728315299144015, + 2.3291008109226823, + 2.5305323830107227, + 2.7303632679395378, + 2.9346369799459353, + 3.138498884974979, + 3.3473149079363793, + 3.557280060951598, + 3.7735771909356117, + 3.9864076540106907, + 4.1982747609727085, + 4.405416017980315, + 4.621313456911594, + 4.8405409479746595, + 5.061421409947798, + 5.269707113970071, + 5.493447358952835, + 5.708391026942991, + 5.926880819024518, + 6.146218381007202, + 6.362889549927786, + 6.580122370971367, + 6.79817170300521, + 7.014711583964527, + 7.233326628920622, + 7.448174378019758, + 7.670648294966668, + 7.883276003994979, + 8.10286202293355, + 8.322128413943574, + 8.537967975018546, + 8.759551371913403, + 8.976842987933196, + 9.19679515901953, + 9.421237373026088, + 9.644337594974786, + 9.870344678987749, + 10.096518980921246, + 10.321170128998347, + 10.543617281946354, + 10.76238851994276, + 10.985295107937418, + 11.206292737973854, + 11.430257914005779, + 11.650803432916291, + 11.87311378098093, + 12.092711054952815, + 12.309885422931984, + 12.532776764943264, + 12.747167169931345, + 12.965553433983587, + 13.18397210992407, + 13.400460365926847, + 13.624370069941506, + 13.844704604009166, + 14.061537008965388, + 14.277302125003189, + 14.499181784922257, + 14.719846957013942, + 14.935197633923963, + 15.148457456962205, + 15.36229557893239, + 15.583419451955706, + 15.804732152959332, + 16.024344204925, + 16.24811929592397, + 16.472470532986335, + 16.68931148899719, + 16.910959791974165, + 17.134470578981563, + 17.35582654492464, + 17.579584110993892, + 17.793785838992335, + 18.01398569298908, + 18.22965564194601, + 18.445194886997342, + 18.657791279023513, + 18.872872118023224, + 19.094442046014592, + 19.31231856602244, + 19.52599598094821, + 19.74522161099594, + 19.958714082022198, + 20.17455826094374, + 20.395437208004296, + 20.614760897005908, + 20.839239577995613, + 21.056738223996945, + 21.274698980967514, + 21.49185835092794, + 21.713864792021923, + 21.93523766892031, + 22.136503689922392, + 22.333498236956075, + 22.527741350932047, + 22.723480482934974 + ], + "chunk_gaps_ms": { + "count": 101, + "p50": 218.61504495609552, + "p95": 270.75289003551006, + "max": 356.2692810082808 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 70.5, + "per_stream_tps": 70.5, + "ttft_s_max": 0.11, + "timing_valid": true, + "end_to_end_tps": 68.71561576230782, + "makespan_s": 3.725499614025466, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 70.53580911858204, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 3.725499614025466, + "end_to_end_tps": 68.71561576230782, + "timing_valid": true, + "per_stream_tps": [ + 70.5 + ], + "ttft_s": [ + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 722235.556462734, + "first": 722235.666611153, + "last": 722239.281796166, + "end": 722239.281962348, + "elapsed_s": 3.725499614025466, + "chunk_offsets_s": [ + 0.11014841904398054, + 0.17006504000164568, + 0.2290628079790622, + 0.2896173569606617, + 0.35552163899410516, + 0.42278774303849787, + 0.49186860595364124, + 0.5571063800016418, + 0.6223378649447113, + 0.6882618790259585, + 0.7523059130180627, + 0.8184156219940633, + 0.8835561720188707, + 0.9478663679910824, + 1.0127805459778756, + 1.0776104719843715, + 1.144090767018497, + 1.2086098439758644, + 1.2655097959795967, + 1.3167079139966518, + 1.3677418460138142, + 1.4207860119640827, + 1.4808064349927008, + 1.5405038469471037, + 1.597893635975197, + 1.6569560370407999, + 1.7226026440039277, + 1.7873444670112804, + 1.8508832299849018, + 1.9122580849798396, + 1.993833772954531, + 2.0741418950492516, + 2.156849639955908, + 2.2421215140493587, + 2.322332708979957, + 2.408764051971957, + 2.492038441938348, + 2.576921056956053, + 2.6614403380081058, + 2.746285393019207, + 2.829703134018928, + 2.9138499239925295, + 2.9968856750056148, + 3.0818242270033807, + 3.166209243936464, + 3.250260420027189, + 3.334207182051614, + 3.420536685967818, + 3.50421585701406, + 3.5851249980041757, + 3.668903399957344, + 3.7253334319684654 + ], + "chunk_gaps_ms": { + "count": 51, + "p50": 65.92401408124715, + "p95": 85.2718740934506, + "max": 86.43134299200028 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 81.8, + "per_stream_tps": 41.6, + "ttft_s_max": 0.28, + "timing_valid": true, + "end_to_end_tps": 80.6698486307507, + "makespan_s": 6.346857080934569, + "steady_tps": 83.5, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 81.76923789576463, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 6.346857080934569, + "end_to_end_tps": 80.6698486307507, + "timing_valid": true, + "steady_tps": 83.54651365610236, + "per_stream_tps": [ + 42.0, + 41.3 + ], + "ttft_s": [ + 0.28, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 722239.284280876, + "first": 722239.561959339, + "last": 722245.630650791, + "end": 722245.630798486, + "elapsed_s": 6.346517609898001, + "chunk_offsets_s": [ + 0.2776784629095346, + 0.35868121800012887, + 0.44771911099087447, + 0.5339826859999448, + 0.6243983539752662, + 0.7134900459786877, + 0.8034643249120563, + 0.8901400530012324, + 0.972851202939637, + 1.0559086999855936, + 1.143570652930066, + 1.2281083209672943, + 1.3196360799483955, + 1.4085606479784474, + 1.497373535996303, + 1.588675809907727, + 1.6760211929213256, + 1.7675170439761132, + 1.8567536359187216, + 1.9481810459401459, + 2.0396741739241406, + 2.1288650159258395, + 2.215023315977305, + 2.3031433989526704, + 2.3928515230072662, + 2.4796591529157013, + 2.563236149959266, + 2.6480966849485412, + 2.7319200659403577, + 2.818062538979575, + 2.9022463489091024, + 2.9884472729172558, + 3.074090309906751, + 3.1598702439805493, + 3.2473167149582878, + 3.335819858009927, + 3.4192061260109767, + 3.50599210197106, + 3.5939306479413062, + 3.6831149619538337, + 3.768670275923796, + 3.855297032976523, + 3.943659326992929, + 4.030332874972373, + 4.117404446937144, + 4.204867295920849, + 4.293557918979786, + 4.380931540974416, + 4.468696760945022, + 4.556263973936439, + 4.6438804409699515, + 4.7310763949062675, + 4.818369143991731, + 4.905587772955187, + 4.992933625006117, + 5.080515565001406, + 5.168838722980581, + 5.255991206970066, + 5.343632833915763, + 5.429834204958752, + 5.517540094908327, + 5.605804109945893, + 5.693168330006301, + 5.779399032937363, + 5.865552333998494, + 5.952585029997863, + 6.036597554921173, + 6.120663077919744, + 6.2069902959046885, + 6.289761474006809, + 6.346369914943352 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 87.3190660495311, + "p95": 91.4274100214243, + "max": 91.52775898110121 + } + }, + { + "start": 722239.283941405, + "first": 722239.393586424, + "last": 722245.574147689, + "end": 722245.574237315, + "elapsed_s": 6.290295909973793, + "chunk_offsets_s": [ + 0.10964501893613487, + 0.17034873901866376, + 0.35886908404063433, + 0.447870907955803, + 0.5343696679919958, + 0.6250330649781972, + 0.7138865010347217, + 0.8036192100262269, + 0.8907435280270875, + 0.9734170089941472, + 1.0560659549664706, + 1.1441889599664137, + 1.2286310319323093, + 1.319768980029039, + 1.4089581100270152, + 1.4977529329480603, + 1.5890689520165324, + 1.6761131749954075, + 1.767676827032119, + 1.8571880799718201, + 1.948574331938289, + 2.0400978149846196, + 2.129243339993991, + 2.215217599994503, + 2.3034319679718465, + 2.3929198399418965, + 2.479761329945177, + 2.563369147013873, + 2.6481909409631044, + 2.7322081859456375, + 2.8183397139655426, + 2.9027889650315046, + 2.988956796936691, + 3.0741873839870095, + 3.1599832070060074, + 3.2474340469343588, + 3.3359004500089213, + 3.4194485129555687, + 3.5060838309582323, + 3.594208671944216, + 3.6831954900408164, + 3.7687636699993163, + 3.855408395989798, + 3.9438085990259424, + 4.030738433939405, + 4.117838233010843, + 4.205244162934832, + 4.293839238933288, + 4.3813432769384235, + 4.468793228035793, + 4.556367498007603, + 4.643958249944262, + 4.731182303046808, + 4.818455626023933, + 4.9058601800352335, + 4.993210384971462, + 5.080796772963367, + 5.169117867015302, + 5.256285632960498, + 5.343911817995831, + 5.43012475897558, + 5.517815110972151, + 5.6060983600327745, + 5.693463108036667, + 5.779691442032345, + 5.865981478942558, + 5.952854044968262, + 6.0368787799961865, + 6.12094519904349, + 6.207261951989494, + 6.290206284029409 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 87.35747647006065, + "p95": 91.38625196646899, + "max": 188.52034502197057 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 115.2, + "per_stream_tps": 30.0, + "ttft_s_max": 0.68, + "timing_valid": true, + "end_to_end_tps": 114.24746459205294, + "makespan_s": 8.962999779963866, + "steady_tps": 121.3, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 115.23171189596215, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 8.962999779963866, + "end_to_end_tps": 114.24746459205294, + "timing_valid": true, + "steady_tps": 121.30607980689312, + "per_stream_tps": [ + 30.2, + 29.8, + 30.8, + 29.6 + ], + "ttft_s": [ + 0.47, + 0.28, + 0.68, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 722245.632842015, + "first": 722246.099720941, + "last": 722254.537415734, + "end": 722254.537550896, + "elapsed_s": 8.904708881047554, + "chunk_offsets_s": [ + 0.4668789260322228, + 0.5711125519592315, + 0.7981910730013624, + 0.9199309659888968, + 1.0391425079433247, + 1.157450407044962, + 1.2752143619582057, + 1.389895173953846, + 1.5041903789388016, + 1.6179317679489031, + 1.7318309179972857, + 1.854578749043867, + 1.981546155991964, + 2.1112286240095273, + 2.240770839038305, + 2.3677952439757064, + 2.496000482002273, + 2.6240619940217584, + 2.751962248934433, + 2.8717869450338185, + 3.0002474760403857, + 3.127823124988936, + 3.2519052849384025, + 3.373925670981407, + 3.4958315020194277, + 3.6152194859459996, + 3.7330282769398764, + 3.8466864259680733, + 3.957342664944008, + 4.072451181942597, + 4.190654707024805, + 4.307896306971088, + 4.425182310980745, + 4.5452170530334115, + 4.663411941961385, + 4.779733118950389, + 4.896230567945167, + 5.017125008045696, + 5.138545282999985, + 5.25984466902446, + 5.380345699959435, + 5.500416390947066, + 5.61809112096671, + 5.736512923962437, + 5.860227226978168, + 5.9820377780124545, + 6.103526687948033, + 6.22335402504541, + 6.345171281020157, + 6.465030140941963, + 6.581741326954216, + 6.701304413028993, + 6.8209362420020625, + 6.940718514029868, + 7.061421480029821, + 7.1827130659949034, + 7.3039084379561245, + 7.421277582994662, + 7.539939626934938, + 7.6590177359757945, + 7.777922019013204, + 7.8972167449537665, + 8.019043972948566, + 8.13925265299622, + 8.256035279016942, + 8.373176218941808, + 8.49222822394222, + 8.608251770958304, + 8.72119989793282, + 8.823055653949268, + 8.904573719017208 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 119.59745752392337, + "p95": 128.46053100656718, + "max": 227.0785210421309 + } + }, + { + "start": 722245.632526781, + "first": 722245.911198041, + "last": 722254.455285531, + "end": 722254.455513931, + "elapsed_s": 8.822987150051631, + "chunk_offsets_s": [ + 0.27867126010823995, + 0.35998529102653265, + 0.5716451531043276, + 0.7987049830844626, + 0.9203713010065258, + 1.0396124940598384, + 1.1581677451031283, + 1.2756174131063744, + 1.3908273980487138, + 1.5047019050689414, + 1.618312864098698, + 1.732326905010268, + 1.8551657370990142, + 1.9820198050001636, + 2.111392916063778, + 2.2412488571135327, + 2.368208119063638, + 2.496703017037362, + 2.624440050101839, + 2.7525575570762157, + 2.871947348001413, + 3.0007541350787506, + 3.1283128870418295, + 3.2524129200028256, + 3.373773006023839, + 3.4962551300413907, + 3.6156090300064534, + 3.733085968066007, + 3.8474110080860555, + 3.95811318710912, + 4.073185781016946, + 4.191177831031382, + 4.308502511004917, + 4.4258205340011045, + 4.546013321029022, + 4.664023905061185, + 4.78030210302677, + 4.896715848008171, + 5.017894183052704, + 5.139273512060754, + 5.260371136013418, + 5.380801762104966, + 5.500939194113016, + 5.618784099002369, + 5.737077554105781, + 5.861069015110843, + 5.982945523108356, + 6.104339129058644, + 6.223888173000887, + 6.345858881017193, + 6.46592921402771, + 6.582497556111775, + 6.702066272031516, + 6.8213008070597425, + 6.941359229036607, + 7.061956154066138, + 7.183219179045409, + 7.30438710807357, + 7.421664199093357, + 7.5404537240974605, + 7.659471748047508, + 7.778464133036323, + 7.897729689022526, + 8.018981692031957, + 8.139670422067866, + 8.25589305604808, + 8.373208872042596, + 8.493334170081653, + 8.609066992998123, + 8.72236234205775, + 8.82275875005871 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 119.55887993099168, + "p95": 129.3731110636145, + "max": 227.05982998013496 + } + }, + { + "start": 722245.632954826, + "first": 722246.310804467, + "last": 722254.595290109, + "end": 722254.595410277, + "elapsed_s": 8.962455451022834, + "chunk_offsets_s": [ + 0.6778496409533545, + 0.7983633950352669, + 0.9202179879648611, + 1.0395357799716294, + 1.1575868530198932, + 1.2754401130368933, + 1.390250522061251, + 1.5044115060009062, + 1.617963643046096, + 1.7319744670530781, + 1.8546735419658944, + 1.9816368359606713, + 2.111167625989765, + 2.2409054279560223, + 2.367505696020089, + 2.496193347964436, + 2.6241112790303305, + 2.751980841974728, + 2.87176820600871, + 3.000263283960521, + 3.1278338129632175, + 3.2519429840613157, + 3.373639611992985, + 3.4957809690386057, + 3.614885229966603, + 3.7324708190280944, + 3.8471403049770743, + 3.957474722992629, + 4.072813292965293, + 4.190340309985913, + 4.307972840964794, + 4.42534594098106, + 4.54564196895808, + 4.66365541703999, + 4.7799593770178035, + 4.896389491041191, + 5.017522815032862, + 5.138897998956963, + 5.2599821030162275, + 5.380332192988135, + 5.500550559954718, + 5.6184538380475715, + 5.736708026030101, + 5.8603436660487205, + 5.982160730054602, + 6.103582642041147, + 6.223641054006293, + 6.345218642964028, + 6.465185096021742, + 6.58184216497466, + 6.7015604299958795, + 6.820679996046238, + 6.941003652988002, + 7.061468329047784, + 7.182711735949852, + 7.303904482978396, + 7.421322223963216, + 7.540288451011293, + 7.659104828024283, + 7.777962050982751, + 7.897208565031178, + 8.018747085006908, + 8.139195222058333, + 8.256122209946625, + 8.372960936045274, + 8.492627272964455, + 8.608952346025035, + 8.721335744019598, + 8.8221419720212, + 8.904226731974632, + 8.962335283053108 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 119.69230097020045, + "p95": 128.49507795181125, + "max": 129.7378019662574 + } + }, + { + "start": 722245.632410497, + "first": 722245.743560119, + "last": 722254.354388657, + "end": 722254.354531771, + "elapsed_s": 8.722121273982339, + "chunk_offsets_s": [ + 0.1111496218945831, + 0.17117959295865148, + 0.3599273009458557, + 0.5718588549643755, + 0.7987252089660615, + 0.9206220778869465, + 1.039849349996075, + 1.1579692459199578, + 1.2757880869321525, + 1.3905385528923944, + 1.5049163108924404, + 1.618472464964725, + 1.7324909219751135, + 1.8553342659724876, + 1.9820947729749605, + 2.111576229915954, + 2.2414171299897134, + 2.3683756879763678, + 2.4966269149444997, + 2.624614659929648, + 2.7525681669358164, + 2.8722786438884214, + 3.0007543129613623, + 3.128483623964712, + 3.252442811965011, + 3.373964320984669, + 3.496412505977787, + 3.6157835759222507, + 3.7331052989466116, + 3.847402944928035, + 3.9579305009683594, + 4.073085309937596, + 4.190969814895652, + 4.308869536966085, + 4.425855082925409, + 4.545899152988568, + 4.66443989390973, + 4.780474390950985, + 4.89677487895824, + 5.017791744903661, + 5.139181921957061, + 5.260446376982145, + 5.380957504967228, + 5.501006498001516, + 5.618808502913453, + 5.7372931418940425, + 5.860945743974298, + 5.982756630983204, + 6.104092167923227, + 6.22412056196481, + 6.345496102934703, + 6.465674907900393, + 6.582325096940622, + 6.701939481892623, + 6.821305290912278, + 6.941377248964272, + 7.062190830940381, + 7.183381449896842, + 7.304637481924146, + 7.421828421996906, + 7.540689520887099, + 7.659529483993538, + 7.778540653991513, + 7.897795633994974, + 8.019004240981303, + 8.139570018975064, + 8.256133876973763, + 8.373389864922501, + 8.492930112988688, + 8.608903432963416, + 8.721978159970604 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 119.86943549709395, + "p95": 129.84090007375926, + "max": 226.86635400168598 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 152.0, + "per_stream_tps": 20.5, + "ttft_s_max": 1.68, + "timing_valid": true, + "end_to_end_tps": 151.34928133354623, + "makespan_s": 13.531613642000593, + "steady_tps": 167.0, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 152.0253876265748, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 13.531613642000593, + "end_to_end_tps": 151.34928133354623, + "timing_valid": true, + "steady_tps": 166.99766328648684, + "per_stream_tps": [ + 20.6, + 20.2, + 20.4, + 21.5, + 21.1, + 20.8, + 20.2, + 20.3 + ], + "ttft_s": [ + 0.9, + 0.11, + 0.68, + 1.68, + 1.41, + 1.15, + 0.28, + 0.47 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 722254.599233274, + "first": 722255.500659583, + "last": 722267.886185144, + "end": 722267.886394686, + "elapsed_s": 13.287161411950365, + "chunk_offsets_s": [ + 0.9014263090211898, + 1.0411629469599575, + 1.3002654029987752, + 1.5770770830567926, + 1.8641999689862132, + 2.04282949201297, + 2.2172786330338567, + 2.3935127949807793, + 2.5695512159727514, + 2.747528644045815, + 2.9315385529771447, + 3.121131104999222, + 3.313268619007431, + 3.50983479002025, + 3.705032716039568, + 3.8994143219897524, + 4.093924334971234, + 4.285486922948621, + 4.475786483031698, + 4.666325967991725, + 4.8584401910193264, + 5.048802325036377, + 5.238901308039203, + 5.423581133945845, + 5.604483693023212, + 5.780208031996153, + 5.953415732947178, + 6.125731391017325, + 6.297319493955001, + 6.465432532946579, + 6.633399275946431, + 6.801877173944376, + 6.97416558698751, + 7.141064106952399, + 7.308723301975988, + 7.478404236026108, + 7.648658205056563, + 7.817916427040473, + 7.987280479981564, + 8.15798412496224, + 8.328228248050436, + 8.499379175947979, + 8.670504136011004, + 8.842958898982033, + 9.013603750965558, + 9.185701444046572, + 9.35960388102103, + 9.531840282958001, + 9.703936127014458, + 9.8740237649763, + 10.046200328040868, + 10.215183028019965, + 10.384885690989904, + 10.554978576954454, + 10.724261853960343, + 10.890964164980687, + 11.06060350604821, + 11.23483060405124, + 11.406150946975686, + 11.574552261037752, + 11.74327810795512, + 11.912955295993015, + 12.079572033020668, + 12.245632463949732, + 12.413878789986484, + 12.58061029901728, + 12.744542562053539, + 12.901004369021393, + 13.04496495903004, + 13.17413880804088, + 13.286951869959012 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.45422293106094, + "p95": 196.5661710128188, + "max": 287.1228859294206 + } + }, + { + "start": 722254.599518269, + "first": 722254.711129841, + "last": 722267.343457682, + "end": 722267.343572342, + "elapsed_s": 12.74405407300219, + "chunk_offsets_s": [ + 0.11161157209426165, + 0.1696907930308953, + 0.3571591810323298, + 0.5689134950516745, + 0.7931717190658674, + 1.041048826999031, + 1.299893363029696, + 1.576740437070839, + 1.863794070086442, + 2.0421405050437897, + 2.216936659067869, + 2.3928516330197453, + 2.569187176064588, + 2.74781253002584, + 2.9315613040234894, + 3.1216518360888585, + 3.313526296056807, + 3.5093068530550227, + 3.705403948086314, + 3.899232532014139, + 4.094068370992318, + 4.285085682058707, + 4.475806529051624, + 4.666312299086712, + 4.858269329997711, + 5.04878832004033, + 5.238888135063462, + 5.423529735067859, + 5.604403173085302, + 5.77960166207049, + 5.953999564051628, + 6.126259747077711, + 6.297882316051982, + 6.465289832092822, + 6.633322067093104, + 6.8018361740978435, + 6.974361286032945, + 7.141243645106442, + 7.308101971051656, + 7.478983969078399, + 7.648431133013219, + 7.817929061013274, + 7.986802389030345, + 8.157952213077806, + 8.328492476022802, + 8.499799155048095, + 8.67037332302425, + 8.84224071900826, + 9.012943925103173, + 9.185378383030184, + 9.359127966105007, + 9.532216674066149, + 9.703495238092728, + 9.874001483083703, + 10.045509813004173, + 10.214699594071135, + 10.384541798033752, + 10.554426084039733, + 10.723645135061815, + 10.890553022036329, + 11.06004234100692, + 11.234153483994305, + 11.405797214014456, + 11.574301491025835, + 11.743076604092494, + 11.912930260994472, + 12.079486884060316, + 12.245287836063653, + 12.41355238610413, + 12.579993693041615, + 12.743939412990585 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 172.06378950504586, + "p95": 247.87710793316364, + "max": 287.0536330156028 + } + }, + { + "start": 722254.598576685, + "first": 722255.275106397, + "last": 722267.772971158, + "end": 722267.773140491, + "elapsed_s": 13.17456380603835, + "chunk_offsets_s": [ + 0.6765297120437026, + 0.7939916150644422, + 1.042084193089977, + 1.3008795260684565, + 1.5780115460511297, + 1.8651409590383992, + 2.042975794058293, + 2.217504844069481, + 2.394220747053623, + 2.570313860080205, + 2.7487858440726995, + 2.9325665400829166, + 3.122512408066541, + 3.314239435014315, + 3.510186208994128, + 3.706106255063787, + 3.900383344036527, + 4.09506147808861, + 4.286252157995477, + 4.477018880075775, + 4.667382722022012, + 4.859353834064677, + 5.049688493018039, + 5.239873434067704, + 5.424540298059583, + 5.605182508006692, + 5.7807317830156535, + 5.954840599093586, + 6.126934391097166, + 6.298856205074117, + 6.466366031090729, + 6.634393034037203, + 6.80274268402718, + 6.975077083101496, + 7.142306211055256, + 7.309149928041734, + 7.479227617033757, + 7.649258311023004, + 7.818835795042105, + 7.988022848032415, + 8.15885749203153, + 8.328659259015694, + 8.500440934090875, + 8.670809431001544, + 8.842802672064863, + 9.013461780035868, + 9.185436700005084, + 9.35924892500043, + 9.531949874013662, + 9.704108714009635, + 9.87446786602959, + 10.046083032037131, + 10.21548742800951, + 10.385129241039976, + 10.555232815095223, + 10.724407561006956, + 10.89122245204635, + 11.060716172098182, + 11.234861284028739, + 11.406339568085968, + 11.575036732014269, + 11.743744003004394, + 11.913529386045411, + 12.079724478069693, + 12.24612290400546, + 12.414300122996792, + 12.580775656038895, + 12.744641054072417, + 12.901382758049294, + 13.045439911074936, + 13.174394473084249 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.8517445260659, + "p95": 248.09257802553475, + "max": 287.12941298726946 + } + }, + { + "start": 722254.598813997, + "first": 722256.282043502, + "last": 722268.129941211, + "end": 722268.130076239, + "elapsed_s": 13.531262242002413, + "chunk_offsets_s": [ + 1.6832295049680397, + 1.8648678050376475, + 2.042491363012232, + 2.2173457769677043, + 2.3936991930240765, + 2.570137655013241, + 2.748595927027054, + 2.9325871949549764, + 3.1225024530431256, + 3.31390635704156, + 3.509803688968532, + 3.705908801057376, + 3.9002210119506344, + 4.09488773799967, + 4.28597319603432, + 4.476733932970092, + 4.667261527036317, + 4.8590271569555625, + 5.049539393978193, + 5.2396729399915785, + 5.424264615983702, + 5.604402784025297, + 5.780756035004742, + 5.953882643952966, + 6.126190430019051, + 6.297942157019861, + 6.4658169919857755, + 6.633750134031288, + 6.802250928943977, + 6.974318980006501, + 7.1415649560512975, + 7.308619434945285, + 7.479413939989172, + 7.649199823033996, + 7.817988264956512, + 7.987029599957168, + 8.158184017054737, + 8.32870387902949, + 8.500839538057335, + 8.670456961961463, + 8.842675060033798, + 9.013372553978115, + 9.185331793036312, + 9.35964719101321, + 9.531828918959945, + 9.704025279963389, + 9.874791583046317, + 10.046560658025555, + 10.215562719036825, + 10.38536961004138, + 10.55536510096863, + 10.724928259034641, + 10.89146672503557, + 11.060376023990102, + 11.234562705969438, + 11.405902249040082, + 11.574633317999542, + 11.743420689017512, + 11.91323496797122, + 12.079435387975536, + 12.24573889898602, + 12.41400516196154, + 12.580473685986362, + 12.74435066897422, + 12.901069108047523, + 13.045327753992751, + 13.174195706029423, + 13.287164437002502, + 13.388711677049287, + 13.472992839990184, + 13.531127213966101 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.24698008410633, + "p95": 194.31221089325845, + "max": 196.10511208884418 + } + }, + { + "start": 722254.599714299, + "first": 722256.005438749, + "last": 722268.072075821, + "end": 722268.072180048, + "elapsed_s": 13.4724657490151, + "chunk_offsets_s": [ + 1.4057244500145316, + 1.5762935749953613, + 1.8631543649826199, + 2.0413826000876725, + 2.216166097088717, + 2.3932718799915165, + 2.5687714850064367, + 2.7473135070176795, + 2.931403324007988, + 3.1211174830095842, + 3.313056266051717, + 3.5089705269783735, + 3.7050673990743235, + 3.899283044040203, + 4.094021804980002, + 4.285159443039447, + 4.475926932063885, + 4.666320407064632, + 4.85826161399018, + 5.048511005006731, + 5.238809984060936, + 5.4232995110796764, + 5.603128095041029, + 5.779635978979059, + 5.953652390977368, + 6.125718197086826, + 6.297649148036726, + 6.4653112370288, + 6.633314062026329, + 6.801675137015991, + 6.973906652070582, + 7.140627164044417, + 7.307438176008873, + 7.477571979048662, + 7.6479719390627, + 7.817310005077161, + 7.986227269982919, + 8.15735899808351, + 8.327339124982245, + 8.498985275044106, + 8.669865393079817, + 8.84161393600516, + 9.012525181984529, + 9.184465333004482, + 9.35840770707, + 9.531001387978904, + 9.703169508022256, + 9.873379533994012, + 10.044894136022776, + 10.214396152063273, + 10.384062748984434, + 10.553965884028003, + 10.723345485981554, + 10.890153608983383, + 11.06008811108768, + 11.234317786060274, + 11.405527485068887, + 11.574037746991962, + 11.742934016045183, + 11.912518416065723, + 12.0792581330752, + 12.245197489042766, + 12.413460807059892, + 12.580090649076737, + 12.744374122004956, + 12.900559921050444, + 13.044522127020173, + 13.173615830019116, + 13.286587632028386, + 13.388091624015942, + 13.472361522028223 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.17071355460212, + "p95": 194.73876093979925, + "max": 286.8607899872586 + } + }, + { + "start": 722254.598654755, + "first": 722255.746336229, + "last": 722267.987425815, + "end": 722267.987540507, + "elapsed_s": 13.388885751948692, + "chunk_offsets_s": [ + 1.147681473987177, + 1.3010360149201006, + 1.57786686392501, + 1.8643931199330837, + 2.043474367936142, + 2.217312271008268, + 2.3934826359618455, + 2.569902424933389, + 2.747930896934122, + 2.932024522917345, + 3.1217987169511616, + 3.3139508239692077, + 3.510459200013429, + 3.7065425330074504, + 3.9002595589263365, + 4.094894794980064, + 4.286250172997825, + 4.476784192956984, + 4.6674802609486505, + 4.859232320915908, + 5.049532722914591, + 5.23971003096085, + 5.424502870999277, + 5.604782117996365, + 5.780535738915205, + 5.954377086949535, + 6.126498494995758, + 6.298442311002873, + 6.466246326919645, + 6.634264768916182, + 6.802629028912634, + 6.975321923964657, + 7.142264573951252, + 7.309106819913723, + 7.479203069000505, + 7.64940932393074, + 7.818722907919437, + 7.9878933519357815, + 8.158743163920008, + 8.329694084008224, + 8.500103988917544, + 8.670851925970055, + 8.842878895928152, + 9.013461056980304, + 9.185703587951139, + 9.359271194902249, + 9.532265195972286, + 9.704068421968259, + 9.874501431942917, + 10.046044738963246, + 10.215335482964292, + 10.385183639940806, + 10.55507504590787, + 10.724729984998703, + 10.891090588993393, + 11.06077252200339, + 11.235173834953457, + 11.406130741001107, + 11.574892755947076, + 11.743635355960578, + 11.913022405933589, + 12.079988850979134, + 12.245964015950449, + 12.414584144949913, + 12.580593774910085, + 12.744603193015791, + 12.901268607005477, + 13.045648744911887, + 13.17427699395921, + 13.287410608958453, + 13.388771059922874 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 170.95391306793317, + "p95": 196.08333299402148, + "max": 286.5262560080737 + } + }, + { + "start": 722254.599641046, + "first": 722254.875670491, + "last": 722267.500428385, + "end": 722267.500507395, + "elapsed_s": 12.900866348994896, + "chunk_offsets_s": [ + 0.2760294449981302, + 0.357215152005665, + 0.5691039559897035, + 0.7928815869381651, + 1.0407179730245844, + 1.3001306250225753, + 1.577094474923797, + 1.86400171299465, + 2.042323953937739, + 2.216761142015457, + 2.3932301339227706, + 2.5693444969365373, + 2.747247694991529, + 2.9313698189798743, + 3.120872349012643, + 3.3130091439234093, + 3.5095305759459734, + 3.704694867017679, + 3.8990516079356894, + 4.093639481929131, + 4.2847988799912855, + 4.475433929939754, + 4.666010104934685, + 4.858075254014693, + 5.048486204934306, + 5.238350393017754, + 5.423263591015711, + 5.603635114035569, + 5.779618442989886, + 5.953101229970343, + 6.125404390972108, + 6.296958075952716, + 6.465066027012654, + 6.6330467709340155, + 6.801504538976587, + 6.973536396981217, + 7.1408209500368685, + 7.307446977938525, + 7.477595533942804, + 7.648471130989492, + 7.817278677015565, + 7.9864361609797925, + 8.157482221024111, + 8.327458380023018, + 8.499275128939189, + 8.670282018953003, + 8.842400480993092, + 9.01285919698421, + 9.184894551988691, + 9.358898752951063, + 9.53192646696698, + 9.703874335973524, + 9.873839617008343, + 10.045663079014048, + 10.214663587976247, + 10.38438065093942, + 10.554427534923889, + 10.724047304945998, + 10.890970646985807, + 11.060229255002923, + 11.234255578950979, + 11.405708294012584, + 11.574032132979482, + 11.743083174922504, + 11.911908862995915, + 12.079115538974293, + 12.245384841924533, + 12.413392471964471, + 12.58011386799626, + 12.744298425968736, + 12.90078733896371 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.98986350558698, + "p95": 247.83638608641922, + "max": 286.90723807085305 + } + }, + { + "start": 722254.598462597, + "first": 722255.06359083, + "last": 722267.643939969, + "end": 722267.644325341, + "elapsed_s": 13.045862743980251, + "chunk_offsets_s": [ + 0.46512823295779526, + 0.570114632952027, + 0.7939168750308454, + 1.0418334730202332, + 1.3006734650116414, + 1.5776837499579415, + 1.8644729349762201, + 2.0427134910132736, + 2.2172016899567097, + 2.3936279440531507, + 2.5699408459477127, + 2.748353003989905, + 2.932876111008227, + 3.1222484899917617, + 3.3137687370181084, + 3.5105626160511747, + 3.706373327993788, + 3.900090705952607, + 4.094897183007561, + 4.286200396949425, + 4.476701583946124, + 4.667137110955082, + 4.859107838943601, + 5.0494960700161755, + 5.239600526052527, + 5.42429167998489, + 5.6048680619569495, + 5.781257619033568, + 5.954634911962785, + 6.127012141048908, + 6.298535775975324, + 6.466399203054607, + 6.634415165055543, + 6.8027857129927725, + 6.9754656159784645, + 7.142336391028948, + 7.308800027007237, + 7.47929338295944, + 7.649287867010571, + 7.818753171013668, + 7.988013939000666, + 8.159088304964826, + 8.329324297956191, + 8.500907917041332, + 8.67152483901009, + 8.843462493969128, + 9.014210036955774, + 9.186337506980635, + 9.359921162948012, + 9.532804999966174, + 9.704901866964065, + 9.875095012946986, + 10.046758724958636, + 10.215881941956468, + 10.385756628005765, + 10.555655858013779, + 10.724996850010939, + 10.892089126049541, + 11.061134957941249, + 11.235473548993468, + 11.406817971961573, + 11.57539482798893, + 11.744294760981575, + 11.91329365503043, + 12.080250706989318, + 12.24649591196794, + 12.414836928946897, + 12.58108762302436, + 12.74552929296624, + 12.901579568046145, + 13.045477372012101 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.80068348534405, + "p95": 247.9165979893878, + "max": 286.78918501827866 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 35.0, + "per_stream_tps": 35.0, + "ttft_s_max": 0.16, + "timing_valid": true, + "end_to_end_tps": 34.365655118950734, + "makespan_s": 7.449297826969996, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 34.98055720456415, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 7.449297826969996, + "end_to_end_tps": 34.365655118950734, + "timing_valid": true, + "per_stream_tps": [ + 35.0 + ], + "ttft_s": [ + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 722287.445200562, + "first": 722287.60431768, + "last": 722294.894081491, + "end": 722294.894498389, + "elapsed_s": 7.449297826969996, + "chunk_offsets_s": [ + 0.15911711798980832, + 0.22083908796776086, + 0.2816730779595673, + 0.3436425499385223, + 0.4088177300291136, + 0.47504516097251326, + 0.5370454640360549, + 0.5892300659324974, + 0.6471634019399062, + 0.7035578989889473, + 0.7600017710356042, + 0.8112504429882392, + 0.8622656200313941, + 0.9132568850181997, + 0.9646999549586326, + 1.0163170869927853, + 1.0669585949508473, + 1.1199631759664044, + 1.171432305010967, + 1.2216188199818134, + 1.2722109470050782, + 1.3239737120456994, + 1.3834694740362465, + 1.4416040779324248, + 1.4982366069452837, + 1.5483680759789422, + 1.600519162020646, + 1.6609905789373443, + 1.718377495999448, + 1.7776824979810044, + 1.8368737809360027, + 1.9014621080132201, + 1.9683027539867908, + 2.0202346589649096, + 2.077290448010899, + 2.1366829599719495, + 2.1884041810408235, + 2.2401770040160045, + 2.291219266015105, + 2.3437961449380964, + 2.3955750199966133, + 2.4480997549835593, + 2.503310633939691, + 2.5549697779351845, + 2.6056174059631303, + 2.6575387709308416, + 2.7072584440466017, + 2.7589942619670182, + 2.80925671197474, + 2.860226043034345, + 2.9114758420037106, + 2.961204524966888, + 3.012490740045905, + 3.0637023199815303, + 3.114333434961736, + 3.165552217978984, + 3.217952757026069, + 3.2749032309511676, + 3.326502484967932, + 3.3768830079352483, + 3.4282571770017967, + 3.4785241599893197, + 3.529164155945182, + 3.5795932159526274, + 3.6315656419610605, + 3.6819615309359506, + 3.733243049005978, + 3.783931885031052, + 3.8362755159614608, + 3.88650962698739, + 3.9367240769788623, + 3.9892588049406186, + 4.039173744036816, + 4.090033740969375, + 4.141718636965379, + 4.193192524951883, + 4.243865487980656, + 4.293831298942678, + 4.344881588011049, + 4.395982334041037, + 4.447427056962624, + 4.498075581970625, + 4.550121449981816, + 4.602597574936226, + 4.659839799045585, + 4.7112122640246525, + 4.763532149954699, + 4.816303153987974, + 4.874310101033188, + 4.9309781229821965, + 4.980829620966688, + 5.031776126008481, + 5.0839919990394264, + 5.14049211400561, + 5.189914849936031, + 5.240503594046459, + 5.2906161689898, + 5.340749157010578, + 5.391771245980635, + 5.442478189012036, + 5.4949893220327795, + 5.551210629986599, + 5.601400880957954, + 5.652315519982949, + 5.7044273460051045, + 5.762344817980193, + 5.820151736959815, + 5.87602072104346, + 5.927532734000124, + 5.980553808039986, + 6.040382563951425, + 6.097937965998426, + 6.154054734040983, + 6.213229096960276, + 6.27829347201623, + 6.340518738026731, + 6.392334536998533, + 6.449372823932208, + 6.508398553007282, + 6.565251274034381, + 6.615880231023766, + 6.6682761770207435, + 6.718963454011828, + 6.771226978045888, + 6.821570954984054, + 6.875288393930532, + 6.931676600011997, + 6.98883277294226, + 7.0455421599326655, + 7.094678187044337, + 7.1468433670233935, + 7.197437500930391, + 7.249253481044434, + 7.299337378935888, + 7.349553511943668, + 7.399858174961992, + 7.448880929034203 + ], + "chunk_gaps_ms": { + "count": 136, + "p50": 51.77584901684895, + "p95": 62.00030306354165, + "max": 66.84064597357064 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.6, + "per_stream_tps": 25.6, + "ttft_s_max": 0.38, + "timing_valid": true, + "end_to_end_tps": 49.95665958263789, + "makespan_s": 10.248883818043396, + "steady_tps": 51.6, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.557385529834946, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 10.248883818043396, + "end_to_end_tps": 49.95665958263789, + "timing_valid": true, + "steady_tps": 51.57784520422515, + "per_stream_tps": [ + 25.8, + 25.4 + ], + "ttft_s": [ + 0.38, + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 722294.897416999, + "first": 722295.276190208, + "last": 722305.145482751, + "end": 722305.145880276, + "elapsed_s": 10.248463277006522, + "chunk_offsets_s": [ + 0.378773209056817, + 0.4639252530178055, + 0.5536414110101759, + 0.6347048709867522, + 0.7208941420540214, + 0.8145763879874721, + 0.8985314239980653, + 0.9840816459618509, + 1.0670779399806634, + 1.1555069260066375, + 1.2447676360607147, + 1.3329959920374677, + 1.4195166169665754, + 1.50382200605236, + 1.5980293810134754, + 1.694113516015932, + 1.7879430339671671, + 1.8756745849968866, + 1.965552389039658, + 2.0579901210730895, + 2.1483311250340194, + 2.234898591064848, + 2.326750783016905, + 2.4220321860630065, + 2.511325392057188, + 2.5989256020402536, + 2.6896583230700344, + 2.7801015360746533, + 2.8644651990616694, + 2.952372826053761, + 3.048325187060982, + 3.1381902010180056, + 3.2314124800032005, + 3.326347755966708, + 3.416926851030439, + 3.503544733976014, + 3.584297704976052, + 3.6716606720583513, + 3.7530645600054413, + 3.8425593150313944, + 3.9296855030115694, + 4.0199864220339805, + 4.109121344052255, + 4.199742904049344, + 4.2949323679786175, + 4.386819952982478, + 4.478873345069587, + 4.569355905987322, + 4.660816165036522, + 4.750478471047245, + 4.840778680052608, + 4.929423696012236, + 5.0210187250049785, + 5.109408388030715, + 5.202434142003767, + 5.291190057992935, + 5.384244034066796, + 5.476992601994425, + 5.5691750170663, + 5.66320967907086, + 5.752360117970966, + 5.843568395008333, + 5.931690738070756, + 6.021462933975272, + 6.113709580968134, + 6.205353772966191, + 6.2919063930166885, + 6.387235913076438, + 6.483817093074322, + 6.5739964480744675, + 6.663054466014728, + 6.747288697049953, + 6.840102376998402, + 6.933707720017992, + 7.021949723013677, + 7.11041629104875, + 7.198473021038808, + 7.291812431067228, + 7.3844059420516714, + 7.47487098898273, + 7.563198928022757, + 7.651168628013693, + 7.739744883961976, + 7.829787760972977, + 7.918506567017175, + 8.009735486004502, + 8.103569288970903, + 8.195468137040734, + 8.27996291499585, + 8.366729466011748, + 8.458525317022577, + 8.54673055105377, + 8.62820402707439, + 8.713577185990289, + 8.805398173048161, + 8.894416665076278, + 8.984572626068257, + 9.075892855995335, + 9.169136345968582, + 9.25641810905654, + 9.345161253004335, + 9.433040912961587, + 9.521196851041168, + 9.609399165026844, + 9.695865654968657, + 9.78170437598601, + 9.87176004901994, + 9.961205802042969, + 10.050142073072493, + 10.131707641063258, + 10.199475180008449, + 10.248065751977265 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 89.44575302302837, + "p95": 95.18946392927319, + "max": 96.58117999788374 + } + }, + { + "start": 722294.896996458, + "first": 722295.057935805, + "last": 722305.097156173, + "end": 722305.097333826, + "elapsed_s": 10.200337368063629, + "chunk_offsets_s": [ + 0.16093934699892998, + 0.22250508004799485, + 0.464566569076851, + 0.5546212861081585, + 0.6353365540271625, + 0.7216187680605799, + 0.8151001760270447, + 0.8991090951021761, + 0.9847159170312807, + 1.0677671330049634, + 1.1560769770294428, + 1.2452555120689794, + 1.3337650441098958, + 1.420112507068552, + 1.504466830054298, + 1.5987878401065245, + 1.6949039740720764, + 1.7888728961115703, + 1.8762748200679198, + 1.9661795120919123, + 2.0585219810018316, + 2.149074870045297, + 2.235453056055121, + 2.327384519041516, + 2.422980441013351, + 2.5120116350008175, + 2.5991399690974504, + 2.6903168311109766, + 2.780691298074089, + 2.865091403014958, + 2.9533255060669035, + 3.048859400092624, + 3.1387991190422326, + 3.2319486730266362, + 3.3269326300360262, + 3.4176355680683628, + 3.5044977540383115, + 3.584900655085221, + 3.6721929240738973, + 3.753781026112847, + 3.843161306111142, + 3.9302244500722736, + 4.020700795110315, + 4.109869412030093, + 4.200294504058547, + 4.295465450035408, + 4.387444303021766, + 4.47941269504372, + 4.569969280040823, + 4.6611348810838535, + 4.75161196209956, + 4.841405635001138, + 4.930147635051981, + 5.021602541091852, + 5.110081118997186, + 5.2030995870009065, + 5.291763904038817, + 5.384814977063797, + 5.477716663037427, + 5.569709673058242, + 5.663745039026253, + 5.7529130700277165, + 5.844190612086095, + 5.932009866111912, + 6.021944192005321, + 6.114239364047535, + 6.205862449016422, + 6.29222317202948, + 6.387745919055305, + 6.484171355026774, + 6.574498033034615, + 6.663563002017327, + 6.747919582063332, + 6.840732031036168, + 6.934284628019668, + 7.022548378095962, + 7.111020139069296, + 7.198774809017777, + 7.292404375039041, + 7.384892109083012, + 7.475228093098849, + 7.56348901509773, + 7.651530142058618, + 7.7402300130343065, + 7.830266722012311, + 7.918826726032421, + 8.01024105201941, + 8.104052197071724, + 8.195832543075085, + 8.280468258075416, + 8.36720286810305, + 8.459024942014366, + 8.547234650002792, + 8.628684317111038, + 8.71407550002914, + 8.805944459047168, + 8.894906973000616, + 8.98518487601541, + 9.076535548083484, + 9.1697148460662, + 9.257333623012528, + 9.345778569113463, + 9.433695084997453, + 9.521835347055458, + 9.610022927052341, + 9.696651697042398, + 9.78235343506094, + 9.872348334058188, + 9.96211075305473, + 10.05104176606983, + 10.13228851207532, + 10.200159715022892 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 89.76241899654269, + "p95": 95.52274702582508, + "max": 242.06148902885616 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.6, + "per_stream_tps": 17.9, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 69.14265796487372, + "makespan_s": 14.809960018028505, + "steady_tps": 72.4, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.63528058013699, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 14.809960018028505, + "end_to_end_tps": 69.14265796487372, + "timing_valid": true, + "steady_tps": 72.38552530908999, + "per_stream_tps": [ + 18.3, + 17.7, + 18.0, + 17.8 + ], + "ttft_s": [ + 0.89, + 0.16, + 0.62, + 0.38 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 722305.149160368, + "first": 722306.035269371, + "last": 722319.958501082, + "end": 722319.959026598, + "elapsed_s": 14.809866230003536, + "chunk_offsets_s": [ + 0.8861090029822662, + 1.0064367690356448, + 1.1263919370248914, + 1.2512002439470962, + 1.3747041309252381, + 1.500991181936115, + 1.6209323889343068, + 1.7433042289922014, + 1.8644265769980848, + 1.9886834670323879, + 2.11164551100228, + 2.232962467940524, + 2.358934158924967, + 2.4883429849287495, + 2.624888535006903, + 2.7602017039898783, + 2.895511867944151, + 3.0270919039612636, + 3.1564011309528723, + 3.2855935549596325, + 3.413876114995219, + 3.5439200029941276, + 3.6771506839431822, + 3.8120398010360077, + 3.9380373449530452, + 4.064272144925781, + 4.189135969034396, + 4.31126409501303, + 4.439305983949453, + 4.569705127971247, + 4.7019006829941645, + 4.834455061936751, + 4.964720319956541, + 5.093022085959092, + 5.217373037012294, + 5.3360037240199745, + 5.452427349984646, + 5.57266822992824, + 5.693635185016319, + 5.819662599009462, + 5.9456218290142715, + 6.071134942001663, + 6.200886943959631, + 6.333256089943461, + 6.464114484027959, + 6.592960470006801, + 6.723190156975761, + 6.853356117964722, + 6.982854834990576, + 7.110079047968611, + 7.237823043949902, + 7.364667980000377, + 7.494792042998597, + 7.6222168549429625, + 7.749997994978912, + 7.878475404926576, + 8.008966151974164, + 8.139634753926657, + 8.26880966394674, + 8.398799384012818, + 8.523697850992903, + 8.649893100024201, + 8.780160992988385, + 8.908224922022782, + 9.036343644023873, + 9.167330806027167, + 9.296819507027976, + 9.428081150981598, + 9.560707557015121, + 9.682770195999183, + 9.809946266002953, + 9.93644893693272, + 10.068103209952824, + 10.19636423594784, + 10.320598824997433, + 10.449373468989506, + 10.581125161028467, + 10.710103963967413, + 10.838166831992567, + 10.962023025960661, + 11.090162029955536, + 11.218230578000657, + 11.343572827987373, + 11.469680691952817, + 11.600978989969008, + 11.738107037032023, + 11.869687267928384, + 11.998931793961674, + 12.124506789958104, + 12.248659656965174, + 12.370390080963261, + 12.490465353941545, + 12.613062244956382, + 12.73963071394246, + 12.866632057004608, + 12.99229089601431, + 13.121672273031436, + 13.24864397093188, + 13.377751342952251, + 13.503457035985775, + 13.632853033952415, + 13.758513901033439, + 13.87784421001561, + 13.99542533699423, + 14.119350784923881, + 14.244957888964564, + 14.370820899959654, + 14.492904377984814, + 14.603317697998136, + 14.693786646006629, + 14.761143962969072, + 14.809340714011341 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 127.22421297803521, + "p95": 133.23068094905466, + "max": 137.1280470630154 + } + }, + { + "start": 722305.149491575, + "first": 722305.310753683, + "last": 722319.752711609, + "end": 722319.75290135, + "elapsed_s": 14.60340977506712, + "chunk_offsets_s": [ + 0.16126210801303387, + 0.22192698798608035, + 0.4641408990137279, + 0.7278249880764633, + 1.005670309998095, + 1.126161252032034, + 1.2502011969918385, + 1.3739557500230148, + 1.5001885930541903, + 1.620137918042019, + 1.7425079999957234, + 1.8636293890886009, + 1.988019451033324, + 2.1108518870314583, + 2.232416611048393, + 2.358212966006249, + 2.4877369090681896, + 2.6247544300276786, + 2.759458473068662, + 2.8947843790519983, + 3.0262826529797167, + 3.1557370320660993, + 3.2849079919978976, + 3.4132507720496505, + 3.543361769989133, + 3.676545543014072, + 3.8113842360908166, + 3.937277305056341, + 4.063748672022484, + 4.1882190480828285, + 4.310761681990698, + 4.438837486086413, + 4.569136788020842, + 4.701637353980914, + 4.833697472000495, + 4.963817001087591, + 5.092227512039244, + 5.216773107997142, + 5.33514650107827, + 5.451683782041073, + 5.572131082066335, + 5.692865899065509, + 5.819068743032403, + 5.944848836050369, + 6.070472312043421, + 6.2001709040487185, + 6.332569124060683, + 6.463369306991808, + 6.592156858067028, + 6.722317320061848, + 6.852600519079715, + 6.982239571050741, + 7.1094911800464615, + 7.237299344036728, + 7.364014476072043, + 7.494190166005865, + 7.621422725031152, + 7.749116673017852, + 7.8777987560024485, + 8.008410756010562, + 8.139149890979752, + 8.267903186031617, + 8.398126067011617, + 8.523424923070706, + 8.649011621018872, + 8.779134745011106, + 8.908057984081097, + 9.035726048052311, + 9.166566322091967, + 9.29617067007348, + 9.427195806987584, + 9.559939604019746, + 9.681976556079462, + 9.809241212089546, + 9.936278599081561, + 10.067222916986793, + 10.195594705990516, + 10.320153038017452, + 10.449208553065546, + 10.580650537041947, + 10.709465175983496, + 10.8375725770602, + 10.961838033050299, + 11.08941572601907, + 11.217377516091801, + 11.343099208083004, + 11.469059129012749, + 11.600353286019526, + 11.737437310046516, + 11.868928758078255, + 11.998166347038932, + 12.123833012068644, + 12.248113507987, + 12.369658129988238, + 12.48997050197795, + 12.612478306051344, + 12.73876342899166, + 12.865842589992099, + 12.992299623088911, + 13.121472966042347, + 13.248091823072173, + 13.376727609080262, + 13.503431815071963, + 13.632605544989929, + 13.758405618020333, + 13.877364733023569, + 13.995358697022311, + 14.11889942502603, + 14.244557972997427, + 14.370355185004883, + 14.492464928072877, + 14.603220034041442 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 127.668063971214, + "p95": 135.32590598333627, + "max": 277.8453219216317 + } + }, + { + "start": 722305.149755393, + "first": 722305.769878794, + "last": 722319.910259073, + "end": 722319.910401591, + "elapsed_s": 14.76064619794488, + "chunk_offsets_s": [ + 0.6201234009349719, + 0.7276027039624751, + 1.0053674669470638, + 1.125718283932656, + 1.2499770929571241, + 1.3736447229748592, + 1.4998789560049772, + 1.619825916015543, + 1.7423005159944296, + 1.8633215579902753, + 1.9877019840059802, + 2.1105428899172693, + 2.2321054550120607, + 2.357904878910631, + 2.487427494954318, + 2.6245689860079437, + 2.7591103829909116, + 2.894469148013741, + 3.0259748209500685, + 3.155316844000481, + 3.2846003029262647, + 3.4128364669159055, + 3.5429679529042915, + 3.676143175922334, + 3.81103273993358, + 3.936957550002262, + 4.063356437953189, + 4.187915864982642, + 4.310451881960034, + 4.438523690914735, + 4.568758421926759, + 4.701431839959696, + 4.833527087932453, + 4.9635072170058265, + 5.092169645940885, + 5.216390328016132, + 5.33507265697699, + 5.45134687400423, + 5.572256209910847, + 5.69255657796748, + 5.818671018932946, + 5.9447388699045405, + 6.0701527770143, + 6.200098039931618, + 6.332519899006002, + 6.463174553005956, + 6.591812786995433, + 6.72209961398039, + 6.852405222016387, + 6.981913849012926, + 7.108997120987624, + 7.236822622013278, + 7.363535307929851, + 7.493705068016425, + 7.621379115968011, + 7.748898408957757, + 7.877367794979364, + 8.008186896913685, + 8.13866934296675, + 8.267573272925802, + 8.397819316014647, + 8.522896469919942, + 8.648589846910909, + 8.778807053924538, + 8.907709769904613, + 9.035408778931014, + 9.166199131985195, + 9.296097756014206, + 9.426987438928336, + 9.559629030991346, + 9.681653034989722, + 9.808904487988912, + 9.935970376012847, + 10.066886257962324, + 10.195381272002123, + 10.319846747908741, + 10.44898468197789, + 10.580309320008382, + 10.709140134975314, + 10.837258901912719, + 10.961621930007823, + 11.08922711701598, + 11.2173142649699, + 11.342878054943867, + 11.468833546969108, + 11.600017118966207, + 11.73721268796362, + 11.868595244013704, + 11.997748254914768, + 12.12352275592275, + 12.247797593008727, + 12.369331856956705, + 12.48975253093522, + 12.612269175006077, + 12.738564128987491, + 12.86551214393694, + 12.992253688978963, + 13.121139368973672, + 13.247864308999851, + 13.37650994292926, + 13.502794757951051, + 13.632016252959147, + 13.757194009958766, + 13.876832464011386, + 13.99489299301058, + 14.118713699979708, + 14.244206261937506, + 14.370013977983035, + 14.492147907963954, + 14.602591648930684, + 14.693134950008243, + 14.760503679979593 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 127.51929298974574, + "p95": 134.54139698296785, + "max": 277.76476298458874 + } + }, + { + "start": 722305.14906658, + "first": 722305.527552042, + "last": 722319.843044994, + "end": 722319.843124478, + "elapsed_s": 14.694057898013853, + "chunk_offsets_s": [ + 0.37848546204622835, + 0.4648486099904403, + 0.7286000569583848, + 1.0063603459857404, + 1.1265276110498235, + 1.2510037169558927, + 1.3745726159540936, + 1.5011902060359716, + 1.6208035900490358, + 1.7432076809927821, + 1.8645728790434077, + 1.9888149870093912, + 2.1117748089600354, + 2.23309368500486, + 2.359080143040046, + 2.4883985130582005, + 2.6249144329922274, + 2.760237702052109, + 2.895571072003804, + 3.0272752860328183, + 3.1565430889604613, + 3.285735593060963, + 3.4139146209927276, + 3.5439777839928865, + 3.677200537058525, + 3.8121817749924958, + 3.9380849129520357, + 4.0644053290598094, + 4.189144090050831, + 4.311403737054206, + 4.439352447050624, + 4.569835084956139, + 4.701890264055692, + 4.83467151701916, + 4.964726316044107, + 5.093164350022562, + 5.217427283059806, + 5.336165216052905, + 5.452477489016019, + 5.572694233967923, + 5.693981840042397, + 5.819990920950659, + 5.94588313496206, + 6.071290535968728, + 6.201044360990636, + 6.3334082489600405, + 6.46425487997476, + 6.592930146958679, + 6.723518038052134, + 6.8533995259786025, + 6.983154201996513, + 7.110120523022488, + 7.238446247996762, + 7.365030247950926, + 7.494795490056276, + 7.622256105998531, + 7.750380145967938, + 7.878779795952141, + 8.00902183400467, + 8.139783798949793, + 8.268685730989091, + 8.399102324969135, + 8.524041504017077, + 8.649710600962862, + 8.779954885016195, + 8.908123046974652, + 9.036655936040916, + 9.16749299201183, + 9.296969368006103, + 9.427948932978325, + 9.560736750951037, + 9.682922216015868, + 9.810080526047386, + 9.936761297984049, + 10.068006205954589, + 10.196528453030623, + 10.320754282991402, + 10.449525043019094, + 10.581298378063366, + 10.710122192045674, + 10.838315000059083, + 10.962363046011887, + 11.090207654982805, + 11.218229845981114, + 11.343740804004483, + 11.469821461010724, + 11.601019626948982, + 11.738255462958477, + 11.869894135044888, + 11.99890761100687, + 12.124541544006206, + 12.248798380955122, + 12.370542263961397, + 12.490619215997867, + 12.61336921795737, + 12.739980192040093, + 12.866630521952175, + 12.99266158696264, + 13.121690446976572, + 13.248693814035505, + 13.37759567098692, + 13.503674010979012, + 13.632603492005728, + 13.757979169953614, + 13.87800112599507, + 13.996050065965392, + 14.119653961039148, + 14.245397827005945, + 14.370839696959592, + 14.493429683032446, + 14.60369365895167, + 14.693978414055891 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 127.84460897091776, + "p95": 135.32326905988157, + "max": 277.7602890273556 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 88.4, + "per_stream_tps": 11.6, + "ttft_s_max": 2.09, + "timing_valid": true, + "end_to_end_tps": 88.11060503909225, + "makespan_s": 23.243513071909547, + "steady_tps": 93.2, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 88.37631198256516, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 23.243513071909547, + "end_to_end_tps": 88.11060503909225, + "timing_valid": true, + "steady_tps": 93.16631574282388, + "per_stream_tps": [ + 11.9, + 11.6, + 11.7, + 11.4, + 12.1, + 11.4, + 11.8, + 11.5 + ], + "ttft_s": [ + 1.76, + 0.88, + 1.16, + 0.16, + 2.09, + 0.38, + 1.45, + 0.62 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 722319.966397093, + "first": 722321.73076203, + "last": 722343.157822132, + "end": 722343.157976576, + "elapsed_s": 23.191579482983798, + "chunk_offsets_s": [ + 1.7643649369711056, + 1.9353080409346148, + 2.278720628004521, + 2.46900965902023, + 2.6582320149755105, + 2.850883151986636, + 3.0461086509749293, + 3.2350036710267887, + 3.4228911319514737, + 3.6128925669472665, + 3.8076531069818884, + 4.004163039964624, + 4.201437811017968, + 4.402314099017531, + 4.607704459922388, + 4.810734685976058, + 5.014383059926331, + 5.218122915015556, + 5.417626958922483, + 5.620913664926775, + 5.824966193991713, + 6.025262224953622, + 6.229443328920752, + 6.432523199007846, + 6.631112660979852, + 6.8293587929802015, + 7.027339871972799, + 7.229349341010675, + 7.431007425999269, + 7.629131224937737, + 7.828310230979696, + 8.023001051973552, + 8.214162660995498, + 8.403961780946702, + 8.595684962929226, + 8.784392612986267, + 8.970052576973103, + 9.15837478695903, + 9.350415656925179, + 9.544378391932696, + 9.743426039000042, + 9.941412083921023, + 10.14109276398085, + 10.342594795976765, + 10.54567343601957, + 10.748447738937102, + 10.947508721030317, + 11.144780676928349, + 11.34121927502565, + 11.54065524903126, + 11.737804088974372, + 11.931574390968308, + 12.127553415950388, + 12.32484888099134, + 12.524418666027486, + 12.723982536001131, + 12.926990991923958, + 13.12583449902013, + 13.323671313934028, + 13.520871168002486, + 13.718183206976391, + 13.912760944920592, + 14.11379684403073, + 14.312999460962601, + 14.51083350693807, + 14.70608920196537, + 14.899982183007523, + 15.097175542963669, + 15.294384478009306, + 15.497091526980512, + 15.695085515966639, + 15.89087041595485, + 16.087698051007465, + 16.28649436298292, + 16.486018370022066, + 16.684972613002174, + 16.882022260921076, + 17.080762009951286, + 17.27535979298409, + 17.46775547892321, + 17.659509317949414, + 17.85793510999065, + 18.062479296000674, + 18.270497153978795, + 18.47498852701392, + 18.681924517964944, + 18.884161720983684, + 19.083465619944036, + 19.277340226923116, + 19.46895264799241, + 19.660854754969478, + 19.85330003895797, + 20.044630968011916, + 20.234991281991825, + 20.423558232956566, + 20.616245100973174, + 20.812172984937206, + 21.01016783795785, + 21.20823525893502, + 21.40341906202957, + 21.591388923930936, + 21.77935650094878, + 21.96412367897574, + 22.145525334984995, + 22.324339880957268, + 22.495745574939065, + 22.651715397951193, + 22.793121310998686, + 22.922725864918903, + 23.033303931937553, + 23.123867676011287, + 23.19142503896728 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.19985406845808, + "p95": 204.49137303512543, + "max": 343.4125870699063 + } + }, + { + "start": 722319.963733399, + "first": 722320.847953105, + "last": 722342.888747785, + "end": 722342.888850306, + "elapsed_s": 22.925116906990297, + "chunk_offsets_s": [ + 0.8842197059420869, + 1.0049304129788652, + 1.2979038029443473, + 1.6097804619930685, + 1.9375592899741605, + 2.2810718269320205, + 2.471066447906196, + 2.660165879991837, + 2.8530872949631885, + 3.0484302839031443, + 3.237130212946795, + 3.425189560977742, + 3.615168805932626, + 3.8097132759867236, + 4.006352274911478, + 4.203973636962473, + 4.404703347943723, + 4.610089520923793, + 4.812356412992813, + 5.0164783908985555, + 5.22015573491808, + 5.4197227609111, + 5.623218317981809, + 5.826671921997331, + 6.027588524972089, + 6.231201336951926, + 6.435282789985649, + 6.633570106932893, + 6.8310106409480795, + 7.029496948001906, + 7.230888711987063, + 7.433592904941179, + 7.631384384003468, + 7.830236856942065, + 8.025158982956782, + 8.216675982926972, + 8.406670547905378, + 8.598091638996266, + 8.78686273097992, + 8.972674155957066, + 9.160678382962942, + 9.35282427992206, + 9.547112789936364, + 9.74619700293988, + 9.943756271968596, + 10.143709849915467, + 10.345382389961742, + 10.548111171927303, + 10.751146752969362, + 10.949978242977522, + 11.147522701998241, + 11.344077047891915, + 11.543280312907882, + 11.740506796981208, + 11.93413624796085, + 12.130252367001958, + 12.327478340943344, + 12.527115666889586, + 12.72653860389255, + 12.929692932986654, + 13.128547040978447, + 13.32628818997182, + 13.523618392995559, + 13.720781726995483, + 13.915382585953921, + 14.116158650955185, + 14.315706279943697, + 14.51331968093291, + 14.708361908909865, + 14.90318987192586, + 15.099872639984824, + 15.296565625932999, + 15.500036039971747, + 15.697820239001885, + 15.893262324971147, + 16.090477305930108, + 16.28929878596682, + 16.488905192934908, + 16.68781546293758, + 16.884884283994325, + 17.08357406198047, + 17.278067908948287, + 17.47059023892507, + 17.662259184988216, + 17.861074010957964, + 18.06510162295308, + 18.272854148992337, + 18.477156860986724, + 18.684188907966018, + 18.886689189937897, + 19.086508276988752, + 19.28006530890707, + 19.471867830958217, + 19.66355565888807, + 19.85613852890674, + 20.046940767904744, + 20.23758608789649, + 20.426336166914552, + 20.618703725980595, + 20.814506943919696, + 21.012542350916192, + 21.210556920967065, + 21.40550329897087, + 21.59421615989413, + 21.782181545975618, + 21.96667880890891, + 22.148333221906796, + 22.32703618099913, + 22.498366619925946, + 22.654341831919737, + 22.795754812890664, + 22.925014385953546 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.33020302373916, + "p95": 207.03204697929323, + "max": 343.51253695786 + } + }, + { + "start": 722319.96451567, + "first": 722321.125055101, + "last": 722342.999734446, + "end": 722342.999850991, + "elapsed_s": 23.035335320979357, + "chunk_offsets_s": [ + 1.1605394310317934, + 1.2975924480706453, + 1.6093100940342993, + 1.9374759810743853, + 2.2806868770858273, + 2.471067116013728, + 2.6600501930806786, + 2.8529558130539954, + 3.0480756050674245, + 3.236934461980127, + 3.425050017074682, + 3.614844111027196, + 3.8094681340735406, + 4.005975829088129, + 4.20368313905783, + 4.404398884973489, + 4.609639810048975, + 4.812089922023006, + 5.016302805044688, + 5.220167333027348, + 5.419301706016995, + 5.622885292046703, + 5.826759812072851, + 6.027225075056776, + 6.231284238980152, + 6.434331987053156, + 6.633354714023881, + 6.831303721060976, + 7.029574619024061, + 7.230628057033755, + 7.433302247081883, + 7.631300984066911, + 7.8302245259983465, + 8.024680573027581, + 8.216211194987409, + 8.405921533005312, + 8.59747977508232, + 8.786211504018866, + 8.971854360075668, + 9.16035868099425, + 9.352085866034031, + 9.546297086053528, + 9.745374327059835, + 9.943434369983152, + 10.143292351043783, + 10.344652228057384, + 10.547886442043819, + 10.75013200007379, + 10.949421499040909, + 11.146629065973684, + 11.342996655032039, + 11.542448347085156, + 11.73964888008777, + 11.933285831008106, + 12.129182607051916, + 12.326774899032898, + 12.526105960016139, + 12.725609501008876, + 12.928559674997814, + 13.127677403041162, + 13.325596197042614, + 13.52266751008574, + 13.71977120207157, + 13.914527754066512, + 14.115262549021281, + 14.314957780065015, + 14.512911351048388, + 14.708013702067547, + 14.902203569072299, + 15.099013457074761, + 15.2962979930453, + 15.498928721062839, + 15.697073050076142, + 15.89293470303528, + 16.089272239012644, + 16.2878925150726, + 16.487954272073694, + 16.687287150067277, + 16.883613747078925, + 17.08273852802813, + 17.277411801973358, + 17.469386841985397, + 17.661561392014846, + 17.86017476907, + 18.064793438999914, + 18.272296432987787, + 18.47661006904673, + 18.683747767005116, + 18.885729038040154, + 19.085687789018266, + 19.279359410051256, + 19.471343748038635, + 19.66257169598248, + 19.85491395101417, + 20.04663590400014, + 20.236392243066803, + 20.42579118697904, + 20.61821941600647, + 20.813928239047527, + 21.012117127072997, + 21.210310110007413, + 21.40461732505355, + 21.59369026008062, + 21.781344905029982, + 21.96582219505217, + 22.147299233009107, + 22.326126219006255, + 22.49770772201009, + 22.65363627206534, + 22.79494070308283, + 22.924510844051838, + 23.035218776087277 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.20756693277508, + "p95": 205.2409250754863, + "max": 343.21089601144195 + } + }, + { + "start": 722319.963550308, + "first": 722320.123046178, + "last": 722342.461554922, + "end": 722342.461671195, + "elapsed_s": 22.498120886972174, + "chunk_offsets_s": [ + 0.15949587000068277, + 0.22017973999027163, + 0.4614580050110817, + 0.7271210729377344, + 1.0050976449856535, + 1.2981331589398906, + 1.6100660759257153, + 1.9378840889548883, + 2.2814818349434063, + 2.47131548600737, + 2.660414870013483, + 2.853334750980139, + 3.0487762360135093, + 3.237482926924713, + 3.425426849978976, + 3.6153954309411347, + 3.8099714280106127, + 4.0066974139772356, + 4.204052553977817, + 4.405206831987016, + 4.609861471923068, + 4.812860856996849, + 5.016894294996746, + 5.220388051937334, + 5.419515824993141, + 5.623528816970065, + 5.827188633964397, + 6.02791993599385, + 6.231724955956452, + 6.435552456998266, + 6.633851413964294, + 6.831146134994924, + 7.029721800005063, + 7.231122246012092, + 7.434200985007919, + 7.631723409984261, + 7.830591439967975, + 8.025399551959708, + 8.21677032194566, + 8.4065266859252, + 8.598000395926647, + 8.786678498028778, + 8.972518878988922, + 9.160750560928136, + 9.352635147981346, + 9.546895758016035, + 9.745832120999694, + 9.943731585983187, + 10.143606564030051, + 10.345498822978698, + 10.548221159959212, + 10.750928740017116, + 10.949860865948722, + 11.14751304301899, + 11.343630334013142, + 11.54306243895553, + 11.740389575017616, + 11.934054091922008, + 12.130059067974798, + 12.327584473998286, + 12.526766719995067, + 12.726259883958846, + 12.929102511960082, + 13.128578310948797, + 13.32618133793585, + 13.523330968921073, + 13.72032850398682, + 13.915400406927802, + 14.115981332957745, + 14.315525503014214, + 14.51328895392362, + 14.708800183958374, + 14.9022048630286, + 15.099282679962926, + 15.296833607950248, + 15.499818241922185, + 15.697298691025935, + 15.893634962965734, + 16.08983193000313, + 16.28927421302069, + 16.488426154945046, + 16.68739683693275, + 16.884186818031594, + 17.08304145699367, + 17.277692600968294, + 17.470439448021352, + 17.661969627952203, + 17.86089558794629, + 18.065576797001995, + 18.272221322986297, + 18.476815489004366, + 18.68379575794097, + 18.886804495938122, + 19.085947368992493, + 19.27930761897005, + 19.471458059968427, + 19.663008974981494, + 19.85595498199109, + 20.046912331017666, + 20.23688702599611, + 20.425936160958372, + 20.61830624693539, + 20.814223433029838, + 21.012280395021662, + 21.2102052239934, + 21.405136276967824, + 21.593698356999084, + 21.781594900996424, + 21.966258215950802, + 22.14802336797584, + 22.326478361035697, + 22.498004614026286 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.52540602348745, + "p95": 265.66306792665273, + "max": 343.597745988518 + } + }, + { + "start": 722319.963272116, + "first": 722322.058205247, + "last": 722343.206154697, + "end": 722343.206581043, + "elapsed_s": 23.243308926932514, + "chunk_offsets_s": [ + 2.094933131011203, + 2.2816962390206754, + 2.4720525889424607, + 2.660836050985381, + 2.8539140130160376, + 3.0492700779577717, + 3.238048424012959, + 3.4261465339222923, + 3.615930764004588, + 3.8105105829890817, + 4.007173466961831, + 4.2048178350087255, + 4.405681032920256, + 4.6103898549918085, + 4.8129327279748395, + 5.017350589972921, + 5.221097799949348, + 5.420328862965107, + 5.624087485019118, + 5.827579492935911, + 6.028420393005945, + 6.2321899509988725, + 6.4362568309297785, + 6.634315053001046, + 6.832027163007297, + 7.030768992961384, + 7.232108652009629, + 7.434180320939049, + 7.632589002023451, + 7.831394564942457, + 8.026411118917167, + 8.2168505450245, + 8.406713246018626, + 8.598230078001507, + 8.78701122791972, + 8.97260950691998, + 9.160769816022366, + 9.352868778980337, + 9.547218878986314, + 9.74615158000961, + 9.94390002998989, + 10.143797167926095, + 10.345941719016992, + 10.548435507924296, + 10.750868681934662, + 10.950190686970018, + 11.147640984971076, + 11.343811678001657, + 11.543247571913525, + 11.740432883962058, + 11.93428188492544, + 12.130153125966899, + 12.32743240299169, + 12.526995823951438, + 12.726394057972357, + 12.930329061928205, + 13.128532154951245, + 13.326389926951379, + 13.52340688998811, + 13.720564435003325, + 13.915404821978882, + 14.116086194990203, + 14.31570581998676, + 14.513498037005775, + 14.70914945192635, + 14.903522291919217, + 15.100205395952798, + 15.297671432956122, + 15.500530869932845, + 15.697953626979142, + 15.894141527009197, + 16.090857486007735, + 16.289797405945137, + 16.48908434191253, + 16.688327210955322, + 16.885566505021416, + 17.083631764980964, + 17.278264502994716, + 17.4710948249558, + 17.66259764402639, + 17.861230593989603, + 18.06564193102531, + 18.272979262983426, + 18.47767524700612, + 18.68487962591462, + 18.887441614991985, + 19.086802856996655, + 19.28015720692929, + 19.472379821003415, + 19.6640988800209, + 19.856469230959192, + 20.047444617957808, + 20.237988715991378, + 20.42707505391445, + 20.619415988912806, + 20.815250437008217, + 21.013409757986665, + 21.211221129982732, + 21.40631189395208, + 21.59446156199556, + 21.78267949691508, + 21.96721752092708, + 22.148761370917782, + 22.327421808964573, + 22.498993829009123, + 22.654968546004966, + 22.796151858987287, + 22.925800245953724, + 23.03634039591998, + 23.12704198702704, + 23.19460392300971, + 23.242882580962032 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.15754501521587, + "p95": 204.41133703570813, + "max": 207.3373319581151 + } + }, + { + "start": 722319.963619577, + "first": 722320.340260656, + "last": 722342.617514639, + "end": 722342.617634382, + "elapsed_s": 22.65401480498258, + "chunk_offsets_s": [ + 0.3766410789685324, + 0.4618020210182294, + 0.7275997749529779, + 1.0052842090371996, + 1.2987276039784774, + 1.6103673030156642, + 1.938459521974437, + 2.2814621089491993, + 2.472062121029012, + 2.660554441041313, + 2.853811922017485, + 3.049089491018094, + 3.237942656967789, + 3.4259052460547537, + 3.6157069340115413, + 3.810313211986795, + 4.006904328009114, + 4.204659361043014, + 4.404898146982305, + 4.609891757951118, + 4.81328963697888, + 5.017238292028196, + 5.221213581971824, + 5.420162896974944, + 5.623834592988715, + 5.827324185986072, + 6.02815277595073, + 6.232149202958681, + 6.435955784050748, + 6.634189525037073, + 6.832331332960166, + 7.030759842949919, + 7.231893604970537, + 7.43453878199216, + 7.632275564945303, + 7.831006109947339, + 8.02587318699807, + 8.216460427036509, + 8.406308465986513, + 8.597956805024296, + 8.786844806978479, + 8.972408022964373, + 9.16056099301204, + 9.352783521055244, + 9.546994740027003, + 9.745951425051317, + 9.943810487049632, + 10.14350063202437, + 10.345312737044878, + 10.548495401977561, + 10.750772058032453, + 10.950019313953817, + 11.147736056009308, + 11.343513992032968, + 11.543044734979048, + 11.740261289989576, + 11.934051978052594, + 12.129941478953697, + 12.327435001032427, + 12.526807075948454, + 12.726121683954261, + 12.929318050039, + 13.128356517991051, + 13.326332440017723, + 13.523109124973416, + 13.720376856043003, + 13.915102484053932, + 14.115867596003227, + 14.315550222992897, + 14.51326962094754, + 14.708655601949431, + 14.902239672956057, + 15.099308614968322, + 15.296823741984554, + 15.499581987038255, + 15.69726837403141, + 15.893497208948247, + 16.089804362971336, + 16.28888192703016, + 16.488291869987734, + 16.68725847301539, + 16.88420784601476, + 17.082917022053152, + 17.27766194799915, + 17.470648669986986, + 17.661942190025, + 17.8611412049504, + 18.065425559994765, + 18.272250651963986, + 18.47679495997727, + 18.683960833004676, + 18.887022180017084, + 19.085941503057256, + 19.27936635504011, + 19.471521834959276, + 19.663034819997847, + 19.85620303603355, + 20.04689368896652, + 20.236930573009886, + 20.42558303498663, + 20.618375763995573, + 20.814558672951534, + 21.012425789958797, + 21.210519658983685, + 21.405256727011874, + 21.593689310946502, + 21.781722659012303, + 21.96632188197691, + 22.147992062033154, + 22.326575994025916, + 22.49803597899154, + 22.653895062045194 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.7167420554906, + "p95": 265.79775393474847, + "max": 343.0025869747624 + } + }, + { + "start": 722319.964381753, + "first": 722321.418932443, + "last": 722343.090128539, + "end": 722343.090343655, + "elapsed_s": 23.125961901969276, + "chunk_offsets_s": [ + 1.4545506900176406, + 1.609168638009578, + 1.936969815986231, + 2.2805071159964427, + 2.470529909012839, + 2.660021024988964, + 2.852603673003614, + 3.0478344480507076, + 3.2366944239474833, + 3.4246999779716134, + 3.6146284790011123, + 3.8092329499777406, + 4.005759524996392, + 4.203584998962469, + 4.404206876992248, + 4.609570923028514, + 4.812127128941938, + 5.016008827020414, + 5.219657968962565, + 5.419019327964634, + 5.622783674974926, + 5.826199947041459, + 6.027184786973521, + 6.2309642740292475, + 6.435083145042881, + 6.633258912945166, + 6.830799516988918, + 7.02939833200071, + 7.230679522966966, + 7.43358154699672, + 7.631236676010303, + 7.830086222966202, + 8.024743356974795, + 8.216504037030973, + 8.406191593967378, + 8.597795148030855, + 8.786449410952628, + 8.971826839027926, + 9.160193321993575, + 9.352533261990175, + 9.54661772097461, + 9.745582752977498, + 9.942960276035592, + 10.143583646975458, + 10.344995251041837, + 10.548119283979759, + 10.750429198029451, + 10.949491395032965, + 11.146830889047123, + 11.34358196798712, + 11.542738480027765, + 11.739979051053524, + 11.933704357943498, + 12.129643714055419, + 12.327056942973286, + 12.526509108953178, + 12.725960251991637, + 12.92942188505549, + 13.127930618007667, + 13.325434479047544, + 13.52308054396417, + 13.720059675979428, + 13.914703372051008, + 14.115903118043207, + 14.31524966203142, + 14.513146693003364, + 14.707652137964033, + 14.901249735965393, + 15.098488364019431, + 15.296162254991941, + 15.498924514977261, + 15.69634479295928, + 15.89268925297074, + 16.0889523869846, + 16.288236254942603, + 16.487641929998063, + 16.686610564007424, + 16.883232502965257, + 17.082009605015628, + 17.276808931026608, + 17.469700764049776, + 17.66108312702272, + 17.85985751298722, + 18.06487423600629, + 18.272333096014336, + 18.47605614596978, + 18.683119599008933, + 18.886112313950434, + 19.08521957602352, + 19.278526369016618, + 19.4707098309882, + 19.662230123998597, + 19.85526054596994, + 20.046428287983872, + 20.2361182969762, + 20.425149793969467, + 20.617569116991945, + 20.813514943001792, + 21.011764244991355, + 21.20952615700662, + 21.404452031012625, + 21.592690082034096, + 21.780909246997908, + 21.96573257097043, + 22.147316765971482, + 22.32576253300067, + 22.49759110296145, + 22.653233887045644, + 22.79490101803094, + 22.92481012805365, + 23.035441045998596, + 23.125746786012314 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.33949401415884, + "p95": 205.01672301907092, + "max": 343.53730001021177 + } + }, + { + "start": 722319.963067971, + "first": 722320.583482987, + "last": 722342.759387451, + "end": 722342.759667747, + "elapsed_s": 22.796599775901996, + "chunk_offsets_s": [ + 0.6204150159610435, + 0.7276547899236903, + 1.0055493199033663, + 1.2986336729954928, + 1.610511595965363, + 1.938331238925457, + 2.2817854549502954, + 2.472084888955578, + 2.6616213449742645, + 2.8538685179082677, + 3.0493494279216975, + 3.2378444490022957, + 3.4260607309406623, + 3.615990094956942, + 3.810497681959532, + 4.007132236962207, + 4.204585346975364, + 4.4055863679386675, + 4.6107048749690875, + 4.8137354999780655, + 5.017457624897361, + 5.220915761892684, + 5.420137465931475, + 5.62405703892, + 5.827604524907656, + 6.02831002289895, + 6.232243169914, + 6.436336315935478, + 6.634297601995058, + 6.8317342340014875, + 7.030260622967035, + 7.231651044916362, + 7.434812204912305, + 7.632287863991223, + 7.831129799946211, + 8.025924945948645, + 8.21714417799376, + 8.406956899911165, + 8.598585315980017, + 8.787360853981227, + 8.97288870299235, + 9.161160724936053, + 9.353254749905318, + 9.54749994596932, + 9.746571687981486, + 9.944701952976175, + 10.144157412927598, + 10.345771285938099, + 10.549113746965304, + 10.751366067910567, + 10.950519098900259, + 11.147942487965338, + 11.34416685893666, + 11.543644468998536, + 11.740767341922037, + 11.934641158906743, + 12.130451512988657, + 12.328028499963693, + 12.527465699939057, + 12.726805395912379, + 12.929928800906055, + 13.129004332935438, + 13.326827346929349, + 13.523898226907477, + 13.72087264189031, + 13.915809286991134, + 14.116517170914449, + 14.316058623953722, + 14.51365958491806, + 14.70887946197763, + 14.902989825932309, + 15.100290015921928, + 15.297682370990515, + 15.500492891995236, + 15.698103309958242, + 15.89427250099834, + 16.090785400941968, + 16.28948204696644, + 16.489439803990535, + 16.6881507699145, + 16.884877716889605, + 17.083768701995723, + 17.278391190920956, + 17.47112209792249, + 17.662886778940447, + 17.861310268985108, + 18.065903545939364, + 18.273014915990643, + 18.477726329001598, + 18.684727353975177, + 18.887693892931566, + 19.086738262907602, + 19.280255492893048, + 19.472180307959206, + 19.664263126906008, + 19.856504096998833, + 20.04724731191527, + 20.23794639389962, + 20.426766222924925, + 20.619263165979646, + 20.814945714897476, + 21.013400118914433, + 21.21117568691261, + 21.40655184397474, + 21.59450136590749, + 21.782537175924517, + 21.96731076692231, + 22.148797271889634, + 22.327477761893533, + 22.499117953935638, + 22.655127786914818, + 22.796319479937665 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.43663200642914, + "p95": 207.11137005127966, + "max": 343.4542160248384 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + } + ] +} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/hostwatch-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/hostwatch-rank0.log new file mode 100644 index 0000000000..20811e2682 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/hostwatch-rank0.log @@ -0,0 +1,659 @@ +01:54:31 114 +01:54:33 114 +01:54:35 112 +01:54:37 112 +01:54:39 111 +01:54:41 110 +01:54:43 110 +01:54:45 108 +01:54:47 108 +01:54:50 107 +01:54:52 107 +01:54:54 107 +01:54:56 105 +01:54:58 105 +01:55:00 104 +01:55:02 104 +01:55:04 104 +01:55:06 102 +01:55:08 102 +01:55:10 101 +01:55:12 101 +01:55:14 101 +01:55:16 101 +01:55:18 100 +01:55:20 100 +01:55:22 98 +01:55:24 98 +01:55:26 97 +01:55:29 97 +01:55:31 97 +01:55:33 97 +01:55:35 96 +01:55:37 96 +01:55:39 94 +01:55:41 94 +01:55:43 94 +01:55:45 93 +01:55:47 93 +01:55:49 92 +01:55:51 91 +01:55:53 91 +01:55:55 90 +01:55:57 90 +01:55:59 89 +01:56:01 88 +01:56:03 88 +01:56:05 87 +01:56:08 87 +01:56:10 86 +01:56:12 85 +01:56:14 85 +01:56:16 84 +01:56:18 84 +01:56:20 83 +01:56:22 82 +01:56:24 82 +01:56:26 80 +01:56:28 80 +01:56:30 80 +01:56:32 79 +01:56:34 79 +01:56:36 79 +01:56:38 78 +01:56:40 78 +01:56:42 77 +01:56:45 77 +01:56:47 77 +01:56:49 75 +01:56:51 75 +01:56:53 74 +01:56:55 74 +01:56:57 74 +01:56:59 73 +01:57:01 73 +01:57:03 72 +01:57:05 71 +01:57:07 71 +01:57:09 70 +01:57:11 70 +01:57:13 68 +01:57:15 68 +01:57:17 68 +01:57:19 66 +01:57:22 67 +01:57:24 65 +01:57:26 65 +01:57:28 65 +01:57:30 64 +01:57:32 64 +01:57:34 64 +01:57:36 62 +01:57:38 62 +01:57:40 61 +01:57:42 61 +01:57:44 61 +01:57:46 60 +01:57:48 59 +01:57:50 60 +01:57:52 58 +01:57:54 58 +01:57:56 58 +01:57:58 57 +01:58:01 57 +01:58:03 56 +01:58:05 55 +01:58:07 55 +01:58:09 54 +01:58:11 54 +01:58:13 54 +01:58:15 52 +01:58:17 50 +01:58:19 50 +01:58:21 50 +01:58:23 50 +01:58:25 50 +01:58:27 52 +01:58:29 12 +01:58:31 12 +01:58:33 12 +01:58:35 12 +01:58:38 12 +01:58:40 12 +01:58:42 12 +01:58:44 12 +01:58:46 12 +01:58:48 12 +01:58:50 12 +01:58:52 12 +01:58:54 12 +01:58:56 12 +01:58:58 12 +01:59:00 12 +01:59:02 12 +01:59:04 12 +01:59:06 12 +01:59:09 12 +01:59:11 12 +01:59:13 12 +01:59:15 12 +01:59:17 12 +01:59:19 12 +01:59:21 12 +01:59:23 12 +01:59:25 12 +01:59:27 12 +01:59:29 12 +01:59:31 12 +01:59:33 12 +01:59:35 12 +01:59:37 12 +01:59:40 12 +01:59:42 12 +01:59:44 12 +01:59:46 12 +01:59:48 12 +01:59:50 12 +01:59:52 12 +01:59:54 12 +01:59:56 12 +01:59:58 12 +02:00:00 12 +02:00:02 12 +02:00:04 12 +02:00:06 12 +02:00:08 12 +02:00:11 12 +02:00:13 12 +02:00:15 12 +02:00:17 12 +02:00:19 12 +02:00:21 12 +02:00:23 12 +02:00:25 12 +02:00:27 12 +02:00:29 12 +02:00:31 12 +02:00:33 12 +02:00:35 12 +02:00:37 12 +02:00:39 12 +02:00:41 12 +02:00:44 12 +02:00:46 12 +02:00:48 12 +02:00:50 12 +02:00:52 12 +02:00:54 12 +02:00:56 12 +02:00:58 12 +02:01:00 12 +02:01:02 12 +02:01:04 12 +02:01:06 12 +02:01:08 12 +02:01:10 12 +02:01:12 12 +02:01:15 12 +02:01:17 12 +02:01:19 12 +02:01:21 12 +02:01:23 12 +02:01:25 12 +02:01:27 12 +02:01:29 12 +02:01:31 12 +02:01:33 12 +02:01:35 12 +02:01:37 12 +02:01:39 12 +02:01:41 12 +02:01:44 12 +02:01:46 12 +02:01:48 12 +02:01:50 12 +02:01:52 12 +02:01:54 12 +02:01:56 12 +02:01:58 12 +02:02:00 12 +02:02:02 12 +02:02:04 12 +02:02:06 12 +02:02:08 12 +02:02:10 12 +02:02:12 12 +02:02:15 12 +02:02:17 12 +02:02:19 12 +02:02:21 12 +02:02:23 12 +02:02:25 12 +02:02:27 12 +02:02:29 12 +02:02:31 12 +02:02:33 12 +02:02:35 12 +02:02:37 12 +02:02:39 12 +02:02:41 12 +02:02:43 12 +02:02:46 12 +02:02:48 12 +02:02:50 12 +02:02:52 12 +02:02:54 12 +02:02:56 12 +02:02:58 12 +02:03:00 12 +02:03:02 12 +02:03:04 12 +02:03:06 12 +02:03:08 12 +02:03:10 12 +02:03:12 12 +02:03:15 12 +02:03:17 12 +02:03:19 12 +02:03:21 12 +02:03:23 12 +02:03:25 12 +02:03:27 12 +02:03:29 12 +02:03:31 12 +02:03:33 12 +02:03:35 12 +02:03:37 12 +02:03:39 12 +02:03:41 12 +02:03:43 12 +02:03:46 12 +02:03:48 12 +02:03:50 12 +02:03:52 12 +02:03:54 12 +02:03:56 12 +02:03:58 12 +02:04:00 12 +02:04:02 12 +02:04:04 12 +02:04:06 12 +02:04:08 12 +02:04:10 12 +02:04:12 12 +02:04:14 12 +02:04:17 12 +02:04:19 12 +02:04:21 12 +02:04:23 12 +02:04:25 12 +02:04:27 12 +02:04:29 12 +02:04:31 12 +02:04:33 12 +02:04:35 12 +02:04:37 12 +02:04:39 12 +02:04:41 12 +02:04:43 12 +02:04:45 12 +02:04:48 12 +02:04:50 12 +02:04:52 12 +02:04:54 12 +02:04:56 12 +02:04:58 12 +02:05:00 12 +02:05:02 12 +02:05:04 12 +02:05:06 12 +02:05:08 12 +02:05:10 12 +02:05:12 12 +02:05:14 12 +02:05:17 12 +02:05:19 12 +02:05:21 12 +02:05:23 12 +02:05:25 12 +02:05:27 12 +02:05:29 12 +02:05:31 12 +02:05:33 12 +02:05:35 12 +02:05:37 12 +02:05:39 12 +02:05:41 12 +02:05:43 12 +02:05:45 12 +02:05:48 12 +02:05:50 12 +02:05:52 12 +02:05:54 12 +02:05:56 12 +02:05:58 12 +02:06:00 12 +02:06:02 12 +02:06:04 12 +02:06:06 12 +02:06:08 12 +02:06:10 12 +02:06:12 12 +02:06:14 12 +02:06:16 12 +02:06:19 12 +02:06:21 12 +02:06:23 12 +02:06:25 12 +02:06:27 12 +02:06:29 12 +02:06:31 12 +02:06:33 12 +02:06:35 12 +02:06:37 12 +02:06:39 12 +02:06:41 12 +02:06:43 12 +02:06:45 12 +02:06:48 12 +02:06:50 12 +02:06:52 12 +02:06:54 12 +02:06:56 12 +02:06:58 12 +02:07:00 12 +02:07:02 12 +02:07:04 12 +02:07:06 12 +02:07:08 12 +02:07:10 12 +02:07:12 12 +02:07:14 12 +02:07:16 12 +02:07:19 12 +02:07:21 12 +02:07:23 12 +02:07:25 12 +02:07:27 12 +02:07:29 12 +02:07:31 12 +02:07:33 12 +02:07:35 12 +02:07:37 12 +02:07:39 12 +02:07:41 12 +02:07:43 12 +02:07:45 12 +02:07:47 12 +02:07:49 12 +02:07:52 12 +02:07:54 12 +02:07:56 12 +02:07:58 12 +02:08:00 12 +02:08:02 12 +02:08:04 12 +02:08:06 12 +02:08:08 12 +02:08:10 12 +02:08:12 12 +02:08:14 12 +02:08:16 12 +02:08:18 12 +02:08:20 12 +02:08:22 12 +02:08:25 12 +02:08:27 12 +02:08:29 12 +02:08:31 12 +02:08:33 12 +02:08:35 12 +02:08:37 12 +02:08:39 12 +02:08:41 12 +02:08:43 12 +02:08:45 12 +02:08:47 12 +02:08:49 12 +02:08:51 12 +02:08:53 12 +02:08:55 12 +02:08:58 12 +02:09:00 12 +02:09:02 12 +02:09:04 12 +02:09:06 12 +02:09:08 12 +02:09:10 12 +02:09:12 12 +02:09:14 12 +02:09:16 12 +02:09:18 12 +02:09:20 12 +02:09:22 12 +02:09:24 12 +02:09:26 12 +02:09:29 12 +02:09:31 12 +02:09:33 12 +02:09:35 12 +02:09:37 12 +02:09:39 12 +02:09:41 12 +02:09:43 12 +02:09:45 12 +02:09:47 12 +02:09:49 12 +02:09:51 12 +02:09:53 12 +02:09:55 12 +02:09:57 12 +02:10:00 12 +02:10:02 12 +02:10:04 12 +02:10:06 12 +02:10:08 12 +02:10:10 12 +02:10:12 12 +02:10:14 12 +02:10:16 12 +02:10:18 12 +02:10:20 12 +02:10:22 12 +02:10:24 12 +02:10:26 12 +02:10:28 12 +02:10:31 12 +02:10:33 12 +02:10:35 12 +02:10:37 12 +02:10:39 12 +02:10:41 12 +02:10:43 12 +02:10:45 12 +02:10:47 12 +02:10:49 12 +02:10:51 12 +02:10:53 12 +02:10:55 12 +02:10:57 12 +02:10:59 12 +02:11:01 12 +02:11:04 12 +02:11:06 12 +02:11:08 12 +02:11:10 12 +02:11:12 12 +02:11:14 12 +02:11:16 12 +02:11:18 12 +02:11:20 12 +02:11:22 12 +02:11:24 12 +02:11:26 12 +02:11:28 12 +02:11:30 12 +02:11:32 12 +02:11:34 12 +02:11:37 12 +02:11:39 12 +02:11:41 12 +02:11:43 12 +02:11:45 12 +02:11:47 12 +02:11:49 12 +02:11:51 12 +02:11:53 12 +02:11:55 12 +02:11:57 12 +02:11:59 12 +02:12:01 12 +02:12:03 12 +02:12:05 12 +02:12:07 12 +02:12:10 12 +02:12:12 12 +02:12:14 12 +02:12:16 12 +02:12:18 12 +02:12:20 12 +02:12:22 12 +02:12:24 12 +02:12:26 12 +02:12:28 12 +02:12:30 12 +02:12:32 12 +02:12:34 12 +02:12:36 12 +02:12:38 12 +02:12:40 12 +02:12:43 12 +02:12:45 12 +02:12:47 12 +02:12:49 12 +02:12:51 12 +02:12:53 12 +02:12:55 12 +02:12:57 12 +02:12:59 12 +02:13:01 12 +02:13:03 12 +02:13:05 12 +02:13:07 12 +02:13:09 12 +02:13:11 12 +02:13:13 12 +02:13:16 12 +02:13:18 12 +02:13:20 12 +02:13:22 12 +02:13:24 12 +02:13:26 12 +02:13:28 12 +02:13:30 12 +02:13:32 12 +02:13:34 12 +02:13:36 12 +02:13:38 12 +02:13:40 12 +02:13:42 12 +02:13:44 12 +02:13:46 12 +02:13:49 12 +02:13:51 12 +02:13:53 12 +02:13:55 12 +02:13:57 12 +02:13:59 12 +02:14:01 12 +02:14:03 12 +02:14:05 12 +02:14:07 12 +02:14:09 12 +02:14:11 12 +02:14:13 12 +02:14:15 12 +02:14:17 12 +02:14:19 12 +02:14:21 12 +02:14:24 12 +02:14:26 12 +02:14:28 12 +02:14:30 12 +02:14:32 12 +02:14:34 12 +02:14:36 12 +02:14:38 12 +02:14:40 12 +02:14:42 12 +02:14:44 12 +02:14:46 12 +02:14:48 12 +02:14:50 12 +02:14:52 12 +02:14:54 12 +02:14:57 12 +02:14:59 12 +02:15:01 12 +02:15:03 12 +02:15:05 12 +02:15:07 12 +02:15:09 12 +02:15:11 12 +02:15:13 12 +02:15:15 12 +02:15:17 12 +02:15:19 12 +02:15:21 12 +02:15:23 12 +02:15:25 12 +02:15:27 12 +02:15:29 12 +02:15:32 12 +02:15:34 12 +02:15:36 12 +02:15:38 12 +02:15:40 12 +02:15:42 12 +02:15:44 12 +02:15:46 12 +02:15:48 12 +02:15:50 12 +02:15:52 12 +02:15:54 12 +02:15:56 12 +02:15:58 12 +02:16:00 12 +02:16:02 12 +02:16:05 12 +02:16:07 12 +02:16:09 12 +02:16:11 12 +02:16:13 12 +02:16:15 12 +02:16:17 12 +02:16:19 12 +02:16:21 12 +02:16:23 12 +02:16:25 12 +02:16:27 12 +02:16:29 12 +02:16:31 12 +02:16:33 12 +02:16:36 12 +02:16:38 12 +02:16:40 12 +02:16:42 12 +02:16:44 12 +02:16:46 12 +02:16:48 12 +02:16:50 12 +02:16:52 12 +02:16:54 12 +02:16:56 12 +02:16:58 12 +02:17:00 12 +02:17:02 12 +02:17:04 12 +02:17:07 78 +02:17:09 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/hostwatch-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/hostwatch-rank1.log new file mode 100644 index 0000000000..070b5f0132 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/hostwatch-rank1.log @@ -0,0 +1,657 @@ +01:54:32 114 +01:54:35 112 +01:54:37 112 +01:54:39 112 +01:54:41 110 +01:54:43 110 +01:54:45 109 +01:54:47 109 +01:54:49 109 +01:54:51 107 +01:54:53 107 +01:54:55 107 +01:54:57 106 +01:54:59 106 +01:55:01 104 +01:55:03 104 +01:55:05 104 +01:55:07 103 +01:55:09 103 +01:55:12 103 +01:55:14 101 +01:55:16 101 +01:55:18 101 +01:55:20 100 +01:55:22 100 +01:55:24 98 +01:55:26 98 +01:55:28 98 +01:55:30 97 +01:55:32 97 +01:55:34 95 +01:55:36 95 +01:55:38 95 +01:55:40 94 +01:55:42 94 +01:55:44 94 +01:55:46 93 +01:55:48 93 +01:55:51 91 +01:55:53 91 +01:55:55 90 +01:55:57 90 +01:55:59 90 +01:56:01 90 +01:56:03 88 +01:56:05 88 +01:56:07 87 +01:56:09 87 +01:56:11 86 +01:56:13 86 +01:56:15 86 +01:56:17 84 +01:56:19 84 +01:56:21 84 +01:56:23 83 +01:56:25 83 +01:56:28 81 +01:56:30 82 +01:56:32 81 +01:56:34 80 +01:56:36 80 +01:56:38 78 +01:56:40 78 +01:56:42 78 +01:56:44 77 +01:56:46 77 +01:56:48 75 +01:56:50 75 +01:56:52 74 +01:56:54 73 +01:56:56 73 +01:56:58 73 +01:57:00 72 +01:57:02 72 +01:57:05 71 +01:57:07 71 +01:57:09 70 +01:57:11 69 +01:57:13 69 +01:57:15 69 +01:57:17 68 +01:57:19 68 +01:57:21 66 +01:57:23 66 +01:57:25 66 +01:57:27 65 +01:57:29 65 +01:57:31 65 +01:57:33 63 +01:57:35 63 +01:57:37 62 +01:57:39 62 +01:57:42 61 +01:57:44 60 +01:57:46 60 +01:57:48 59 +01:57:50 59 +01:57:52 59 +01:57:54 58 +01:57:56 58 +01:57:58 56 +01:58:00 56 +01:58:02 56 +01:58:04 55 +01:58:06 55 +01:58:08 54 +01:58:10 54 +01:58:12 53 +01:58:14 52 +01:58:16 50 +01:58:19 50 +01:58:21 50 +01:58:23 50 +01:58:25 50 +01:58:27 50 +01:58:29 11 +01:58:31 11 +01:58:33 11 +01:58:35 11 +01:58:37 12 +01:58:39 12 +01:58:41 12 +01:58:43 12 +01:58:45 12 +01:58:47 12 +01:58:49 12 +01:58:52 12 +01:58:54 12 +01:58:56 12 +01:58:58 12 +01:59:00 12 +01:59:02 12 +01:59:04 12 +01:59:06 12 +01:59:08 12 +01:59:10 12 +01:59:12 12 +01:59:14 12 +01:59:16 12 +01:59:18 12 +01:59:20 12 +01:59:23 12 +01:59:25 12 +01:59:27 12 +01:59:29 12 +01:59:31 12 +01:59:33 12 +01:59:35 12 +01:59:37 12 +01:59:39 12 +01:59:41 12 +01:59:43 12 +01:59:45 12 +01:59:47 12 +01:59:49 12 +01:59:51 12 +01:59:54 12 +01:59:56 12 +01:59:58 12 +02:00:00 12 +02:00:02 12 +02:00:04 12 +02:00:06 12 +02:00:08 12 +02:00:10 12 +02:00:12 12 +02:00:14 12 +02:00:16 12 +02:00:18 12 +02:00:20 12 +02:00:22 12 +02:00:25 12 +02:00:27 12 +02:00:29 12 +02:00:31 12 +02:00:33 12 +02:00:35 12 +02:00:37 12 +02:00:39 12 +02:00:41 12 +02:00:43 12 +02:00:45 12 +02:00:47 12 +02:00:49 12 +02:00:51 12 +02:00:53 12 +02:00:56 12 +02:00:58 12 +02:01:00 12 +02:01:02 12 +02:01:04 12 +02:01:06 12 +02:01:08 12 +02:01:10 12 +02:01:12 12 +02:01:14 12 +02:01:16 12 +02:01:18 12 +02:01:20 12 +02:01:22 12 +02:01:25 12 +02:01:27 12 +02:01:29 12 +02:01:31 12 +02:01:33 12 +02:01:35 12 +02:01:37 12 +02:01:39 12 +02:01:41 12 +02:01:43 12 +02:01:45 12 +02:01:47 12 +02:01:49 12 +02:01:51 12 +02:01:53 12 +02:01:56 12 +02:01:58 12 +02:02:00 12 +02:02:02 12 +02:02:04 12 +02:02:06 12 +02:02:08 12 +02:02:10 12 +02:02:12 12 +02:02:14 12 +02:02:16 12 +02:02:18 12 +02:02:20 12 +02:02:22 12 +02:02:24 12 +02:02:27 12 +02:02:29 12 +02:02:31 12 +02:02:33 12 +02:02:35 12 +02:02:37 12 +02:02:39 12 +02:02:41 12 +02:02:43 12 +02:02:45 12 +02:02:47 12 +02:02:49 12 +02:02:51 12 +02:02:53 12 +02:02:55 12 +02:02:58 12 +02:03:00 12 +02:03:02 12 +02:03:04 12 +02:03:06 12 +02:03:08 12 +02:03:10 12 +02:03:12 12 +02:03:14 12 +02:03:16 12 +02:03:18 12 +02:03:20 12 +02:03:22 12 +02:03:24 12 +02:03:26 12 +02:03:29 12 +02:03:31 12 +02:03:33 12 +02:03:35 12 +02:03:37 12 +02:03:39 12 +02:03:41 12 +02:03:43 12 +02:03:45 12 +02:03:47 12 +02:03:49 12 +02:03:51 12 +02:03:53 12 +02:03:55 12 +02:03:57 12 +02:04:00 12 +02:04:02 12 +02:04:04 12 +02:04:06 12 +02:04:08 12 +02:04:10 12 +02:04:12 12 +02:04:14 12 +02:04:16 12 +02:04:18 12 +02:04:20 12 +02:04:22 12 +02:04:24 12 +02:04:26 12 +02:04:28 12 +02:04:31 12 +02:04:33 12 +02:04:35 12 +02:04:37 12 +02:04:39 12 +02:04:41 12 +02:04:43 12 +02:04:45 12 +02:04:47 12 +02:04:49 12 +02:04:51 12 +02:04:53 12 +02:04:55 12 +02:04:57 12 +02:05:00 12 +02:05:02 12 +02:05:04 12 +02:05:06 12 +02:05:08 12 +02:05:10 12 +02:05:12 12 +02:05:14 12 +02:05:16 12 +02:05:18 12 +02:05:20 12 +02:05:22 12 +02:05:24 12 +02:05:26 12 +02:05:28 12 +02:05:31 12 +02:05:33 12 +02:05:35 12 +02:05:37 12 +02:05:39 12 +02:05:41 12 +02:05:43 12 +02:05:45 12 +02:05:47 12 +02:05:49 12 +02:05:51 12 +02:05:53 12 +02:05:55 12 +02:05:57 12 +02:05:59 12 +02:06:02 12 +02:06:04 12 +02:06:06 12 +02:06:08 12 +02:06:10 12 +02:06:12 12 +02:06:14 12 +02:06:16 12 +02:06:18 12 +02:06:20 12 +02:06:22 12 +02:06:24 12 +02:06:26 12 +02:06:28 12 +02:06:30 12 +02:06:33 12 +02:06:35 12 +02:06:37 12 +02:06:39 12 +02:06:41 12 +02:06:43 12 +02:06:45 12 +02:06:47 12 +02:06:49 12 +02:06:51 12 +02:06:53 12 +02:06:55 12 +02:06:57 12 +02:06:59 12 +02:07:02 12 +02:07:04 12 +02:07:06 12 +02:07:08 12 +02:07:10 12 +02:07:12 12 +02:07:14 12 +02:07:16 12 +02:07:18 12 +02:07:20 12 +02:07:22 12 +02:07:24 12 +02:07:26 12 +02:07:28 12 +02:07:30 12 +02:07:32 12 +02:07:35 12 +02:07:37 12 +02:07:39 12 +02:07:41 12 +02:07:43 12 +02:07:45 12 +02:07:47 12 +02:07:49 12 +02:07:51 12 +02:07:53 12 +02:07:55 12 +02:07:57 12 +02:07:59 12 +02:08:01 12 +02:08:03 12 +02:08:05 12 +02:08:08 12 +02:08:10 12 +02:08:12 12 +02:08:14 12 +02:08:16 12 +02:08:18 12 +02:08:20 12 +02:08:22 12 +02:08:24 12 +02:08:26 12 +02:08:28 12 +02:08:30 12 +02:08:32 12 +02:08:34 12 +02:08:36 12 +02:08:39 12 +02:08:41 12 +02:08:43 12 +02:08:45 12 +02:08:47 12 +02:08:49 12 +02:08:51 12 +02:08:53 12 +02:08:55 12 +02:08:57 12 +02:08:59 12 +02:09:01 12 +02:09:03 12 +02:09:05 12 +02:09:07 12 +02:09:10 12 +02:09:12 12 +02:09:14 12 +02:09:16 12 +02:09:18 12 +02:09:20 12 +02:09:22 12 +02:09:24 12 +02:09:26 12 +02:09:28 12 +02:09:30 12 +02:09:32 12 +02:09:34 12 +02:09:36 12 +02:09:38 12 +02:09:41 12 +02:09:43 12 +02:09:45 12 +02:09:47 12 +02:09:49 12 +02:09:51 12 +02:09:53 12 +02:09:55 12 +02:09:57 12 +02:09:59 12 +02:10:01 12 +02:10:03 12 +02:10:05 12 +02:10:07 12 +02:10:09 12 +02:10:11 12 +02:10:14 12 +02:10:16 12 +02:10:18 12 +02:10:20 12 +02:10:22 12 +02:10:24 12 +02:10:26 12 +02:10:28 12 +02:10:30 12 +02:10:32 12 +02:10:34 12 +02:10:36 12 +02:10:38 12 +02:10:40 12 +02:10:42 12 +02:10:45 12 +02:10:47 12 +02:10:49 12 +02:10:51 12 +02:10:53 12 +02:10:55 12 +02:10:57 12 +02:10:59 12 +02:11:01 12 +02:11:03 12 +02:11:05 12 +02:11:07 12 +02:11:09 12 +02:11:11 12 +02:11:13 12 +02:11:15 12 +02:11:18 12 +02:11:20 12 +02:11:22 12 +02:11:24 12 +02:11:26 12 +02:11:28 12 +02:11:30 12 +02:11:32 12 +02:11:34 12 +02:11:36 12 +02:11:38 12 +02:11:40 12 +02:11:42 12 +02:11:44 12 +02:11:46 12 +02:11:49 12 +02:11:51 12 +02:11:53 12 +02:11:55 12 +02:11:57 12 +02:11:59 12 +02:12:01 12 +02:12:03 12 +02:12:05 12 +02:12:07 12 +02:12:09 12 +02:12:11 12 +02:12:13 12 +02:12:15 12 +02:12:17 12 +02:12:19 12 +02:12:22 12 +02:12:24 12 +02:12:26 12 +02:12:28 12 +02:12:30 12 +02:12:32 12 +02:12:34 12 +02:12:36 12 +02:12:38 12 +02:12:40 12 +02:12:42 12 +02:12:44 12 +02:12:46 12 +02:12:48 12 +02:12:50 12 +02:12:52 12 +02:12:54 12 +02:12:57 12 +02:12:59 12 +02:13:01 12 +02:13:03 12 +02:13:05 12 +02:13:07 12 +02:13:09 12 +02:13:11 12 +02:13:13 12 +02:13:15 12 +02:13:17 12 +02:13:19 12 +02:13:21 12 +02:13:23 12 +02:13:25 12 +02:13:27 12 +02:13:30 12 +02:13:32 12 +02:13:34 12 +02:13:36 12 +02:13:38 12 +02:13:40 12 +02:13:42 12 +02:13:44 12 +02:13:46 12 +02:13:48 12 +02:13:50 12 +02:13:52 12 +02:13:54 12 +02:13:56 12 +02:13:58 12 +02:14:00 12 +02:14:03 12 +02:14:05 12 +02:14:07 12 +02:14:09 12 +02:14:11 12 +02:14:13 12 +02:14:15 12 +02:14:17 12 +02:14:19 12 +02:14:21 12 +02:14:23 12 +02:14:25 12 +02:14:27 12 +02:14:29 12 +02:14:31 12 +02:14:33 12 +02:14:35 12 +02:14:38 12 +02:14:40 12 +02:14:42 12 +02:14:44 12 +02:14:46 12 +02:14:48 12 +02:14:50 12 +02:14:52 12 +02:14:54 12 +02:14:56 12 +02:14:58 12 +02:15:00 12 +02:15:02 12 +02:15:04 12 +02:15:06 12 +02:15:08 12 +02:15:10 12 +02:15:13 12 +02:15:15 12 +02:15:17 12 +02:15:19 12 +02:15:21 12 +02:15:23 12 +02:15:25 12 +02:15:27 12 +02:15:29 12 +02:15:31 12 +02:15:33 12 +02:15:35 12 +02:15:37 12 +02:15:39 12 +02:15:41 12 +02:15:43 12 +02:15:46 12 +02:15:48 12 +02:15:50 12 +02:15:52 12 +02:15:54 12 +02:15:56 12 +02:15:58 12 +02:16:00 12 +02:16:02 12 +02:16:04 12 +02:16:06 12 +02:16:08 12 +02:16:10 12 +02:16:12 12 +02:16:14 12 +02:16:16 12 +02:16:18 12 +02:16:21 12 +02:16:23 12 +02:16:25 12 +02:16:27 12 +02:16:29 12 +02:16:31 12 +02:16:33 12 +02:16:35 12 +02:16:37 12 +02:16:39 12 +02:16:41 12 +02:16:43 12 +02:16:45 12 +02:16:47 12 +02:16:49 12 +02:16:52 12 +02:16:54 12 +02:16:56 12 +02:16:58 12 +02:17:00 12 +02:17:02 12 +02:17:04 12 +02:17:06 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/hostwatch-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/hostwatch-rank2.log new file mode 100644 index 0000000000..dc44b8bf20 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/hostwatch-rank2.log @@ -0,0 +1,657 @@ +01:54:33 112 +01:54:35 111 +01:54:37 111 +01:54:39 109 +01:54:41 110 +01:54:43 108 +01:54:46 108 +01:54:48 108 +01:54:50 107 +01:54:52 107 +01:54:54 106 +01:54:56 106 +01:54:58 105 +01:55:00 104 +01:55:02 104 +01:55:04 103 +01:55:06 103 +01:55:08 103 +01:55:10 102 +01:55:12 102 +01:55:14 100 +01:55:16 100 +01:55:18 100 +01:55:20 99 +01:55:23 99 +01:55:25 98 +01:55:27 98 +01:55:29 97 +01:55:31 96 +01:55:33 96 +01:55:35 95 +01:55:37 95 +01:55:39 94 +01:55:41 94 +01:55:43 93 +01:55:45 93 +01:55:47 93 +01:55:49 91 +01:55:51 91 +01:55:53 91 +01:55:55 90 +01:55:57 90 +01:55:59 89 +01:56:02 89 +01:56:04 88 +01:56:06 87 +01:56:08 87 +01:56:10 86 +01:56:12 86 +01:56:14 85 +01:56:16 85 +01:56:18 84 +01:56:20 84 +01:56:22 84 +01:56:24 82 +01:56:26 82 +01:56:28 81 +01:56:30 81 +01:56:32 80 +01:56:34 80 +01:56:37 80 +01:56:39 78 +01:56:41 78 +01:56:43 77 +01:56:45 77 +01:56:47 76 +01:56:49 76 +01:56:51 76 +01:56:53 74 +01:56:55 74 +01:56:57 73 +01:56:59 73 +01:57:01 73 +01:57:03 73 +01:57:05 72 +01:57:07 72 +01:57:09 70 +01:57:11 70 +01:57:14 69 +01:57:16 69 +01:57:18 69 +01:57:20 68 +01:57:22 68 +01:57:24 67 +01:57:26 66 +01:57:28 66 +01:57:30 65 +01:57:32 65 +01:57:34 65 +01:57:36 64 +01:57:38 64 +01:57:40 64 +01:57:42 63 +01:57:44 62 +01:57:46 61 +01:57:48 61 +01:57:51 60 +01:57:53 60 +01:57:55 60 +01:57:57 59 +01:57:59 59 +01:58:01 57 +01:58:03 55 +01:58:05 55 +01:58:07 55 +01:58:09 55 +01:58:11 55 +01:58:13 55 +01:58:15 42 +01:58:17 42 +01:58:19 42 +01:58:21 42 +01:58:23 42 +01:58:25 42 +01:58:28 42 +01:58:30 17 +01:58:32 17 +01:58:34 17 +01:58:36 17 +01:58:38 17 +01:58:40 17 +01:58:42 17 +01:58:44 17 +01:58:46 17 +01:58:48 17 +01:58:50 17 +01:58:52 17 +01:58:54 17 +01:58:56 17 +01:58:58 17 +01:59:01 17 +01:59:03 17 +01:59:05 17 +01:59:07 17 +01:59:09 17 +01:59:11 17 +01:59:13 17 +01:59:15 17 +01:59:17 17 +01:59:19 17 +01:59:21 17 +01:59:23 17 +01:59:25 17 +01:59:27 17 +01:59:29 17 +01:59:32 17 +01:59:34 17 +01:59:36 17 +01:59:38 17 +01:59:40 17 +01:59:42 17 +01:59:44 17 +01:59:46 17 +01:59:48 17 +01:59:50 17 +01:59:52 17 +01:59:54 17 +01:59:56 17 +01:59:58 17 +02:00:00 17 +02:00:03 17 +02:00:05 17 +02:00:07 17 +02:00:09 17 +02:00:11 17 +02:00:13 17 +02:00:15 17 +02:00:17 17 +02:00:19 17 +02:00:21 17 +02:00:23 17 +02:00:25 17 +02:00:27 17 +02:00:29 17 +02:00:32 17 +02:00:34 17 +02:00:36 17 +02:00:38 17 +02:00:40 17 +02:00:42 17 +02:00:44 17 +02:00:46 17 +02:00:48 17 +02:00:50 17 +02:00:52 17 +02:00:54 17 +02:00:56 17 +02:00:58 17 +02:01:00 17 +02:01:03 17 +02:01:05 17 +02:01:07 17 +02:01:09 17 +02:01:11 17 +02:01:13 17 +02:01:15 17 +02:01:17 17 +02:01:19 17 +02:01:21 17 +02:01:23 17 +02:01:25 17 +02:01:27 17 +02:01:29 17 +02:01:32 17 +02:01:34 17 +02:01:36 17 +02:01:38 17 +02:01:40 17 +02:01:42 17 +02:01:44 17 +02:01:46 17 +02:01:48 17 +02:01:50 17 +02:01:52 17 +02:01:54 17 +02:01:56 17 +02:01:58 17 +02:02:00 17 +02:02:03 17 +02:02:05 17 +02:02:07 17 +02:02:09 17 +02:02:11 17 +02:02:13 17 +02:02:15 17 +02:02:17 17 +02:02:19 17 +02:02:21 17 +02:02:23 17 +02:02:25 17 +02:02:27 17 +02:02:29 17 +02:02:32 17 +02:02:34 17 +02:02:36 17 +02:02:38 17 +02:02:40 17 +02:02:42 17 +02:02:44 17 +02:02:46 17 +02:02:48 17 +02:02:50 17 +02:02:52 17 +02:02:54 17 +02:02:56 17 +02:02:58 17 +02:03:00 17 +02:03:03 17 +02:03:05 17 +02:03:07 17 +02:03:09 17 +02:03:11 17 +02:03:13 17 +02:03:15 17 +02:03:17 17 +02:03:19 17 +02:03:21 17 +02:03:23 17 +02:03:25 17 +02:03:27 17 +02:03:29 17 +02:03:31 17 +02:03:34 17 +02:03:36 17 +02:03:38 17 +02:03:40 17 +02:03:42 17 +02:03:44 17 +02:03:46 17 +02:03:48 17 +02:03:50 17 +02:03:52 17 +02:03:54 17 +02:03:56 17 +02:03:58 17 +02:04:00 17 +02:04:02 17 +02:04:04 17 +02:04:07 17 +02:04:09 17 +02:04:11 17 +02:04:13 17 +02:04:15 17 +02:04:17 17 +02:04:19 17 +02:04:21 17 +02:04:23 17 +02:04:25 17 +02:04:27 17 +02:04:29 17 +02:04:31 17 +02:04:33 17 +02:04:36 17 +02:04:38 17 +02:04:40 17 +02:04:42 17 +02:04:44 17 +02:04:46 17 +02:04:48 17 +02:04:50 17 +02:04:52 17 +02:04:54 17 +02:04:56 17 +02:04:58 17 +02:05:00 17 +02:05:02 17 +02:05:04 17 +02:05:07 17 +02:05:09 17 +02:05:11 17 +02:05:13 17 +02:05:15 17 +02:05:17 17 +02:05:19 17 +02:05:21 17 +02:05:23 17 +02:05:25 17 +02:05:27 17 +02:05:29 17 +02:05:31 17 +02:05:33 17 +02:05:36 17 +02:05:38 17 +02:05:40 17 +02:05:42 17 +02:05:44 17 +02:05:46 17 +02:05:48 17 +02:05:50 17 +02:05:52 17 +02:05:54 17 +02:05:56 17 +02:05:58 17 +02:06:00 17 +02:06:02 17 +02:06:04 17 +02:06:07 17 +02:06:09 17 +02:06:11 17 +02:06:13 17 +02:06:15 17 +02:06:17 17 +02:06:19 17 +02:06:21 17 +02:06:23 17 +02:06:25 17 +02:06:27 17 +02:06:29 17 +02:06:31 17 +02:06:33 17 +02:06:35 17 +02:06:38 17 +02:06:40 17 +02:06:42 17 +02:06:44 17 +02:06:46 17 +02:06:48 17 +02:06:50 17 +02:06:52 17 +02:06:54 17 +02:06:56 17 +02:06:58 17 +02:07:00 17 +02:07:02 17 +02:07:04 17 +02:07:07 17 +02:07:09 17 +02:07:11 17 +02:07:13 17 +02:07:15 17 +02:07:17 17 +02:07:19 17 +02:07:21 17 +02:07:23 17 +02:07:25 17 +02:07:27 17 +02:07:29 17 +02:07:31 17 +02:07:33 17 +02:07:35 17 +02:07:37 17 +02:07:40 17 +02:07:42 17 +02:07:44 17 +02:07:46 17 +02:07:48 17 +02:07:50 17 +02:07:52 17 +02:07:54 17 +02:07:56 17 +02:07:58 17 +02:08:00 17 +02:08:02 17 +02:08:04 17 +02:08:06 17 +02:08:08 17 +02:08:11 17 +02:08:13 17 +02:08:15 17 +02:08:17 17 +02:08:19 17 +02:08:21 17 +02:08:23 17 +02:08:25 17 +02:08:27 17 +02:08:29 17 +02:08:31 17 +02:08:33 17 +02:08:35 17 +02:08:37 17 +02:08:39 17 +02:08:41 17 +02:08:44 17 +02:08:46 17 +02:08:48 17 +02:08:50 17 +02:08:52 17 +02:08:54 17 +02:08:56 17 +02:08:58 17 +02:09:00 17 +02:09:02 17 +02:09:04 17 +02:09:06 17 +02:09:08 17 +02:09:10 17 +02:09:12 17 +02:09:15 17 +02:09:17 17 +02:09:19 17 +02:09:21 17 +02:09:23 17 +02:09:25 17 +02:09:27 17 +02:09:29 17 +02:09:31 17 +02:09:33 17 +02:09:35 17 +02:09:37 17 +02:09:39 17 +02:09:41 17 +02:09:43 17 +02:09:45 17 +02:09:48 17 +02:09:50 17 +02:09:52 17 +02:09:54 17 +02:09:56 17 +02:09:58 17 +02:10:00 17 +02:10:02 17 +02:10:04 17 +02:10:06 17 +02:10:08 17 +02:10:10 17 +02:10:12 17 +02:10:14 17 +02:10:17 17 +02:10:19 17 +02:10:21 17 +02:10:23 17 +02:10:25 17 +02:10:27 17 +02:10:29 17 +02:10:31 17 +02:10:33 17 +02:10:35 17 +02:10:37 17 +02:10:39 17 +02:10:41 17 +02:10:44 17 +02:10:46 17 +02:10:48 17 +02:10:50 17 +02:10:52 17 +02:10:54 17 +02:10:56 17 +02:10:58 17 +02:11:00 17 +02:11:02 17 +02:11:04 17 +02:11:06 17 +02:11:08 17 +02:11:10 17 +02:11:12 17 +02:11:15 17 +02:11:17 17 +02:11:19 17 +02:11:21 17 +02:11:23 17 +02:11:25 17 +02:11:27 17 +02:11:29 17 +02:11:31 17 +02:11:33 17 +02:11:35 17 +02:11:37 17 +02:11:39 17 +02:11:41 17 +02:11:43 17 +02:11:45 17 +02:11:48 17 +02:11:50 17 +02:11:52 17 +02:11:54 17 +02:11:56 17 +02:11:58 17 +02:12:00 17 +02:12:02 17 +02:12:04 17 +02:12:06 17 +02:12:08 17 +02:12:10 17 +02:12:12 17 +02:12:14 17 +02:12:16 17 +02:12:18 17 +02:12:20 17 +02:12:23 17 +02:12:25 17 +02:12:27 17 +02:12:29 17 +02:12:31 17 +02:12:33 17 +02:12:35 17 +02:12:37 17 +02:12:39 17 +02:12:41 17 +02:12:43 17 +02:12:45 17 +02:12:47 17 +02:12:49 17 +02:12:51 17 +02:12:53 17 +02:12:56 17 +02:12:58 17 +02:13:00 17 +02:13:02 17 +02:13:04 17 +02:13:06 17 +02:13:08 17 +02:13:10 17 +02:13:12 17 +02:13:14 17 +02:13:16 17 +02:13:18 17 +02:13:20 17 +02:13:22 17 +02:13:24 17 +02:13:26 17 +02:13:29 17 +02:13:31 17 +02:13:33 17 +02:13:35 17 +02:13:37 17 +02:13:39 17 +02:13:41 17 +02:13:43 17 +02:13:45 17 +02:13:47 17 +02:13:49 17 +02:13:51 17 +02:13:53 17 +02:13:55 17 +02:13:57 17 +02:13:59 17 +02:14:01 17 +02:14:04 17 +02:14:06 17 +02:14:08 17 +02:14:10 17 +02:14:12 17 +02:14:14 17 +02:14:16 17 +02:14:18 17 +02:14:20 17 +02:14:22 17 +02:14:24 17 +02:14:26 17 +02:14:28 17 +02:14:30 17 +02:14:32 17 +02:14:34 17 +02:14:37 17 +02:14:39 17 +02:14:41 17 +02:14:43 17 +02:14:45 17 +02:14:47 17 +02:14:49 17 +02:14:51 17 +02:14:53 17 +02:14:55 17 +02:14:57 17 +02:14:59 17 +02:15:01 17 +02:15:03 17 +02:15:05 17 +02:15:07 17 +02:15:10 17 +02:15:12 17 +02:15:14 17 +02:15:16 17 +02:15:18 17 +02:15:20 17 +02:15:22 17 +02:15:24 17 +02:15:26 17 +02:15:28 17 +02:15:30 17 +02:15:32 17 +02:15:34 17 +02:15:36 17 +02:15:38 17 +02:15:40 17 +02:15:43 17 +02:15:45 17 +02:15:47 17 +02:15:49 17 +02:15:51 17 +02:15:53 17 +02:15:55 17 +02:15:57 17 +02:15:59 17 +02:16:01 17 +02:16:03 17 +02:16:05 17 +02:16:07 17 +02:16:09 17 +02:16:11 17 +02:16:14 17 +02:16:16 17 +02:16:18 17 +02:16:20 17 +02:16:22 17 +02:16:24 17 +02:16:26 17 +02:16:28 17 +02:16:30 17 +02:16:32 17 +02:16:34 17 +02:16:36 17 +02:16:38 17 +02:16:40 17 +02:16:42 17 +02:16:44 17 +02:16:47 17 +02:16:49 17 +02:16:51 17 +02:16:53 17 +02:16:55 17 +02:16:57 17 +02:16:59 17 +02:17:01 17 +02:17:03 17 +02:17:05 17 +02:17:07 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/long-beside-short.json b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/long-beside-short.json new file mode 100644 index 0000000000..cfb7bddf6f --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/long-beside-short.json @@ -0,0 +1 @@ +{"alone": [{"prompt": "p2048-0-0", "seed": 1000, "sent": 722876.721980331, "first": 722877.974724726, "last": 722881.504443072, "tokens": 128, "pieces": [[722877.974724726, 4], [722878.035163603, 8], [722878.096963241, 13], [722878.156195263, 7], [722878.208501543, 11], [722878.263299271, 8], [722878.316718913, 14], [722878.370832998, 4], [722878.425703274, 4], [722878.48576657, 19], [722878.545330578, 22], [722878.605124198, 6], [722878.666662153, 5], [722878.725337171, 12], [722878.777439888, 12], [722878.830831969, 10], [722878.884813705, 7], [722878.937997988, 9], [722878.99203864, 4], [722879.04552488, 13], [722879.097755817, 4], [722879.15365721, 10], [722879.216512585, 12], [722879.279971623, 13], [722879.342894337, 7], [722879.405391531, 13], [722879.470091023, 14], [722879.539102969, 4], [722879.607711227, 14], [722879.670729557, 12], [722879.725047209, 10], [722879.784739835, 9], [722879.845943038, 15], [722879.902291202, 3], [722879.954245909, 11], [722880.00695351, 2], [722880.058529544, 6], [722880.111835861, 15], [722880.164699163, 3], [722880.217903813, 17], [722880.277671263, 7], [722880.336319676, 18], [722880.397152736, 6], [722880.457091172, 10], [722880.524369016, 21], [722880.590540132, 6], [722880.658730094, 9], [722880.724684448, 4], [722880.78598826, 11], [722880.844164593, 10], [722880.905640239, 10], [722880.964558728, 13], [722881.023538309, 7], [722881.075333559, 8], [722881.130713468, 9], [722881.189996753, 8], [722881.242012174, 1], [722881.294555502, 13], [722881.347708707, 19], [722881.400986562, 9], [722881.453640733, 13], [722881.504443072, 8]], "completed": 722881.504564009, "ttft_s": 1.2527443950530142, "decode_tps": 35.98020792374875, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "7d62171119f8", "token_sha256": "7d62171119f86883ec9ea4e3542846c00509f1ad0baba0a0676ccd3b8e14cd56", "output_sha256": "ac406cb14fdb49d5cf4a0fcac1a4d3e1ea135c258abd0ad94ead39fc0308e2c7", "elapsed_s": 4.782583678024821, "chunk_offsets_s": [1.2527443950530142, 1.3131832720246166, 1.3749829099979252, 1.4342149320291355, 1.4865212120348588, 1.541318939998746, 1.5947385820327327, 1.6488526669563726, 1.7037229429697618, 1.763786238967441, 1.8233502469956875, 1.88314386701677, 1.944681821973063, 2.0033568399958313, 2.055459556984715, 2.1088516380405053, 2.162833373993635, 2.2160176569595933, 2.27005830896087, 2.32354454905726, 2.3757754859980196, 2.431676878943108, 2.4945322539424524, 2.5579912919783965, 2.6209140060236678, 2.6834111999487504, 2.7481106920167804, 2.8171226379927248, 2.88573089602869, 2.948749226052314, 3.003066877950914, 3.0627595039550215, 3.123962707002647, 3.180310870986432, 3.2322655780008063, 3.2849731789901853, 3.3365492130396888, 3.3898555299965665, 3.4427188320551068, 3.495923481998034, 3.555690931971185, 3.6143393450183794, 3.6751724049681798, 3.735110840993002, 3.8023886850569397, 3.868559801019728, 3.9367497629718855, 4.0027041169814765, 4.064007928944193, 4.122184262028895, 4.183659908012487, 4.242578397039324, 4.301557978033088, 4.353353227954358, 4.408733136951923, 4.468016422004439, 4.5200318429851905, 4.572575171012431, 4.625728375976905, 4.679006231017411, 4.731660402030684, 4.782462741015479], "chunk_gaps_ms": {"count": 61, "p50": 58.64841304719448, "p95": 67.27784406393766, "max": 69.0119459759444}}, {"prompt": "p2048-1-0", "seed": 1001, "sent": 722881.505101409, "first": 722882.750566079, "last": 722886.596116118, "tokens": 128, "pieces": [[722882.750566079, 3], [722882.809522329, 17], [722882.872500323, 6], [722882.929714889, 3], [722882.984700356, 10], [722883.038666067, 10], [722883.097478843, 19], [722883.158265479, 6], [722883.219626984, 9], [722883.283470621, 22], [722883.351716481, 18], [722883.417635271, 1], [722883.470998089, 4], [722883.523949918, 10], [722883.575597428, 7], [722883.628265032, 16], [722883.683846724, 5], [722883.743474492, 10], [722883.796973005, 11], [722883.852298176, 3], [722883.905134686, 5], [722883.957939938, 9], [722884.00987553, 6], [722884.062313087, 7], [722884.115130823, 3], [722884.168630648, 12], [722884.221722318, 7], [722884.27461593, 7], [722884.327871304, 8], [722884.38104465, 6], [722884.434488009, 7], [722884.489371463, 9], [722884.551326261, 13], [722884.612860179, 8], [722884.674628977, 13], [722884.726599269, 4], [722884.778799576, 5], [722884.831929675, 6], [722884.885032693, 8], [722884.936625847, 4], [722884.990209719, 8], [722885.043176257, 3], [722885.096614366, 10], [722885.149639005, 5], [722885.203890466, 14], [722885.257343028, 10], [722885.30974647, 8], [722885.364141948, 6], [722885.417982451, 5], [722885.471223067, 6], [722885.525070276, 7], [722885.577944158, 8], [722885.630766229, 6], [722885.684945969, 5], [722885.745023711, 11], [722885.802260297, 4], [722885.854584149, 7], [722885.907524533, 6], [722885.960803563, 5], [722886.0145694, 4], [722886.06681801, 16], [722886.120685434, 5], [722886.173205017, 9], [722886.225268123, 7], [722886.280025196, 10], [722886.33286391, 6], [722886.386544932, 13], [722886.439034006, 11], [722886.492811541, 7], [722886.54552538, 8], [722886.596116118, 6]], "completed": 722886.59623339, "ttft_s": 1.2454646700061858, "decode_tps": 33.02518461938281, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "c948b4bcbd58", "token_sha256": "c948b4bcbd5833ccd9c3bbd0f64e960f0ed36def137ba411ed3ade9437416854", "output_sha256": "78dbe5350838766da1d257beb3649f97e5ed2aab47e7249e7dcefae81dba7cef", "elapsed_s": 5.091131980996579, "chunk_offsets_s": [1.2454646700061858, 1.304420919972472, 1.367398913949728, 1.4246134799905121, 1.4795989469857886, 1.5335646579042077, 1.592377433902584, 1.6531640699831769, 1.7145255749346688, 1.7783692119410262, 1.8466150719905272, 1.9125338619342074, 1.9658966799033806, 2.0188485089456663, 2.070496018975973, 2.1231636229204014, 2.1787453149445355, 2.2383730829460546, 2.291871596011333, 2.347196766990237, 2.4000332769937813, 2.4528385289013386, 2.5047741209855303, 2.5572116780094802, 2.6100294139469042, 2.6635292390128598, 2.7166209089336917, 2.769514521001838, 2.8227698949631304, 2.8759432409424335, 2.92938659992069, 2.9842700539156795, 3.046224851976149, 3.107758769998327, 3.169527567923069, 3.2214978599222377, 3.2736981669440866, 3.3268282659118995, 3.379931283998303, 3.431524437968619, 3.48510831000749, 3.5380748480092734, 3.5915129570057616, 3.644537595915608, 3.6987890569726005, 3.7522416189312935, 3.8046450609108433, 3.859040539013222, 3.912881041993387, 3.9661216579843313, 4.019968866952695, 4.072842748952098, 4.125664819963276, 4.179844559985213, 4.239922301960178, 4.297158887959085, 4.349482739926316, 4.402423123945482, 4.455702153965831, 4.509467990952544, 4.56171660090331, 4.6155840249266475, 4.668103607953526, 4.7201667139306664, 4.774923786986619, 4.8277625009650365, 4.881443522986956, 4.933932596934028, 4.987710131914355, 5.040423970902339, 5.091014708974399], "chunk_gaps_ms": {"count": 70, "p50": 53.44073398737237, "p95": 62.97799397725612, "max": 68.24586004950106}}, {"prompt": "p2048-2-0", "seed": 1002, "sent": 722886.596519665, "first": 722887.832363037, "last": 722891.697308344, "tokens": 128, "pieces": [[722887.832363037, 4], [722887.892566091, 5], [722887.945955008, 5], [722888.000763825, 11], [722888.054104965, 7], [722888.10718808, 4], [722888.159595441, 8], [722888.21483175, 12], [722888.273689595, 11], [722888.326107546, 2], [722888.379276285, 10], [722888.431963593, 12], [722888.485467009, 14], [722888.539018969, 8], [722888.592414939, 6], [722888.650412396, 6], [722888.702693947, 11], [722888.757296896, 9], [722888.812109823, 7], [722888.871873877, 17], [722888.930962481, 1], [722888.984288827, 5], [722889.037666374, 15], [722889.092392391, 3], [722889.152622991, 13], [722889.21295385, 11], [722889.272997961, 16], [722889.334769512, 7], [722889.405491331, 13], [722889.473481928, 18], [722889.538148359, 11], [722889.591439582, 6], [722889.643884975, 11], [722889.698761294, 9], [722889.749346423, 7], [722889.80216369, 4], [722889.855493198, 2], [722889.908612922, 2], [722889.961513721, 4], [722890.014691801, 2], [722890.067346627, 4], [722890.119722059, 3], [722890.172667034, 2], [722890.224328954, 9], [722890.276299984, 6], [722890.328570587, 7], [722890.381135341, 11], [722890.432473683, 6], [722890.484670773, 5], [722890.537603159, 5], [722890.589994285, 14], [722890.642618643, 9], [722890.694890255, 8], [722890.748043006, 6], [722890.800270122, 5], [722890.852798784, 8], [722890.905148006, 10], [722890.955657788, 14], [722891.007850972, 3], [722891.058926614, 18], [722891.11081302, 5], [722891.162876655, 18], [722891.215051865, 11], [722891.268761887, 9], [722891.322397725, 7], [722891.374830167, 10], [722891.429415218, 9], [722891.489629725, 6], [722891.542322722, 9], [722891.594821683, 14], [722891.646556905, 5], [722891.697308344, 4]], "completed": 722891.69748727, "ttft_s": 1.235843372065574, "decode_tps": 32.859455934084174, "prompt_tokens": 2047, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "6a2fb209181c", "token_sha256": "6a2fb209181c72c924d2f9c33c76a176bd29daed59cf2e524f9797dc123b2b19", "output_sha256": "2c3b9b193b579f84e18d91f168239df0eea3505e429b38f4090ef34ba0e6f2f0", "elapsed_s": 5.100967605016194, "chunk_offsets_s": [1.235843372065574, 1.2960464260540903, 1.3494353430578485, 1.404244160046801, 1.4575852999696508, 1.510668414994143, 1.5630757759790868, 1.6183120850473642, 1.6771699300734326, 1.729587881010957, 1.7827566199703142, 1.835443928022869, 1.8889473440358415, 1.9424993040738627, 1.9958952739834785, 2.053892730968073, 2.1061742820311338, 2.1607772309798747, 2.2155901580117643, 2.2753542120335624, 2.3344428159762174, 2.3877691620728, 2.441146709024906, 2.495872725965455, 2.556103326031007, 2.6164341850671917, 2.6764782960526645, 2.738249846966937, 2.8089716660324484, 2.876962263020687, 2.9416286940686405, 2.994919916964136, 3.0473653100198135, 3.1022416290361434, 3.152826757985167, 3.205644025001675, 3.2589735330548137, 3.31209325697273, 3.364994056057185, 3.41817213606555, 3.470826962031424, 3.523202394018881, 3.576147368992679, 3.6278092890279368, 3.6797803189838305, 3.7320509220007807, 3.7846156760351732, 3.8359540180535987, 3.888151108054444, 3.941083493991755, 3.9934746200451627, 4.046098978025839, 4.098370590014383, 4.151523341075517, 4.203750457032584, 4.256279119057581, 4.308628341066651, 4.35913812299259, 4.411331307026558, 4.462406949023716, 4.514293355052359, 4.566356990020722, 4.618532199994661, 4.67224222200457, 4.7258780599804595, 4.778310502064414, 4.832895552972332, 4.893110060016625, 4.945803057053126, 4.998302018037066, 5.05003724002745, 5.100788678973913], "chunk_gaps_ms": {"count": 71, "p50": 53.083115024492145, "p95": 61.771550914272666, "max": 70.72181906551123}}, {"prompt": "p2048-0-1", "seed": 1003, "sent": 722891.697819354, "first": 722892.946386072, "last": 722896.343821354, "tokens": 128, "pieces": [[722892.946386072, 4], [722893.007603116, 5], [722893.060150291, 5], [722893.113604649, 11], [722893.168761042, 8], [722893.228819776, 15], [722893.291699839, 5], [722893.351442856, 17], [722893.409484059, 9], [722893.461509696, 13], [722893.51512605, 2], [722893.568800857, 4], [722893.622736204, 5], [722893.674624287, 5], [722893.725914514, 10], [722893.778739342, 1], [722893.831040835, 10], [722893.883751896, 11], [722893.934797812, 17], [722893.98808794, 13], [722894.040159176, 11], [722894.095072978, 4], [722894.155904761, 15], [722894.217537097, 12], [722894.279956057, 11], [722894.341087425, 4], [722894.400562308, 1], [722894.452497114, 6], [722894.508873339, 11], [722894.571287126, 14], [722894.631707653, 4], [722894.693766196, 12], [722894.757723662, 18], [722894.82468076, 7], [722894.886328982, 12], [722894.952509262, 12], [722895.015884604, 2], [722895.07663808, 11], [722895.136667637, 3], [722895.195049399, 4], [722895.247028813, 8], [722895.301511053, 13], [722895.361483428, 14], [722895.422591372, 5], [722895.48527936, 4], [722895.542407038, 6], [722895.594354714, 11], [722895.648096511, 10], [722895.707012307, 20], [722895.767649268, 10], [722895.826911474, 16], [722895.888150488, 6], [722895.956048599, 18], [722896.024530869, 11], [722896.089828113, 10], [722896.159753467, 10], [722896.228350433, 11], [722896.293222473, 19], [722896.343821354, 7]], "completed": 722896.343964605, "ttft_s": 1.2485667179571465, "decode_tps": 37.38113884687348, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "f8aaeecc59d0", "token_sha256": "f8aaeecc59d0c280cda17bf00ec2c94ab68bf77c8f76edf8f6198403d239a62b", "output_sha256": "2b70e346e03ff0a8636f7d9647c7f430909b6a7e725cf82d4cd2a4f3fab84952", "elapsed_s": 4.646145250997506, "chunk_offsets_s": [1.2485667179571465, 1.3097837619716302, 1.3623309369431809, 1.415785294957459, 1.4709416879341006, 1.5310004219645634, 1.5938804850447923, 1.653623502003029, 1.711664704955183, 1.7636903419625014, 1.817306695971638, 1.8709815030451864, 1.92491684993729, 1.9768049330450594, 2.0280951600288972, 2.080919987987727, 2.1332214809954166, 2.1859325419645756, 2.2369784580077976, 2.2902685860171914, 2.342339821974747, 2.3972536239307374, 2.458085407037288, 2.5197177430382, 2.582136702956632, 2.6432680709986016, 2.7027429540175945, 2.754677759949118, 2.8110539850313216, 2.873467772034928, 2.9338882990414277, 2.995946842012927, 3.059904307941906, 3.1268614060245454, 3.1885096279438585, 3.254689908004366, 3.3180652499431744, 3.378818725934252, 3.438848283025436, 3.497230045031756, 3.5492094589862972, 3.6036916989833117, 3.6636640740325674, 3.724772018031217, 3.7874600059585646, 3.8445876840269193, 3.8965353600215167, 3.950277157011442, 4.009192953002639, 4.069829914020374, 4.129092119983397, 4.190331134013832, 4.25822924496606, 4.3267115149647, 4.392008758964948, 4.461934112943709, 4.530531079042703, 4.595403118990362, 4.646001999964938], "chunk_gaps_ms": {"count": 58, "p50": 59.60894998861477, "p95": 68.48226999863982, "max": 69.92535397876054}}, {"prompt": "p2048-1-1", "seed": 1004, "sent": 722896.344278667, "first": 722897.594183445, "last": 722901.734555002, "tokens": 128, "pieces": [[722897.594183445, 4], [722897.653755224, 5], [722897.707033723, 5], [722897.759518245, 9], [722897.814787479, 16], [722897.87524386, 10], [722897.935106046, 5], [722897.995995379, 5], [722898.058418919, 8], [722898.126450825, 25], [722898.193114007, 14], [722898.258849518, 11], [722898.32519949, 15], [722898.395317455, 19], [722898.462223978, 8], [722898.520415912, 4], [722898.571523113, 10], [722898.623420434, 5], [722898.675844869, 12], [722898.727737116, 9], [722898.780493877, 16], [722898.83414782, 5], [722898.88738315, 10], [722898.94110182, 11], [722898.99447784, 4], [722899.047288012, 14], [722899.101693661, 9], [722899.15114618, 6], [722899.20391962, 4], [722899.256120888, 9], [722899.309298658, 8], [722899.362460184, 11], [722899.414531446, 14], [722899.466511134, 3], [722899.519021043, 6], [722899.571053313, 4], [722899.6237117, 1], [722899.675383627, 7], [722899.727541613, 6], [722899.780433154, 10], [722899.831763834, 21], [722899.884558583, 15], [722899.937432644, 5], [722899.990477465, 2], [722900.044310215, 5], [722900.098190473, 10], [722900.150282655, 4], [722900.204117646, 7], [722900.257436069, 1], [722900.310938288, 8], [722900.36366191, 15], [722900.416644239, 1], [722900.470007471, 4], [722900.523495492, 8], [722900.576266999, 5], [722900.628849304, 6], [722900.681660192, 5], [722900.732191768, 5], [722900.78469198, 10], [722900.836989825, 4], [722900.889860401, 5], [722900.942473695, 9], [722900.995354153, 7], [722901.048500652, 5], [722901.101817536, 5], [722901.153309136, 2], [722901.204318954, 8], [722901.257163185, 10], [722901.310769364, 5], [722901.36368185, 8], [722901.416703334, 4], [722901.468837398, 7], [722901.523212496, 5], [722901.575592582, 7], [722901.629614999, 9], [722901.683676093, 8], [722901.734555002, 6]], "completed": 722901.734651316, "ttft_s": 1.2499047779710963, "decode_tps": 30.673575608455927, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "854ffaa5d759", "token_sha256": "854ffaa5d75926074fa12f5f072ace31fab67f02428f5cc5cde0f8536b956273", "output_sha256": "5f25c0765ad2cacf455149436ab15653f18fa842e275ff063e318e99d60b7e45", "elapsed_s": 5.3903726489515975, "chunk_offsets_s": [1.2499047779710963, 1.3094765569549054, 1.3627550560049713, 1.4152395779965445, 1.470508811995387, 1.5309651929419488, 1.5908273790264502, 1.651716712047346, 1.7141402519773692, 1.7821721580112353, 1.8488353400025517, 1.9145708510186523, 1.9809208229416981, 2.051038788049482, 2.1179453110089526, 2.1761372450273484, 2.2272444459376857, 2.279141766950488, 2.3315662019886076, 2.3834584490396082, 2.4362152100075036, 2.489869153010659, 2.5431044830475003, 2.5968231529695913, 2.6501991730183363, 2.70300934498664, 2.7574149939464405, 2.8068675129907206, 2.8596409530146047, 2.9118422210449353, 2.9650199910392985, 3.0181815170217305, 3.070252778939903, 3.1222324669361115, 3.1747423759661615, 3.2267746459692717, 3.279433032963425, 3.331104959943332, 3.3832629460375756, 3.4361544869607314, 3.4874851669883355, 3.540279916021973, 3.593153976951726, 3.64619879797101, 3.7000315480399877, 3.7539118060376495, 3.8060039880219847, 3.859838978969492, 3.9131574019556865, 3.96665962098632, 4.019383243052289, 4.072365571977571, 4.125728804036044, 4.179216824937612, 4.231988332001492, 4.284570637042634, 4.337381524965167, 4.387913101003505, 4.440413312986493, 4.492711158003658, 4.545581734040752, 4.598195028025657, 4.651075485977344, 4.7042219849536195, 4.757538869045675, 4.809030469041318, 4.860040287021548, 4.9128845180384815, 4.966490696999244, 5.019403183017857, 5.072424666956067, 5.124558730982244, 5.178933828952722, 5.231313914991915, 5.285336332046427, 5.33939742599614, 5.390276334946975], "chunk_gaps_ms": {"count": 76, "p50": 52.90201347088441, "p95": 66.66318199131638, "max": 70.11796510778368}}, {"prompt": "p2048-2-1", "seed": 1005, "sent": 722901.734960933, "first": 722902.976675545, "last": 722906.12782438, "tokens": 128, "pieces": [[722902.976675545, 3], [722903.039107352, 13], [722903.102244821, 13], [722903.162756292, 16], [722903.232472469, 11], [722903.300981432, 10], [722903.365819914, 2], [722903.420398024, 9], [722903.478583603, 1], [722903.531526262, 12], [722903.583742061, 10], [722903.638708128, 8], [722903.69707155, 2], [722903.748368123, 8], [722903.801333506, 10], [722903.857185473, 16], [722903.91853911, 6], [722903.977813176, 23], [722904.038899592, 18], [722904.096797356, 5], [722904.148911046, 17], [722904.203750324, 8], [722904.263860542, 17], [722904.322849679, 10], [722904.383133513, 17], [722904.445322601, 10], [722904.511129876, 7], [722904.566156704, 5], [722904.626576775, 15], [722904.687712472, 13], [722904.748356794, 15], [722904.807015582, 12], [722904.868968567, 9], [722904.936677092, 24], [722905.004709144, 18], [722905.071926005, 8], [722905.140815685, 10], [722905.2081086, 12], [722905.277467287, 26], [722905.346021689, 27], [722905.413901867, 20], [722905.478640219, 5], [722905.539013719, 15], [722905.598104383, 6], [722905.65972456, 13], [722905.72118091, 19], [722905.783940115, 3], [722905.838559288, 12], [722905.899084141, 16], [722905.959438665, 11], [722906.019644206, 7], [722906.077639032, 16], [722906.12782438, 3]], "completed": 722906.127943688, "ttft_s": 1.2417146119987592, "decode_tps": 40.30276151695653, "prompt_tokens": 2047, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "0effe3e2e261", "token_sha256": "0effe3e2e261cfefd427f381703c20e16f63a81a352520ec51fc2d517053699c", "output_sha256": "f287d0e9df37feb7e700f1bba37f292275b8c2f5ada5cc3161dbcba17024f69b", "elapsed_s": 4.392982754972763, "chunk_offsets_s": [1.2417146119987592, 1.3041464190464467, 1.367283888044767, 1.4277953590499237, 1.4975115359993652, 1.566020498983562, 1.630858981050551, 1.6854370910441503, 1.743622670066543, 1.7965653290739283, 1.8487811280647293, 1.9037471950287, 1.9621106169652194, 2.0134071899810806, 2.066372573026456, 2.1222245399840176, 2.183578177005984, 2.2428522430127487, 2.303938659024425, 2.361836423049681, 2.4139501129975542, 2.4687893909867853, 2.528899608994834, 2.5878887460567057, 2.648172580054961, 2.7103616680251434, 2.7761689430335537, 2.8311957709956914, 2.8916158420033753, 2.9527515390655026, 3.013395861024037, 3.0720546490047127, 3.13400763401296, 3.201716159004718, 3.2697482110233977, 3.3369650719687343, 3.405854752054438, 3.4731476670131087, 3.542506354046054, 3.611060756025836, 3.678940934012644, 3.743679285980761, 3.8040527859702706, 3.8631434499984607, 3.9247636270010844, 3.986219977028668, 4.048979182029143, 4.10359835496638, 4.164123208029196, 4.2244777319720015, 4.284683273057453, 4.3426780990557745, 4.39286344696302], "chunk_gaps_ms": {"count": 52, "p50": 60.465771006420255, "p95": 68.88968008570373, "max": 69.71617694944143}}, {"prompt": "p2048-0-2", "seed": 1006, "sent": 722906.128229482, "first": 722907.376302161, "last": 722910.362400481, "tokens": 128, "pieces": [[722907.376302161, 3], [722907.439587822, 12], [722907.502153371, 18], [722907.565455943, 13], [722907.632880942, 6], [722907.700642561, 10], [722907.769875918, 16], [722907.836799721, 14], [722907.905139965, 26], [722907.972180736, 10], [722908.041453664, 9], [722908.106635553, 4], [722908.165214795, 14], [722908.224485258, 17], [722908.285310161, 16], [722908.346674482, 13], [722908.41287937, 24], [722908.477251028, 9], [722908.530808345, 8], [722908.584766719, 5], [722908.637545062, 8], [722908.689620705, 5], [722908.742245969, 5], [722908.79535321, 17], [722908.847981949, 13], [722908.90114764, 11], [722908.953548098, 7], [722909.005679567, 7], [722909.058862426, 11], [722909.11191453, 12], [722909.163965272, 4], [722909.217282505, 2], [722909.268838258, 21], [722909.321683828, 5], [722909.376676251, 11], [722909.431172191, 7], [722909.492463013, 12], [722909.55619259, 15], [722909.618838852, 16], [722909.682693835, 6], [722909.751732003, 25], [722909.822194343, 14], [722909.892523544, 14], [722909.962556805, 21], [722910.033724369, 5], [722910.103588843, 17], [722910.173077101, 16], [722910.241556236, 5], [722910.306943185, 6], [722910.362400481, 7]], "completed": 722910.362514116, "ttft_s": 1.2480726790381595, "decode_tps": 42.53041473878913, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "2d30fcc66f99", "token_sha256": "2d30fcc66f9925bb2dbab2bfa2c3b819cfbe1e820d089d4774f26089df2b46b3", "output_sha256": "d34bdddcdc734baf9133c26fab126a3ff57ccc943549ce75ea13ddb4d7bfe2ef", "elapsed_s": 4.234284634003416, "chunk_offsets_s": [1.2480726790381595, 1.3113583399681374, 1.373923889012076, 1.437226461013779, 1.5046514599816874, 1.5724130789749324, 1.6416464360663667, 1.7085702390177175, 1.7769104830222204, 1.8439512540353462, 1.913224181975238, 1.978406070964411, 2.036985313054174, 2.0962557760067284, 2.157080678967759, 2.218445000005886, 2.2846498880535364, 2.349021545960568, 2.4025788629660383, 2.45653723704163, 2.509315580013208, 2.5613912229891866, 2.6140164870303124, 2.6671237279661, 2.7197524670045823, 2.7729181579779834, 2.825318616000004, 2.877450085012242, 2.9306329439859837, 2.983685048064217, 3.0357357900356874, 3.089053023024462, 3.1406087760115042, 3.19345434603747, 3.248446768964641, 3.3029427090659738, 3.3642335310578346, 3.427963108057156, 3.4906093700556085, 3.5544643530156463, 3.6235025209607556, 3.6939648609841242, 3.764294062042609, 3.8343273230129853, 3.905494887032546, 3.9753593610366806, 4.044847619021311, 4.11332675407175, 4.178713703062385, 4.23417099902872], "chunk_gaps_ms": {"count": 49, "p50": 62.56554904393852, "p95": 70.32920105848461, "max": 71.16756401956081}}], "together": [{"prompt": "p2048-0-0", "seed": 1000, "sent": 722912.364521128, "first": 722917.673923477, "last": 722931.73233458, "tokens": 128, "pieces": [[722917.673923477, 4], [722917.833640261, 8], [722919.246229615, 20], [722920.688026066, 22], [722922.150287831, 12], [722922.360262249, 14], [722922.570555571, 12], [722922.780446542, 24], [722922.994346018, 9], [722923.212392013, 12], [722923.432117228, 12], [722923.649179024, 10], [722923.86636378, 18], [722924.08528275, 17], [722924.305302818, 14], [722924.526882008, 21], [722924.74094108, 5], [722924.962577835, 17], [722925.184750481, 14], [722925.408158828, 4], [722925.629627086, 14], [722925.859360271, 12], [722926.088384361, 19], [722926.317491554, 15], [722926.551327095, 3], [722926.780532249, 13], [722927.015778333, 6], [722927.241836999, 16], [722927.463591524, 10], [722927.696338734, 13], [722927.927690667, 21], [722928.162614808, 6], [722928.399578493, 10], [722928.633176205, 21], [722928.866771323, 6], [722929.100991685, 9], [722929.336328731, 4], [722929.570670026, 11], [722929.801058976, 10], [722930.016960454, 13], [722930.234597483, 10], [722930.452988643, 7], [722930.659963543, 8], [722930.85079412, 17], [722931.047052531, 1], [722931.238082969, 32], [722931.418649355, 9], [722931.582169674, 17], [722931.73233458, 4]], "completed": 722931.732474156, "ttft_s": 5.309402348939329, "decode_tps": 9.033737814942322, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "7d62171119f8", "token_sha256": "7d62171119f86883ec9ea4e3542846c00509f1ad0baba0a0676ccd3b8e14cd56", "output_sha256": "ac406cb14fdb49d5cf4a0fcac1a4d3e1ea135c258abd0ad94ead39fc0308e2c7", "elapsed_s": 19.367953027947806, "chunk_offsets_s": [5.309402348939329, 5.469119132962078, 6.8817084869369864, 8.323504938045517, 9.785766702960245, 9.995741121005267, 10.20603444299195, 10.415925413952209, 10.62982488994021, 10.847870885045268, 11.067596100037917, 11.284657896030694, 11.50184265198186, 11.720761621952988, 11.940781689947471, 12.162360879941843, 12.376419952022843, 12.598056706949137, 12.820229353033938, 13.043637700029649, 13.265105957980268, 13.494839143007994, 13.723863232997246, 13.95297042594757, 14.186805967008695, 14.416011121007614, 14.651257204939611, 14.877315870951861, 15.099070396041498, 15.331817605998367, 15.563169538974762, 15.798093679943122, 16.035057364962995, 16.26865507697221, 16.502250195015222, 16.736470556934364, 16.97180760302581, 17.206148897996172, 17.436537847970612, 17.652439326047897, 17.870076354942285, 18.0884675149573, 18.29544241493568, 18.48627299198415, 18.682531403028406, 18.87356184097007, 19.05412822694052, 19.21764854597859, 19.367813452030532], "chunk_gaps_ms": {"count": 48, "p50": 221.60797246033326, "p95": 1412.5893539749086, "max": 1462.2617649147287}}, {"prompt": "p2048-1-0", "seed": 1001, "sent": 722912.364931766, "first": 722916.3005504, "last": 722932.447807638, "tokens": 128, "pieces": [[722916.3005504, 3], [722916.435957136, 17], [722917.833948947, 6], [722919.246699406, 3], [722920.688826582, 10], [722922.149353309, 10], [722922.35911622, 19], [722922.570403786, 7], [722922.7799395, 13], [722922.994277275, 17], [722923.212495327, 18], [722923.43247764, 1], [722923.649902057, 4], [722923.866833222, 17], [722924.085487471, 21], [722924.305388276, 10], [722924.526716166, 11], [722924.741147065, 3], [722924.962518621, 5], [722925.184955313, 15], [722925.408201245, 7], [722925.629734209, 3], [722925.858565317, 19], [722926.087843942, 11], [722926.316922443, 10], [722926.551464416, 7], [722926.780604978, 9], [722927.015720945, 16], [722927.242744306, 10], [722927.463497804, 12], [722927.696253933, 5], [722927.927615826, 6], [722928.16289491, 8], [722928.399788637, 4], [722928.632142139, 11], [722928.866704168, 15], [722929.101482122, 24], [722929.336179882, 8], [722929.571658805, 6], [722929.80090225, 11], [722930.016832805, 7], [722930.234641724, 8], [722930.452954313, 11], [722930.65978653, 11], [722930.8510123, 4], [722931.047673941, 11], [722931.23816961, 7], [722931.418567903, 4], [722931.582225475, 16], [722931.732268211, 5], [722931.858712489, 16], [722931.961686904, 10], [722932.057709399, 6], [722932.155659572, 13], [722932.25469758, 11], [722932.353092047, 7], [722932.447807638, 14]], "completed": 722932.447893942, "ttft_s": 3.935618634102866, "decode_tps": 7.86511282558205, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "c948b4bcbd58", "token_sha256": "c948b4bcbd5833ccd9c3bbd0f64e960f0ed36def137ba411ed3ade9437416854", "output_sha256": "78dbe5350838766da1d257beb3649f97e5ed2aab47e7249e7dcefae81dba7cef", "elapsed_s": 20.08296217606403, "chunk_offsets_s": [3.935618634102866, 4.0710253700381145, 5.469017181079835, 6.8817676400067285, 8.323894816101529, 9.784421543008648, 9.9941844540881, 10.205472020083107, 10.415007734089158, 10.629345509107225, 10.847563561052084, 11.0675458740443, 11.284970291075297, 11.501901456038468, 11.720555705018342, 11.94045651005581, 12.161784399999306, 12.376215299009345, 12.59758685505949, 12.820023547043093, 13.043269479065202, 13.264802443096414, 13.493633551057428, 13.72291217604652, 13.951990677043796, 14.186532650026493, 14.415673212031834, 14.650789179024287, 14.877812540042214, 15.098566038068384, 15.33132216706872, 15.562684060074389, 15.797963144024834, 16.034856871003285, 16.267210373072885, 16.501772402087227, 16.7365503560286, 16.971248116111383, 17.206727039068937, 17.435970484046265, 17.651901039062068, 17.869709958089516, 18.088022547075525, 18.29485476401169, 18.48608053405769, 18.68274217506405, 18.87323784409091, 19.05363613704685, 19.217293709050864, 19.367336445022374, 19.49378072307445, 19.596755138016306, 19.692777633084916, 19.790727806044742, 19.88976581406314, 19.98816028109286, 20.082875872030854], "chunk_gaps_ms": {"count": 56, "p50": 219.9415590148419, "p95": 1412.7504589268938, "max": 1460.5267269071192}}, {"prompt": "p2048-2-0", "seed": 1002, "sent": 722912.365246403, "first": 722913.618753813, "last": 722931.859078682, "tokens": 128, "pieces": [[722913.618753813, 4], [722913.704628123, 5], [722915.049438273, 13], [722916.435669234, 10], [722917.833682744, 12], [722919.246300521, 25], [722920.688780593, 18], [722922.151222433, 4], [722922.359901451, 14], [722922.570087728, 8], [722922.779848916, 12], [722922.994210519, 17], [722923.21278647, 10], [722923.432515365, 17], [722923.649058385, 1], [722923.866427879, 13], [722924.085658678, 7], [722924.305806283, 11], [722924.527259279, 5], [722924.741291658, 11], [722924.963521565, 18], [722925.185293529, 9], [722925.408524167, 23], [722925.629871403, 15], [722925.858951441, 6], [722926.088215926, 11], [722926.317390436, 16], [722926.552029316, 5], [722926.780663951, 7], [722927.016175055, 2], [722927.242495907, 4], [722927.463943714, 3], [722927.696453494, 2], [722927.927925455, 9], [722928.162245805, 6], [722928.399429116, 18], [722928.632758464, 6], [722928.867538114, 7], [722929.101549853, 3], [722929.336481262, 14], [722929.571073211, 17], [722929.801423299, 6], [722930.017289678, 5], [722930.234948484, 18], [722930.453185991, 17], [722930.660263224, 23], [722930.850657554, 29], [722931.047136321, 16], [722931.238847239, 10], [722931.41940672, 15], [722931.582479233, 23], [722931.733160796, 5], [722931.859078682, 4]], "completed": 722931.859197747, "ttft_s": 1.2535074099432677, "decode_tps": 6.962595288829991, "prompt_tokens": 2047, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "6a2fb209181c", "token_sha256": "6a2fb209181c72c924d2f9c33c76a176bd29daed59cf2e524f9797dc123b2b19", "output_sha256": "2c3b9b193b579f84e18d91f168239df0eea3505e429b38f4090ef34ba0e6f2f0", "elapsed_s": 19.493951343931258, "chunk_offsets_s": [1.2535074099432677, 1.3393817199394107, 2.684191869921051, 4.070422830991447, 5.468436340917833, 6.881054117926396, 8.323534189956263, 9.785976029932499, 9.994655048009008, 10.204841324943118, 10.41460251295939, 10.628964115981944, 10.84754006692674, 11.067268961924128, 11.283811982022598, 11.501181475934573, 11.720412274939008, 11.940559879993089, 12.162012876011431, 12.376045254990458, 12.59827516193036, 12.82004712591879, 13.04327776399441, 13.264624999952503, 13.493705037981272, 13.722969522932544, 13.952144032926299, 14.186782912933268, 14.415417547919787, 14.650928651914, 14.877249503973871, 15.098697311012074, 15.331207090988755, 15.56267905200366, 15.79699940199498, 16.0341827130178, 16.267512060934678, 16.502291710930876, 16.73630344995763, 16.971234858967364, 17.205826807999983, 17.43617689597886, 17.65204327495303, 17.86970208096318, 18.087939588003792, 18.295016821008176, 18.485411150963046, 18.681889917934313, 18.87360083591193, 19.054160316940397, 19.21723283000756, 19.36791439296212, 19.493832278996706], "chunk_gaps_ms": {"count": 52, "p50": 221.61248000338674, "p95": 1412.6177770085633, "max": 1462.4418399762362}}, {"prompt": "p2048-0-1", "seed": 1003, "sent": 722912.365617103, "first": 722921.934608585, "last": 722931.859465691, "tokens": 128, "pieces": [[722921.934608585, 4], [722922.149784325, 5], [722922.359230487, 5], [722922.570812783, 19], [722922.780735715, 16], [722922.994531989, 12], [722923.213342415, 23], [722923.433183763, 10], [722923.64973812, 9], [722923.86697639, 5], [722924.085726107, 10], [722924.305239104, 1], [722924.526764908, 20], [722924.740732421, 18], [722924.96244349, 13], [722925.184616408, 15], [722925.408035372, 17], [722925.629249689, 21], [722925.858642706, 4], [722926.088113111, 1], [722926.316801374, 8], [722926.551777346, 13], [722926.780481875, 14], [722927.015444444, 15], [722927.242316024, 22], [722927.463323415, 12], [722927.696084956, 12], [722927.927571792, 2], [722928.162654246, 12], [722928.398907391, 6], [722928.632623497, 8], [722928.867018119, 13], [722929.100612805, 15], [722929.33623181, 6], [722929.570337471, 8], [722929.800971982, 21], [722930.016772568, 20], [722930.234307279, 10], [722930.452682333, 16], [722930.660009978, 6], [722930.850279446, 18], [722931.046874365, 11], [722931.23801937, 10], [722931.41922404, 10], [722931.582112031, 11], [722931.732718338, 19], [722931.859465691, 7]], "completed": 722931.859537649, "ttft_s": 9.568991482025012, "decode_tps": 12.796154004318913, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "f8aaeecc59d0", "token_sha256": "f8aaeecc59d0c280cda17bf00ec2c94ab68bf77c8f76edf8f6198403d239a62b", "output_sha256": "2b70e346e03ff0a8636f7d9647c7f430909b6a7e725cf82d4cd2a4f3fab84952", "elapsed_s": 19.493920546025038, "chunk_offsets_s": [9.568991482025012, 9.784167222096585, 9.993613384081982, 10.205195680027828, 10.415118611999787, 10.628914886037819, 10.847725312109105, 11.067566660000011, 11.284121017088182, 11.501359287067316, 11.720109004061669, 11.939622001023963, 12.161147805047221, 12.375115318107419, 12.596826387103647, 12.818999305018224, 13.042418269091286, 13.263632586109452, 13.493025603005663, 13.722496008034796, 13.951184271019883, 14.186160242999904, 14.414864771999419, 14.649827341083437, 14.876698921085335, 15.097706312080845, 15.330467853113078, 15.561954689095728, 15.79703714302741, 16.033290288061835, 16.2670063941041, 16.50140101602301, 16.734995702048764, 16.97061470709741, 17.204720368026756, 17.435354879009537, 17.65115546504967, 17.86869017605204, 18.087065230007283, 18.29439287504647, 18.484662343049422, 18.68125726201106, 18.8724022670649, 19.05360693705734, 19.216494928114116, 19.367101235082373, 19.493848588084802], "chunk_gaps_ms": {"count": 46, "p50": 220.42436944320798, "p95": 235.0824539316818, "max": 236.25314503442496}}, {"prompt": "p2048-1-1", "seed": 1004, "sent": 722912.365976754, "first": 722919.071366557, "last": 722933.098257022, "tokens": 128, "pieces": [[722919.071366557, 4], [722919.246535384, 5], [722920.688355451, 5], [722922.149487672, 9], [722922.360621317, 16], [722922.57087211, 10], [722922.780668942, 8], [722922.994630788, 10], [722923.213530948, 25], [722923.432665243, 14], [722923.648994078, 11], [722923.86620303, 15], [722924.085984886, 19], [722924.305489139, 8], [722924.527065911, 4], [722924.740850145, 10], [722924.963298286, 5], [722925.185579705, 12], [722925.408557521, 15], [722925.629910409, 10], [722925.859043961, 15], [722926.088260457, 11], [722926.317424542, 4], [722926.551984834, 14], [722926.780777831, 15], [722927.016401774, 4], [722927.242569227, 9], [722927.463759269, 8], [722927.696689034, 11], [722927.927969425, 14], [722928.162153174, 3], [722928.399638475, 6], [722928.632822624, 5], [722928.866904513, 7], [722929.101376441, 6], [722929.336560058, 10], [722929.570985481, 21], [722929.801326042, 20], [722930.017322759, 2], [722930.234986785, 5], [722930.453258414, 10], [722930.660367735, 11], [722930.850734444, 1], [722931.047406188, 8], [722931.238640748, 15], [722931.418900438, 1], [722931.582522161, 4], [722931.732948366, 13], [722931.859526616, 11], [722931.962223976, 15], [722932.057937105, 4], [722932.15597552, 5], [722932.255151214, 9], [722932.353500446, 7], [722932.448664849, 5], [722932.51369655, 7], [722932.566869805, 8], [722932.619811288, 10], [722932.673106456, 5], [722932.725360374, 8], [722932.778658201, 4], [722932.832127178, 7], [722932.885448462, 5], [722932.938224383, 7], [722932.992877213, 9], [722933.047509964, 8], [722933.098257022, 6]], "completed": 722933.098402153, "ttft_s": 6.705389803042635, "decode_tps": 9.054038050515667, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "854ffaa5d759", "token_sha256": "854ffaa5d75926074fa12f5f072ace31fab67f02428f5cc5cde0f8536b956273", "output_sha256": "5f25c0765ad2cacf455149436ab15653f18fa842e275ff063e318e99d60b7e45", "elapsed_s": 20.732425399008207, "chunk_offsets_s": [6.705389803042635, 6.880558629985899, 8.3223786969902, 9.783510918030515, 9.994644563063048, 10.20489535597153, 10.41469218803104, 10.62865403399337, 10.84755419404246, 11.066688489052467, 11.283017324050888, 11.500226275995374, 11.720008132047951, 11.939512385055423, 12.161089156987146, 12.374873391003348, 12.597321531968191, 12.819602951058187, 13.042580766952597, 13.263933654990979, 13.493067207047716, 13.722283703042194, 13.951447788043879, 14.186008080025204, 14.414801076985896, 14.650425020023249, 14.876592473010533, 15.097782514989376, 15.330712280003354, 15.561992670991458, 15.796176420059055, 16.033661721041426, 16.266845869948156, 16.5009277589852, 16.73539968696423, 16.970583303947933, 17.20500872703269, 17.43534928804729, 17.6513460050337, 17.869010030990466, 18.0872816600604, 18.294390980969183, 18.484757689991966, 18.681429433985613, 18.87266399397049, 19.05292368400842, 19.216545406961814, 19.366971611976624, 19.493549862061627, 19.59624722204171, 19.69196035095956, 19.78999876603484, 19.889174459967762, 19.98752369196154, 20.08268809504807, 20.147719796048477, 20.200893050991, 20.253834533970803, 20.307129701948725, 20.359383619972505, 20.412681446992792, 20.466150423977524, 20.519471707986668, 20.57224762905389, 20.62690045905765, 20.681533209979534, 20.732280267984606], "chunk_gaps_ms": {"count": 66, "p50": 216.16277599241585, "p95": 235.62394303735346, "max": 1461.132221040316}}, {"prompt": "p2048-2-1", "seed": 1005, "sent": 722912.366362107, "first": 722914.930030965, "last": 722930.659871186, "tokens": 128, "pieces": [[722914.930030965, 3], [722915.04952095, 13], [722916.435561032, 21], [722917.834128246, 14], [722919.246278166, 8], [722920.687886113, 9], [722922.150107047, 10], [722922.359811652, 12], [722922.570491056, 18], [722922.78007351, 2], [722922.994699354, 8], [722923.213280141, 10], [722923.432933006, 17], [722923.648919523, 26], [722923.866678732, 2], [722924.085934383, 23], [722924.306235483, 23], [722924.526973583, 19], [722924.741220915, 10], [722924.963727005, 18], [722925.18573698, 9], [722925.408856201, 7], [722925.629837329, 18], [722925.858438835, 15], [722926.087894765, 18], [722926.316983418, 18], [722926.551651889, 24], [722926.780833746, 18], [722927.015517925, 8], [722927.242127366, 10], [722927.463678006, 12], [722927.696207818, 26], [722927.92808814, 27], [722928.163194884, 20], [722928.399858562, 5], [722928.632200023, 15], [722928.867429808, 6], [722929.101590956, 13], [722929.336593267, 19], [722929.571545712, 3], [722929.801103842, 12], [722930.016884861, 22], [722930.234733121, 5], [722930.452787132, 11], [722930.659871186, 15]], "completed": 722930.660048295, "ttft_s": 2.5636688580270857, "decode_tps": 8.073826448055742, "prompt_tokens": 2047, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "0effe3e2e261", "token_sha256": "0effe3e2e261cfefd427f381703c20e16f63a81a352520ec51fc2d517053699c", "output_sha256": "f287d0e9df37feb7e700f1bba37f292275b8c2f5ada5cc3161dbcba17024f69b", "elapsed_s": 18.293686188058928, "chunk_offsets_s": [2.5636688580270857, 2.6831588429631665, 4.069198925048113, 5.467766138957813, 6.879916058969684, 8.32152400596533, 9.783744940068573, 9.993449544999748, 10.20412894897163, 10.413711403030902, 10.628337247064337, 10.846918034018017, 11.066570898983628, 11.282557416008785, 11.500316624995321, 11.719572275993414, 11.939873376046307, 12.16061147605069, 12.374858808005229, 12.59736489795614, 12.819374873070046, 13.042494093999267, 13.263475222047418, 13.492076728027314, 13.721532657975331, 13.950621310970746, 14.185289782006294, 14.414471638971008, 14.649155818042345, 14.875765259028412, 15.097315898980014, 15.329845711006783, 15.5617260329891, 15.79683277697768, 16.033496455056593, 16.265837916056626, 16.50106770102866, 16.735228848992847, 16.970231160055846, 17.205183604964986, 17.434741734992713, 17.650522754061967, 17.86837101401761, 18.08642502501607, 18.29350907902699], "chunk_gaps_ms": {"count": 44, "p50": 224.8643309576437, "p95": 1412.1499200118706, "max": 1462.220934103243}}, {"prompt": "p2048-0-2", "seed": 1006, "sent": 722912.366639608, "first": 722920.493867379, "last": 722931.418841563, "tokens": 128, "pieces": [[722920.493867379, 3], [722920.688587766, 12], [722922.150439872, 21], [722922.360054036, 10], [722922.569973413, 6], [722922.780261659, 10], [722922.994667137, 16], [722923.212878927, 14], [722923.432880741, 26], [722923.649595768, 10], [722923.866530567, 9], [722924.085589695, 4], [722924.305980723, 14], [722924.527439531, 22], [722924.741059348, 23], [722924.963263107, 12], [722925.185629568, 5], [722925.408275158, 8], [722925.629791549, 9], [722925.858984923, 8], [722926.088183213, 5], [722926.317357931, 8], [722926.551709581, 18], [722926.780414447, 9], [722927.015993235, 24], [722927.242531822, 14], [722927.463978396, 11], [722927.696643735, 12], [722927.927883631, 6], [722928.162534731, 21], [722928.399386267, 14], [722928.632722612, 17], [722928.866502255, 17], [722929.101108109, 18], [722929.336527329, 7], [722929.571022917, 26], [722929.801264459, 20], [722930.017240073, 14], [722930.234913481, 15], [722930.453100439, 9], [722930.660329386, 15], [722930.850620998, 16], [722931.047447739, 1], [722931.238477714, 6], [722931.418841563, 7]], "completed": 722931.419239619, "ttft_s": 8.127227770979516, "decode_tps": 11.624741428294717, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "2d30fcc66f99", "token_sha256": "2d30fcc66f9925bb2dbab2bfa2c3b819cfbe1e820d089d4774f26089df2b46b3", "output_sha256": "d34bdddcdc734baf9133c26fab126a3ff57ccc943549ce75ea13ddb4d7bfe2ef", "elapsed_s": 19.052600010996684, "chunk_offsets_s": [8.127227770979516, 8.321948157972656, 9.78380026400555, 9.993414427968673, 10.20333380496595, 10.413622050895356, 10.62802752899006, 10.846239318954758, 11.066241132910363, 11.282956159906462, 11.499890958890319, 11.718950086971745, 11.939341114950366, 12.160799923003651, 12.374419739935547, 12.596623498946428, 12.81898995989468, 13.04163554997649, 13.263151941006072, 13.492345314938575, 13.721543604973704, 13.950718323001638, 14.185069972998463, 14.413774838903919, 14.649353626999073, 14.875892213894986, 15.097338787978515, 15.330004126997665, 15.561244022916071, 15.795895122922957, 16.032746658893302, 16.2660830039531, 16.49986264691688, 16.734468500944786, 16.969887721003033, 17.20438330899924, 17.4346248509828, 17.65060046489816, 17.868273872998543, 18.08646083099302, 18.293689777958207, 18.4839813899016, 18.68080813100096, 18.871838105958886, 19.052201954997145], "chunk_gaps_ms": {"count": 44, "p50": 221.48759954143316, "p95": 235.57878809515387, "max": 1461.852106032893}}, {"prompt": "long65536", "seed": 999, "sent": 722910.363900197, "first": 722911.801374814, "last": 722929.796246525, "tokens": 128, "pieces": [[722911.801374814, 7], [722911.879424083, 8], [722911.94079326, 8], [722912.003924736, 17], [722912.072479694, 8], [722912.140708756, 7], [722912.204581959, 10], [722912.259162111, 8], [722912.319785469, 8], [722912.38206122, 8], [722913.704717687, 16], [722915.049693271, 19], [722916.435816796, 29], [722917.833775748, 6], [722919.246496713, 5], [722920.688498846, 13], [722922.151060256, 15], [722922.35957476, 2], [722922.570918482, 5], [722922.780378856, 15], [722922.994871362, 17], [722923.213685455, 14], [722923.432581866, 2], [722923.649254507, 3], [722923.866278193, 9], [722924.086039265, 7], [722924.306034958, 13], [722924.527022902, 17], [722924.741257295, 13], [722924.962928301, 5], [722925.185061236, 14], [722925.408595054, 5], [722925.629471317, 15], [722925.859197985, 11], [722926.08803798, 8], [722926.317151836, 10], [722926.552071636, 11], [722926.78097624, 11], [722927.016503869, 12], [722927.242057937, 5], [722927.4640456, 16], [722927.696865745, 16], [722927.927474534, 19], [722928.163025875, 17], [722928.399095102, 20], [722928.633339786, 17], [722928.867199809, 2], [722929.101665973, 4], [722929.336632945, 10], [722929.57043127, 14], [722929.796246525, 7]], "completed": 722929.796407478, "ttft_s": 1.4374746170360595, "decode_tps": 7.057566291084029, "prompt_tokens": 65536, "cached_tokens": 63488, "cache_state": "reused", "complete": true, "token_sha": "37aa1a600539", "token_sha256": "37aa1a600539ec62fc9123edbb8a23c5014fb4d7b318fa839fd6fe79d626df8c", "output_sha256": "cfbc23fbdd79a4a87c680fe7c5f58640080f30ba1ac2d03226876b86f6123c86", "elapsed_s": 19.43250728107523, "chunk_offsets_s": [1.4374746170360595, 1.5155238860752434, 1.5768930630292743, 1.6400245390832424, 1.7085794970626011, 1.7768085590796545, 1.8406817620852962, 1.8952619140036404, 1.9558852720074356, 2.0181610230356455, 3.3408174900105223, 4.6857930740807205, 6.0719165990594774, 7.469875551061705, 8.882596516050398, 10.32459864905104, 11.787160058971494, 11.995674562989734, 12.207018285058439, 12.416478659026325, 12.630971165024675, 12.849785258062184, 13.068681669072248, 13.285354310064577, 13.502377995988354, 13.72213906806428, 13.942134761018679, 14.163122705067508, 14.377357098041102, 14.599028103984892, 14.821161039057188, 15.044694856973365, 15.265571120078675, 15.49529778806027, 15.724137783050537, 15.953251639031805, 16.18817143898923, 16.417076043086126, 16.652603672002442, 16.878157739993185, 17.100145403062925, 17.332965548033826, 17.56357433705125, 17.799125678022392, 18.03519490500912, 18.269439588999376, 18.503299612086266, 18.737765775993466, 18.97273274802137, 19.20653107308317, 19.43234632804524], "chunk_gaps_ms": {"count": 50, "p50": 222.8333764942363, "p95": 1412.7209649886936, "max": 1462.561409920454}}], "rows": [{"name": "p2048-0-0", "prompt": 2048, "sent": 2.001782753970474, "first": 7.311185102909803, "last": 21.369596206001006, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-1-0", "prompt": 2048, "sent": 2.002193391905166, "first": 5.937812026008032, "last": 22.08506926393602, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-2-0", "prompt": 2047, "sent": 2.0025080289924517, "first": 3.2560154389357194, "last": 21.496340307989158, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-0-1", "prompt": 2048, "sent": 2.0028787289047614, "first": 11.571870210929774, "last": 21.496727316989563, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-1-1", "prompt": 2048, "sent": 2.0032383799552917, "first": 8.708628182997927, "last": 22.735518647939898, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-2-1", "prompt": 2047, "sent": 2.0036237329477444, "first": 4.56729259097483, "last": 20.297132811974734, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-0-2", "prompt": 2048, "sent": 2.0039012340130284, "first": 10.131129004992545, "last": 21.056103189010173, "error": null, "equal_alone": true, "started_after": []}, {"name": "long65536", "prompt": 65536, "sent": 0.0011618229327723384, "first": 1.4386364399688318, "last": 19.433508150978014, "error": null, "equal_alone": null, "started_after": []}]} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/rows-without-bodies.jsonl b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/rows-without-bodies.jsonl new file mode 100644 index 0000000000..7de61d7460 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/rows-without-bodies.jsonl @@ -0,0 +1,44 @@ +{"i": 0, "conv": "S1", "turn": 1, "arm": "drafted", "group": "", "history": 7757, "allowed": [0], "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.120233714, "prefill": 5.118784751, "wall": 6.751589284045622, "finish": "tool_calls"}, "ok": true, "body_sha256": "c89863023849e1c523580f91d6e3e557e187ee1345ec141afe8843f12f6a48bb", "body_bytes": 29036} +{"i": 1, "conv": "S1", "turn": 1, "arm": "plain", "group": "", "history": 7757, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 6144, "prompt": 7759, "tokens": 61, "ttft": 1.012918355, "prefill": 1.011209944, "wall": 3.762124193017371, "finish": "tool_calls"}, "ok": true, "body_sha256": "990b29e03db5234f33dad6e1fa5929871234fa91c7f57ae49f27a1167d79423a", "body_bytes": 29050} +{"i": 2, "conv": "S1", "turn": 2, "arm": "drafted", "group": "", "history": 15548, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b", "cached": 6144, "prompt": 15550, "tokens": 46, "ttft": 6.130074685, "prefill": 6.127433175, "wall": 6.980270387954079, "finish": "tool_calls"}, "ok": true, "body_sha256": "f3b81e601375931ca8d96da0dca5e5dccf26cd5229c10a4760cd8d2712631710", "body_bytes": 52453} +{"i": 3, "conv": "S1", "turn": 2, "arm": "plain", "group": "", "history": 15548, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b", "cached": 14336, "prompt": 15550, "tokens": 46, "ttft": 0.830346183, "prefill": 0.827081829, "wall": 2.906474970979616, "finish": "tool_calls"}, "ok": true, "body_sha256": "293ad7bbfd939569ce1e34a3855067e1b1c8769b49a96a3dd5aeb250aa6d74d2", "body_bytes": 52467} +{"i": 4, "conv": "S1", "turn": 3, "arm": "drafted", "group": "", "history": 19460, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa", "cached": 14336, "prompt": 19462, "tokens": 96, "ttft": 3.434047037, "prefill": 3.430740029, "wall": 5.743769327993505, "finish": "tool_calls"}, "ok": true, "body_sha256": "b35a59e72c341d3e54e98292572c02da5bbd714bf1c5c6a598bf3d19542fa84d", "body_bytes": 64237} +{"i": 5, "conv": "S1", "turn": 3, "arm": "plain", "group": "", "history": 19460, "allowed": {"min": 11268, "max": 19460}, "got": {"sha": "8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa", "cached": 18432, "prompt": 19462, "tokens": 96, "ttft": 0.738095229, "prefill": 0.734514748, "wall": 5.067014549043961, "finish": "tool_calls"}, "ok": true, "body_sha256": "75be2cca32ffafe8e281cafb78b61125aa37274e48d06dfd9f5d469afa56b85b", "body_bytes": 64251} +{"i": 6, "conv": "S1", "turn": 4, "arm": "drafted", "group": "", "history": 21890, "allowed": {"min": 11268, "max": 19460}, "got": {"sha": "dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af", "cached": 18432, "prompt": 21892, "tokens": 113, "ttft": 2.31106115, "prefill": 2.307184658, "wall": 5.662535720970482, "finish": "tool_calls"}, "ok": true, "body_sha256": "5edfc245ff056e32f7c0773b5355c998b0050024551917d0944eec9c9fad7c18", "body_bytes": 76205} +{"i": 7, "conv": "S1", "turn": 4, "arm": "plain", "group": "", "history": 21890, "allowed": {"min": 13698, "max": 21890}, "got": {"sha": "dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af", "cached": 20480, "prompt": 21892, "tokens": 113, "ttft": 0.922836503, "prefill": 0.918251788, "wall": 6.0358214400475845, "finish": "tool_calls"}, "ok": true, "body_sha256": "46d76cf74f7632811ccda298b91f63d33cd5ea8e66a81ac506b8091f360f90cd", "body_bytes": 76219} +{"i": 8, "conv": "S1", "turn": 5, "arm": "drafted", "group": "", "history": 24386, "allowed": {"min": 13698, "max": 21890}, "got": {"sha": "8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa", "cached": 20480, "prompt": 24388, "tokens": 160, "ttft": 2.534417058, "prefill": 2.529780325, "wall": 7.120319110923447, "finish": "length"}, "ok": true, "body_sha256": "d4a044c7ab9fa548dd8f8b8a20631e0f0039657669c3035650aa36c4a86fd5e8", "body_bytes": 88280} +{"i": 9, "conv": "S1", "turn": 5, "arm": "plain", "group": "", "history": 24386, "allowed": {"min": 16194, "max": 24386}, "got": {"sha": "8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa", "cached": 22528, "prompt": 24388, "tokens": 160, "ttft": 1.131168964, "prefill": 1.126038313, "wall": 8.38478770502843, "finish": "length"}, "ok": true, "body_sha256": "de0e0037be2fda54a59f1f41322cd69cd265eba207ec762b6f62f071ebb13b35", "body_bytes": 88294} +{"i": 10, "conv": "S1", "turn": 6, "arm": "drafted", "group": "", "history": 26893, "allowed": {"min": 16194, "max": 24386}, "got": {"sha": "c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1", "cached": 22528, "prompt": 26895, "tokens": 154, "ttft": 3.091537947, "prefill": 3.086114849, "wall": 7.665547112934291, "finish": "tool_calls"}, "ok": true, "body_sha256": "b77ba1f3d48d49e101d2aee6fc3b8212f62dafbad99bc8e74d19bf0828c1176c", "body_bytes": 100825} +{"i": 11, "conv": "S1", "turn": 6, "arm": "plain", "group": "", "history": 26893, "allowed": {"min": 18701, "max": 26893}, "got": {"sha": "c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1", "cached": 26624, "prompt": 26895, "tokens": 154, "ttft": 0.393300354, "prefill": 0.38713773, "wall": 7.339349479996599, "finish": "tool_calls"}, "ok": true, "body_sha256": "d16dbb46641e7988599a9cb050a97d75f85414d58657ae383b57b36d82494196", "body_bytes": 100839} +{"i": 12, "conv": "S1", "turn": 7, "arm": "drafted", "group": "", "history": 29796, "allowed": {"min": 18701, "max": 26893}, "got": {"sha": "7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb", "cached": 26624, "prompt": 29798, "tokens": 136, "ttft": 2.206723533, "prefill": 2.200360998, "wall": 5.935562752070837, "finish": "tool_calls"}, "ok": true, "body_sha256": "0237bd9666ffbb8ac2b5bee32a17174e64623cc9adc5a662b9042fb58cc252b5", "body_bytes": 113157} +{"i": 13, "conv": "S1", "turn": 7, "arm": "plain", "group": "", "history": 29796, "allowed": {"min": 21604, "max": 29796}, "got": {"sha": "7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb", "cached": 28672, "prompt": 29798, "tokens": 136, "ttft": 0.814942134, "prefill": 0.807457136, "wall": 6.960275253048167, "finish": "tool_calls"}, "ok": true, "body_sha256": "47403ee22097841335be670be5d88ab401be8b9b59b7f9a560c86619dff77eef", "body_bytes": 113171} +{"i": 14, "conv": "S1", "turn": 8, "arm": "drafted", "group": "", "history": 35432, "allowed": {"min": 21604, "max": 29796}, "got": {"sha": "1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e", "cached": 28672, "prompt": 35434, "tokens": 160, "ttft": 4.629323788, "prefill": 4.621701032, "wall": 8.452136109001003, "finish": "length"}, "ok": true, "body_sha256": "3d1d416f1a16d2d44ed43707733e319e4850be7bca15f8d524e7e1c543329db3", "body_bytes": 136885} +{"i": 15, "conv": "S1", "turn": 8, "arm": "plain", "group": "", "history": 35432, "allowed": {"min": 27240, "max": 35432}, "got": {"sha": "1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e", "cached": 34816, "prompt": 35434, "tokens": 160, "ttft": 0.575534476, "prefill": 0.566396292, "wall": 7.813482742058113, "finish": "length"}, "ok": true, "body_sha256": "0bffc204ca59f3152a50080682ec7f804f552b6642b290db9c90554e540b3bfa", "body_bytes": 136899} +{"i": 16, "conv": "S1", "turn": 9, "arm": "drafted", "group": "", "history": 38150, "allowed": {"min": 27240, "max": 35432}, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 34816, "prompt": 38152, "tokens": 160, "ttft": 2.293306014, "prefill": 2.284637304, "wall": 4.669658715953119, "finish": "length"}, "ok": true, "body_sha256": "7f20fff3bd87dcda0ce3ac6d1f565364c2b432d579a954051fed8da73f5a81c5", "body_bytes": 149338} +{"i": 17, "conv": "S1", "turn": 9, "arm": "plain", "group": "", "history": 38150, "allowed": {"min": 29958, "max": 38150}, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 36864, "prompt": 38152, "tokens": 160, "ttft": 0.890302996, "prefill": 0.880882257, "wall": 8.154899014974944, "finish": "length"}, "ok": true, "body_sha256": "6a548de1e05b29b7246e89e798085f6de203bfe681b87912d392ac72fe0ac197", "body_bytes": 149352} +{"i": 18, "conv": "X", "turn": 1, "arm": "drafted", "group": "", "history": 2113, "allowed": [0], "got": {"sha": "7b1461bf71da95a2b74977aec25ecf7215b9f0fa044c82d2e1f4905ed66b798a", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.483307502, "prefill": 1.482367272, "wall": 5.776505226967856, "finish": "length"}, "ok": true, "body_sha256": "ea7f1fb5d610d52f549d66c8d5bac3876b0c32a1f369f7230266491a9a674737", "body_bytes": 9524} +{"i": 19, "conv": "A", "turn": 1, "arm": "drafted", "group": "", "history": 7410, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666", "cached": 6144, "prompt": 7412, "tokens": 90, "ttft": 0.895974236, "prefill": 0.894399952, "wall": 3.0761880530044436, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a153933bfbacdc64dd54d832df32d21139497927a2527fd60217851614cf603", "body_bytes": 28006} +{"i": 20, "conv": "A", "turn": 1, "arm": "plain", "group": "", "history": 7410, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666", "cached": 6144, "prompt": 7412, "tokens": 90, "ttft": 0.863995697, "prefill": 0.862083242, "wall": 4.910491242073476, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a6c1cda2af13b7752b30514bbb91dc76381167e03b207540e31a0f810a462b3", "body_bytes": 28020} +{"i": 21, "conv": "B", "turn": 1, "arm": "drafted", "group": "", "history": 7472, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb", "cached": 6144, "prompt": 7474, "tokens": 73, "ttft": 0.91993942, "prefill": 0.91836909, "wall": 2.492111276020296, "finish": "tool_calls"}, "ok": true, "body_sha256": "6685678d8326dff3c264f306810b45e76303da9dbc2ff27f9b4eec69f8a6384c", "body_bytes": 27976} +{"i": 22, "conv": "B", "turn": 1, "arm": "plain", "group": "", "history": 7472, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb", "cached": 6144, "prompt": 7474, "tokens": 73, "ttft": 0.883561743, "prefill": 0.881374376, "wall": 4.17014298599679, "finish": "tool_calls"}, "ok": true, "body_sha256": "ac1e48a68c55011216bc0008268ca26a6a788c311372fed6bc6dc9ed62769bf4", "body_bytes": 27990} +{"i": 23, "conv": "A", "turn": 2, "arm": "drafted", "group": "", "history": 12630, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe", "cached": 6144, "prompt": 12632, "tokens": 160, "ttft": 4.392286752, "prefill": 4.389820755, "wall": 8.673607576056384, "finish": "length"}, "ok": true, "body_sha256": "b9b2a6b8376c34c7a83ff0c89a5f6cdd47eac9e30a4e88a30c75ac65c25ff71c", "body_bytes": 51833} +{"i": 24, "conv": "A", "turn": 2, "arm": "plain", "group": "", "history": 12630, "allowed": {"min": 4438, "max": 12630}, "got": {"sha": "e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe", "cached": 12288, "prompt": 12632, "tokens": 160, "ttft": 0.427896627, "prefill": 0.424940424, "wall": 7.612685707979836, "finish": "length"}, "ok": true, "body_sha256": "2997010c96c6abe2561789c4d34cbcfe9e07dd46c741ec67a4e1b2658b205119", "body_bytes": 51847} +{"i": 25, "conv": "B", "turn": 2, "arm": "drafted", "group": "", "history": 12719, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2", "cached": 6144, "prompt": 12721, "tokens": 117, "ttft": 4.430895801, "prefill": 4.428539872, "wall": 7.170423732954077, "finish": "tool_calls"}, "ok": true, "body_sha256": "d000aaf5b179c43dc0745713a79cf04efa44eade03881c2bd096c7db63df55c9", "body_bytes": 51401} +{"i": 26, "conv": "B", "turn": 2, "arm": "plain", "group": "", "history": 12719, "allowed": {"min": 4527, "max": 12719}, "got": {"sha": "f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2", "cached": 12288, "prompt": 12721, "tokens": 117, "ttft": 0.466889771, "prefill": 0.46387771, "wall": 5.756194533081725, "finish": "tool_calls"}, "ok": true, "body_sha256": "4f1ea39aece017b0f4dbc29c172b772eb2e1e73e293d6d4825a439351fea9f57", "body_bytes": 51415} +{"i": 27, "conv": "A", "turn": 3, "arm": "drafted", "group": "", "history": 15381, "allowed": {"min": 4438, "max": 12630}, "got": {"sha": "7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8", "cached": 12288, "prompt": 15383, "tokens": 137, "ttft": 2.144120182, "prefill": 2.141098874, "wall": 6.085371199995279, "finish": "tool_calls"}, "ok": true, "body_sha256": "88ea4199cf95f9a7ce87b08887e68b3cc895c36a21e09648237031e9a85ab684", "body_bytes": 63752} +{"i": 28, "conv": "A", "turn": 3, "arm": "plain", "group": "", "history": 15381, "allowed": {"min": 7189, "max": 15381}, "got": {"sha": "7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8", "cached": 14336, "prompt": 15383, "tokens": 137, "ttft": 0.76783996, "prefill": 0.764200542, "wall": 7.009286788990721, "finish": "tool_calls"}, "ok": true, "body_sha256": "87cf9c6021b6c5a7fc1011f7af2975679ae2672e37b1190454f8204a35b434c8", "body_bytes": 63766} +{"i": 29, "conv": "B", "turn": 3, "arm": "drafted", "group": "", "history": 17328, "allowed": {"min": 4527, "max": 12719}, "got": {"sha": "8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49", "cached": 12288, "prompt": 17330, "tokens": 160, "ttft": 3.387332292, "prefill": 3.383525394, "wall": 7.459295344073325, "finish": "length"}, "ok": true, "body_sha256": "01aee2e1af92689ebf986f4736b4cb64662cf0c58b0c390d11bc140cad6e4b3f", "body_bytes": 75090} +{"i": 30, "conv": "B", "turn": 3, "arm": "plain", "group": "", "history": 17328, "allowed": {"min": 9136, "max": 17328}, "got": {"sha": "8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49", "cached": 16384, "prompt": 17330, "tokens": 160, "ttft": 0.730147735, "prefill": 0.726075896, "wall": 8.078089721035212, "finish": "length"}, "ok": true, "body_sha256": "66d42467f8020d9dab1f14d59831d6dd51476119b9b6f3cea1859b3409928f35", "body_bytes": 75104} +{"i": 31, "conv": "E", "turn": 1, "arm": "drafted", "group": "", "history": 40899, "allowed": {"min": 0, "max": 6611}, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 6144, "prompt": 40901, "tokens": 160, "ttft": 22.721882224, "prefill": 22.712871871, "wall": 24.58687757095322, "finish": "length"}, "ok": true, "body_sha256": "28a3c774651bf9ec3b06b537e215f1a3ee755fd95c81853119f2bb0e50bb93da", "body_bytes": 161836} +{"i": 32, "conv": "E", "turn": 1, "arm": "plain", "group": "", "history": 40899, "allowed": {"min": 32707, "max": 40899}, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 38912, "prompt": 40901, "tokens": 160, "ttft": 1.265668604, "prefill": 1.255718277, "wall": 8.59323136205785, "finish": "length"}, "ok": true, "body_sha256": "a792e1b1e47a964afec6f671b7507ffbe1556e2a3762818d25c8db929a376af7", "body_bytes": 161850} +{"i": 33, "conv": "T", "turn": 1, "arm": "drafted", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89", "cached": 0, "prompt": 1520, "tokens": 150, "ttft": 0.943877284, "prefill": 0.943270843, "wall": 4.778369015082717, "finish": "stop"}, "ok": true, "body_sha256": "45cd3d7f3f990fbf5badf7e4179006aece0dac6ff89e43c1a049c9c47e65c0af", "body_bytes": 6672} +{"i": 34, "conv": "T", "turn": 1, "arm": "plain", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89", "cached": 0, "prompt": 1520, "tokens": 150, "ttft": 0.91221373, "prefill": 0.911474581, "wall": 7.6072310850722715, "finish": "stop"}, "ok": true, "body_sha256": "17133648c09b8f37c9e3f3f57cf38d2952a766d4633876fb61d87b50e01f6e13", "body_bytes": 6686} +{"i": 35, "conv": "T", "turn": 2, "arm": "drafted", "group": "", "history": 4199, "allowed": [0], "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4202, "tokens": 12, "ttft": 3.032434677, "prefill": 3.031148946, "wall": 3.278865054016933, "finish": "stop"}, "ok": true, "body_sha256": "febc9e78b42428ba1d15c676438f3972b3e8b00c9da6d814d866087291e9b894", "body_bytes": 19045} +{"i": 36, "conv": "T", "turn": 2, "arm": "plain", "group": "", "history": 4199, "allowed": {"min": 1, "max": 4199}, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 4096, "prompt": 4202, "tokens": 12, "ttft": 0.269759614, "prefill": 0.267924326, "wall": 0.7799990230705589, "finish": "stop"}, "ok": true, "body_sha256": "b0d4fa26b0d4b22fbe1d08df95e42b66857dd19fd41ad69195d183c829e4b327", "body_bytes": 19059} +{"i": 37, "conv": "C", "turn": 1, "arm": "cancel", "group": "", "history": 0, "allowed": null, "got": {"cancelled_after": 4.0}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 38, "conv": "C", "turn": 1, "arm": "drafted", "group": "", "history": 9700, "allowed": {"min": 0, "max": 9688}, "got": {"sha": "88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c", "cached": 6144, "prompt": 9702, "tokens": 110, "ttft": 2.374964188, "prefill": 2.314261987, "wall": 5.137436063960195, "finish": "tool_calls"}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 39, "conv": "C", "turn": 1, "arm": "plain", "group": "", "history": 9700, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c", "cached": 8192, "prompt": 9702, "tokens": 110, "ttft": 0.973354306, "prefill": 0.971044849, "wall": 5.938178415992297, "finish": "tool_calls"}, "ok": true, "body_sha256": "a3d8ec39bf405406cdbde28193f74e4e2ac61b2857b32a308870da9fadd5869f", "body_bytes": 43551} +{"i": 40, "conv": "C", "turn": 2, "arm": "drafted", "group": "", "history": 15178, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8", "cached": 8192, "prompt": 15180, "tokens": 160, "ttft": 4.668291839, "prefill": 4.665382033, "wall": 8.96801668102853, "finish": "length"}, "ok": true, "body_sha256": "0297d3b1eda69c9be6f8a0dd01961ad12f0aaccf31f9a856a9b94ea91942a9e3", "body_bytes": 67043} +{"i": 41, "conv": "C", "turn": 2, "arm": "plain", "group": "", "history": 15178, "allowed": {"min": 6986, "max": 15178}, "got": {"sha": "5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8", "cached": 14336, "prompt": 15180, "tokens": 160, "ttft": 0.675530675, "prefill": 0.672016579, "wall": 7.873604597058147, "finish": "length"}, "ok": true, "body_sha256": "09a438ff02ed6f174d8f840fb46eb3daff9eff7d1e51c373b85913a8219a806b", "body_bytes": 67057} +{"i": 42, "conv": "A", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 18244, "allowed": {"min": 7189, "max": 15381}, "got": {"sha": "9599ff0fa9b19bbb5cb5218a5b6f538de1146ce12dad36c729686f12f666389e", "cached": 14336, "prompt": 18246, "tokens": 160, "ttft": 2.516291451, "prefill": 2.512418456, "wall": 10.345024258946069, "finish": "length"}, "ok": true, "body_sha256": "95b2fd052a3e91457eb959881525ef535efb4dabd02e7ad91c7b8f4356b23ab9", "body_bytes": 75737} +{"i": 43, "conv": "B", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 19806, "allowed": {"min": 9136, "max": 17328}, "got": {"sha": "025c4856c697674513547e9be3c33a84537e8a4fcb74e731ea09263af3e6d80d", "cached": 16384, "prompt": 19808, "tokens": 141, "ttft": 4.934404437, "prefill": 4.14292469, "wall": 9.58728368091397, "finish": "tool_calls"}, "ok": true, "body_sha256": "f65eec17dedd769f472f069e0b4c9a8f56a99b0e2715e3622618c3bccba9076c", "body_bytes": 86804} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/server-rank0.log new file mode 100644 index 0000000000..cef42b0e68 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/server-rank0.log @@ -0,0 +1,14157 @@ +[kvp-serve] 2026-10-22T16:54:20+00:00 rank 0 of 3 context 1048576 parallel 8 binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=unset TF_GLM_MM=unset args +info: rank 0: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:112:112 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:112:112 [0] NCCL INFO cudaDriverVersion 13030 +host-a:112:112 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:112:112 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:112:112 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:112:112 [0] NCCL INFO Count not open libibverbs.so +host-a:112:112 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:112:112 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:112:112 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:112:112 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x3913e7a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:112:112 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x3917fea0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:112:112 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x391c15a0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:112:112 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x39202ca0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:112:112 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:112:112 [0] NCCL INFO Initialized NET plugin IB +host-a:112:112 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:112:112 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:112:112 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:112:112 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:112:112 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:112:112 [0] NCCL INFO Using network IB +host-a:112:112 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:112:112 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x365070e0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xfa53eb0fed62d7d6 - Init START +host-a:112:112 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:112:112 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:112:112 [0] NCCL INFO Rank 0: 6 Net devices +host-a:112:112 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:112:112 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:112:112 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:112:112 [0] NCCL INFO comm 0x365070e0 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:112:112 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:112:112 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:112:112 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:112:112 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:112:112 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:112:112 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:112:112 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:112:112 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:112:130 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 10 +host-a:112:129 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-a:112:129 [0] NCCL INFO proxy listening socket at <32893> +host-a:112:131 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-a:112:112 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:112:112 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:112:112 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:112:112 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989c010000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989c054000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989c09c000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989c0e0000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6792 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6793 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6792 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6792 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6793 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6793 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1764 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1765 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1766 pkey=0 pd=0xef989c0989d0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6824 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6825 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6824 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ccccc ctsFifoLkey=0x1ccccc +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6824 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6825 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ccccc ctsFifoLkey=0x1ccccc +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6825 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989c277000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6792 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6793 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1892 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1893 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1894 pkey=0 pd=0xef989c224ec0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6794 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6795 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6794 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b3532 ctsFifoLkey=0x1b3532 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6794 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6795 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b3532 ctsFifoLkey=0x1b3532 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6795 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989c394000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6824 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6825 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6794 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6795 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1767 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1768 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1769 pkey=0 pd=0xef989c0989d0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989c44e000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989c492000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6826 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6827 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6826 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d8e88 ctsFifoLkey=0x1d8e88 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6826 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6827 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d8e88 ctsFifoLkey=0x1d8e88 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6827 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6826 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6827 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1895 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1896 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1897 pkey=0 pd=0xef989c224ec0 +host-a:112:112 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:112:112 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:112:112 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989c5a9000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989c5ea000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1770 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1771 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1770 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1770 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1771 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1771 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989c689000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989c6cd000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1898 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1899 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1898 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c2c1 ctsFifoLkey=0x19c2c1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1898 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1899 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c2c1 ctsFifoLkey=0x19c2c1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1899 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1770 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1771 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1898 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1899 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1772 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1773 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1772 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1772 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1773 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1773 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1900 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1901 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1900 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1900 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1901 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1901 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1772 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1773 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1900 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1901 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:112 [0] NCCL INFO Connected all trees +host-a:112:112 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:112:112 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989c81e000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989c863000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989c8a8000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989c8f0000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6796 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6797 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6798 pkey=0 pd=0xef989c053b70 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6828 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6829 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6830 pkey=0 pd=0xef989c1a7d10 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6799 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6800 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6801 pkey=0 pd=0xef989c053b70 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6831 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6832 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6833 pkey=0 pd=0xef989c1a7d10 +host-a:112:112 [0] NCCL INFO Connected binomial trees +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:112:112 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:112:112 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:112:112 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-a:112:112 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:112:112 [0] NCCL INFO ncclCommInitRank comm 0x365070e0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xfa53eb0fed62d7d6 - Init COMPLETE +host-a:112:112 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 6.56 (kernels 0.24, alloc 0.04, bootstrap 5.96, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.30, rest 0.00) +host-a:112:112 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:112:112 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:112:112 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:112:112 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:112:112 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:112:112 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:112:112 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:112:112 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989cb16000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989cb57000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989cb9f000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989cbe3000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989cc2b000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6802 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6803 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6802 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1be6e3 ctsFifoLkey=0x1be6e3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6802 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6803 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1be6e3 ctsFifoLkey=0x1be6e3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6803 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6834 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6835 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6834 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1be7ea ctsFifoLkey=0x1be7ea +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6834 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6835 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1be7ea ctsFifoLkey=0x1be7ea +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6835 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1902 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1903 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1902 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c0bd ctsFifoLkey=0x19c0bd +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1902 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1903 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c0bd ctsFifoLkey=0x19c0bd +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1903 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989cd80000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1774 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1775 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1776 pkey=0 pd=0xef989c0989d0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef989ce42000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6802 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6803 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6834 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6835 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1904 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1905 pkey=0 pd=0xef989c224ec0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1906 pkey=0 pd=0xef989c224ec0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1902 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1903 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1777 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1778 pkey=0 pd=0xef989c0989d0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1777 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x177632 ctsFifoLkey=0x177632 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1777 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1778 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x177632 ctsFifoLkey=0x177632 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1778 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef989cf53000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6836 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6837 pkey=0 pd=0xef989c1a7d10 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6838 pkey=0 pd=0xef989c1a7d10 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] Ninfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 78.03 GiB; 8 streams at once, the engine's own at a 1048576-token window (12.88 GiB, in the model's), 7 more taking 1284 pages of 512 rows (7.99 GiB) as they need them, 131 MiB fixed each, of 24.9 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 16.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 1048576; loaded in 249.3s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-96d955ef86e1194acd0f82cb4a5209e1 prompt=14 cached=0 thinking=False effort=none tokens=16 sha=e195338a9a9d finish=length rounds=6 accepted=9/13 ttft=0.112 decode=0.349 elapsed=0.461 queued=0.000 tok/s=43.0 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-b2561fc12758f49ce4fd9c42854fcc75 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.109 decode=5.242 elapsed=5.350 queued=0.000 tok/s=48.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-3dbdd3add6a0cf2470af72353291acee prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=108 accepted=147/202 ttft=0.108 decode=6.149 elapsed=6.257 queued=0.000 tok/s=41.5 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 4, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-ee6ae7d9fa6ebb8decac40d7b5fb342c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=106 accepted=150/212 ttft=0.108 decode=6.210 elapsed=6.317 queued=0.000 tok/s=41.1 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 5, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-920fcb9164ef7bdc1b6b65401f4c7333 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=105 accepted=150/218 ttft=0.108 decode=6.184 elapsed=6.291 queued=0.000 tok/s=41.2 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-f26f19f5074d3cb9cb8b65b5f1c72893 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=77 accepted=179/227 ttft=0.108 decode=4.918 elapsed=5.027 queued=0.000 tok/s=51.8 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-b08dfcc24606bc8d86b3c2eee6a42819 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=104 accepted=151/218 ttft=0.109 decode=6.134 elapsed=6.243 queued=0.000 tok/s=41.6 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 8, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-bcae31d3e641dcea5e846f94d61897ea prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=106 accepted=149/206 ttft=0.108 decode=6.199 elapsed=6.308 queued=0.000 tok/s=41.1 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 9, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-918b471704f54a5c2a50d89a5b603424 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=119 accepted=136/196 ttft=0.108 decode=6.664 elapsed=6.772 queued=0.000 tok/s=38.3 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 10, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-ba2efca993308c1c170a655538fadc69 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.157 elapsed=11.260 queued=0.000 tok/s=22.9 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 11, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-2585118e00b1fa1b52897be2f9677ea0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=255 accepted=0/0 ttft=0.101 decode=11.222 elapsed=11.324 queued=0.000 tok/s=22.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-42bd51a5c23bb750579305e59e506169 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.180 elapsed=11.283 queued=0.000 tok/s=22.8 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 13, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-307d088b09c2f37a3be2f713034c95e3 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.190 elapsed=11.293 queued=0.000 tok/s=22.8 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 14, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-463dca1b50764fd055bdc308805b9407 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.241 elapsed=11.344 queued=0.000 tok/s=22.7 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 15, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-d1c514542a0b5900613c6a945b98eece prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=255 accepted=0/0 ttft=0.105 decode=11.189 elapsed=11.294 queued=0.000 tok/s=22.8 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-c3dcc7173e3b98d2ea5d66013e93f37b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.196 elapsed=11.299 queued=0.000 tok/s=22.8 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 17, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-0d5905b6b61b4010fe4d8b05454c95b8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=255 accepted=0/0 ttft=0.101 decode=11.225 elapsed=11.326 queued=0.000 tok/s=22.7 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 18, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-bec69f6973fed72ae6549f12aeb0f1c6 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.109 decode=5.246 elapsed=5.355 queued=0.000 tok/s=48.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done cmpl-bc9867b686651582a88aaff8b5eb7b68 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.274 decode=7.332 elapsed=7.605 queued=0.000 tok/s=34.8 seed=1234 +[tensorfold] done cmpl-ca1c7ecfece73a1e3feab367a44b8239 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=96 accepted=159/273 ttft=0.108 decode=8.577 elapsed=8.685 queued=0.000 tok/s=29.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 22, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-11b90156427a320f513e11cfbf80ae5e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.274 decode=11.046 elapsed=11.321 queued=0.000 tok/s=23.1 seed=1234 +[tensorfold] done cmpl-747fa79e39ac1045315377ff6dd8c59d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=89 accepted=166/264 ttft=0.460 decode=12.389 elapsed=12.849 queued=0.165 tok/s=20.6 seed=1236 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done cmpl-dc440ecc0da03291676b5c70e678a9b8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=89 accepted=167/264 ttft=0.678 decode=12.251 elapsed=12.928 queued=0.165 tok/s=20.8 seed=1237 +[tensorfold] done cmpl-7e7d6355d396d733bbadf98624fcb1ba prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=96 accepted=160/285 ttft=0.110 decode=13.074 elapsed=13.183 queued=0.000 tok/s=19.5 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 26, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-716463a0f10b74f103d4b5ac124b7087 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.463 decode=17.227 elapsed=17.691 queued=0.165 tok/s=14.8 seed=1234 +[tensorfold] done cmpl-53568c3197f95d3c22b1b91272c4b2e8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=79 accepted=176/236 ttft=1.159 decode=17.547 elapsed=18.706 queued=0.164 tok/s=14.5 seed=1238 +[tensorfold] done cmpl-597eeeaea8db5140dd8acc0f18337e6c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=89 accepted=167/263 ttft=0.108 decode=19.507 elapsed=19.615 queued=0.000 tok/s=13.1 seed=1237 +[tensorfold] done cmpl-e25a6fd3688aed2c743fe94cf951e7d4 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=89 accepted=166/264 ttft=0.904 decode=19.358 elapsed=20.261 queued=0.164 tok/s=13.2 seed=1239 +[tensorfold] done cmpl-21fdc4dd240743643e44fc1a30e14838 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=89 accepted=166/264 ttft=1.730 decode=18.934 elapsed=20.663 queued=0.163 tok/s=13.5 seed=1236 +[tensorfold] done cmpl-c6c8ac40c552fc683bc1a9721286dbca prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=91 accepted=164/269 ttft=1.435 decode=19.344 elapsed=20.779 queued=0.164 tok/s=13.2 seed=1240 +[tensorfold] done cmpl-3a73b7deda321912326343343f0b21dc prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=95 accepted=160/284 ttft=0.676 decode=20.197 elapsed=20.873 queued=0.164 tok/s=12.6 seed=1235 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done cmpl-b613d92e73130c854649a47364a3a1d3 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=103 accepted=152/301 ttft=0.274 decode=20.938 elapsed=21.213 queued=0.000 tok/s=12.2 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 33, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-a6cfb1fa8b7b1e93ca2315eaaa6ff8ce prompt=32 cached=0 thinking=False effort=none tokens=16 sha=3f46b826a241 finish=length rounds=10 accepted=5/13 ttft=0.158 decode=0.538 elapsed=0.696 queued=0.000 tok/s=27.9 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 34, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-a16df8d9cd81b412a6f05178af385def prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=135 accepted=120/176 ttft=0.159 decode=7.235 elapsed=7.395 queued=0.000 tok/s=35.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 35, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-27a35fb54165cfc5b16974d7ac3a68db prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=106 accepted=150/234 ttft=0.161 decode=6.381 elapsed=6.541 queued=0.000 tok/s=40.0 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 36, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0dd1cb74c07b1dd1edc15dd824d326e2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=133 accepted=123/191 ttft=0.160 decode=7.254 elapsed=7.413 queued=0.000 tok/s=35.2 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 37, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-034687d431e64dd36b8399c834b68886 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=141 accepted=115/175 ttft=0.160 decode=7.492 elapsed=7.652 queued=0.000 tok/s=34.0 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 38, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-d43f731d1ab6d3dcb11e12c10802087b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=135 accepted=120/197 ttft=0.160 decode=7.434 elapsed=7.594 queued=0.000 tok/s=34.3 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 39, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-8c28ad900fe83e61c83c50c1b15c4c94 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=128 accepted=128/208 ttft=0.158 decode=7.161 elapsed=7.319 queued=0.000 tok/s=35.6 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 40, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-8f5ce595a1e54ee6ed7e372b56810f15 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=124 accepted=131/205 ttft=0.159 decode=6.943 elapsed=7.102 queued=0.000 tok/s=36.7 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 41, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-c3bb9543ce3216ade7c9058c921256d9 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=123 accepted=132/204 ttft=0.158 decode=6.919 elapsed=7.077 queued=0.000 tok/s=36.9 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-8c9a55dddc6d47c56676f7e534a78e1d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=255 accepted=0/0 ttft=0.150 decode=11.184 elapsed=11.334 queued=0.000 tok/s=22.8 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 43, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-fde0dfafb320582c590685765b5d91dc prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.189 elapsed=11.342 queued=0.000 tok/s=22.8 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 44, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-5a4363db871d0776b5d1af7cf3d88045 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=255 accepted=0/0 ttft=0.151 decode=11.181 elapsed=11.332 queued=0.000 tok/s=22.8 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 45, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0f73a2914ac7ac9ea07bcb4677a21981 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=255 accepted=0/0 ttft=0.150 decode=11.182 elapsed=11.332 queued=0.000 tok/s=22.8 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 46, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-7f0d12614b785a3d98274001b4dc10de prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=255 accepted=0/0 ttft=0.150 decode=11.213 elapsed=11.364 queued=0.000 tok/s=22.7 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 47, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-116e3916a6f2c7a928cb98fdcfddd0b2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.183 elapsed=11.335 queued=0.000 tok/s=22.8 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-913b363ed30107d46e3b8a5d5db7aa11 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.245 elapsed=11.398 queued=0.000 tok/s=22.7 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 49, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0cce5b6e6cf8bf3fa9eabc33ec1863fb prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.225 elapsed=11.377 queued=0.000 tok/s=22.7 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 50, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-87e89b0a1971fac234866b8de0a8beb1 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=135 accepted=120/176 ttft=0.161 decode=7.250 elapsed=7.411 queued=0.000 tok/s=35.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 51, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-30f4b38901aedde122145c8424badc24 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=94 accepted=161/280 ttft=0.161 decode=9.061 elapsed=9.222 queued=0.000 tok/s=28.1 seed=1235 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done chatcmpl-445811671eddd7abdd7d97fda77e57b0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=104 accepted=151/291 ttft=0.380 decode=9.413 elapsed=9.793 queued=0.222 tok/s=27.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 53, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-9b332f5ea7da5be87e984512fe64d4bb prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=93 accepted=162/278 ttft=0.384 decode=13.227 elapsed=13.611 queued=0.224 tok/s=19.3 seed=1235 +[tensorfold] done chatcmpl-dfded8cf790e9bc6a2585d96b7acdd57 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=102 accepted=154/302 ttft=0.162 decode=14.391 elapsed=14.553 queued=0.000 tok/s=17.7 seed=1234 +[tensorfold] done chatcmpl-1652fab57a84c3f14551b9d1daeea9b1 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=100 accepted=156/297 ttft=0.893 decode=13.741 elapsed=14.634 queued=0.224 tok/s=18.6 seed=1237 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done chatcmpl-da7a57b6b7a8fc1a1b683dce4b812145 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=105 accepted=151/310 ttft=0.625 decode=14.245 elapsed=14.870 queued=0.224 tok/s=17.9 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 57, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-a67e80cb03f65e92ef4e1e813471cde1 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=93 accepted=162/278 ttft=1.477 decode=20.455 elapsed=21.932 queued=0.216 tok/s=12.5 seed=1235 +[tensorfold] done chatcmpl-612a46fe17a648b3f470e289c0553cf0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=101 accepted=154/298 ttft=0.375 decode=22.347 elapsed=22.722 queued=0.217 tok/s=11.4 seed=1241 +[tensorfold] done chatcmpl-bd46b976dae7a67cbd0e290e7fda7492 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=103 accepted=153/304 ttft=0.884 decode=22.551 elapsed=23.435 queued=0.216 tok/s=11.3 seed=1239 +[tensorfold] done chatcmpl-a414036ff8515ef64f8820d259bc10e2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=100 accepted=156/297 ttft=1.792 decode=21.643 elapsed=23.435 queued=0.216 tok/s=11.8 seed=1237 +[tensorfold] done chatcmpl-291d88b52cc7583dcd18747e4cbe35c1 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=106 accepted=149/316 ttft=0.615 decode=23.083 elapsed=23.698 queued=0.217 tok/s=11.0 seed=1240 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done chatcmpl-86c34cfe66a7bb3ff4148b38fa6775bc prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=102 accepted=154/303 ttft=2.126 decode=21.661 elapsed=23.787 queued=0.215 tok/s=11.8 seed=1234 +[tensorfold] done chatcmpl-fcfd3f0187ab7198eda624973c25e812 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=105 accepted=151/311 ttft=1.167 decode=22.620 elapsed=23.787 queued=0.216 tok/s=11.3 seed=1236 +[tensorfold] done chatcmpl-2e963ae1e1ec63722e12b9ef1e8e910e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=112 accepted=143/329 ttft=0.159 decode=23.796 elapsed=23.956 queued=0.000 tok/s=10.7 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 65, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-56d78688bf2a93853a4a704da2897050 prompt=14 cached=0 thinking=False effort=none tokens=16 sha=3000ccd48624 finish=length rounds=5 accepted=10/11 ttft=0.108 decode=0.299 elapsed=0.408 queued=0.000 tok/s=50.1 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 66, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-d4ab3ad57fbde3e6e348aadbf9b91246 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=51 accepted=204/222 ttft=0.109 decode=3.611 elapsed=3.720 queued=0.000 tok/s=70.6 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 67, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-7f8a27abd46b88d130c2e919f5895c46 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=255 accepted=0/0 ttft=0.101 decode=11.220 elapsed=11.321 queued=0.000 tok/s=22.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 68, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-72cf9895f622881d0ce718b134c3dccc prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=51 accepted=204/222 ttft=0.109 decode=3.615 elapsed=3.725 queued=0.000 tok/s=70.5 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 69, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 70, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-083714eb4c9f3f969a35bca365c4f0c5 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.109 decode=6.181 elapsed=6.289 queued=0.000 tok/s=41.3 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done cmpl-66625481e74756830602b5b8c192685d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.277 decode=6.069 elapsed=6.346 queued=0.169 tok/s=42.0 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 71, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 74, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 74, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 74, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-8649767eaae17de628dcb1de699f59a7 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.110 decode=8.611 elapsed=8.720 queued=0.000 tok/s=29.6 +[tensorfold] done cmpl-e56bc97b58cff160502df122b518a9a0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.277 decode=8.544 elapsed=8.821 queued=0.168 tok/s=29.8 +[tensorfold] done cmpl-e3c3d5084371b384030491c1af9f2b81 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.465 decode=8.438 elapsed=8.903 queued=0.168 tok/s=30.2 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done cmpl-645cebec862743fcf8ffe2fec52312f2 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.676 decode=8.285 elapsed=8.961 queued=0.168 tok/s=30.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 75, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-a7e5e773461d342a4c50e650b241289f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.110 decode=12.632 elapsed=12.743 queued=0.000 tok/s=20.2 +[tensorfold] done cmpl-c4f047741bc68e071cdca5d8c43590cf prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.275 decode=12.625 elapsed=12.899 queued=0.168 tok/s=20.2 +[tensorfold] done cmpl-b428e2145d7c892bfdeb404f0e3a5d24 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.462 decode=12.580 elapsed=13.043 queued=0.167 tok/s=20.3 +[tensorfold] done cmpl-dae10a414239558636f0b6285aaa89e7 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.673 decode=12.498 elapsed=13.171 queued=0.167 tok/s=20.4 +[tensorfold] done cmpl-803af9f7522a66b571515bd09bc19fcf prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.899 decode=12.386 elapsed=13.284 queued=0.167 tok/s=20.6 +[tensorfold] done cmpl-961eb2c5613fd2faf1c3099c5d586de1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.144 decode=12.241 elapsed=13.386 queued=0.167 tok/s=20.8 +[tensorfold] done cmpl-711bd20013f64aa37208191966cf663f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.403 decode=12.067 elapsed=13.470 queued=0.166 tok/s=21.1 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done cmpl-8eea6f5dc079f45caf74be2b51fa4214 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.679 decode=11.848 elapsed=13.527 queued=0.166 tok/s=21.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 83, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-79742bf37f52352f0f9fe64948c12571 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=e3dfbb415b68 finish=length rounds=5 accepted=10/12 ttft=0.159 decode=0.311 elapsed=0.470 queued=0.000 tok/s=48.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 84, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-18e9a7929006209dc2b4f5f4495334eb prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=136 accepted=119/185 ttft=0.160 decode=7.298 elapsed=7.458 queued=0.000 tok/s=34.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 85, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-155e869e0b2785595d29d70df9d46d0d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=255 accepted=0/0 ttft=0.150 decode=11.230 elapsed=11.380 queued=0.000 tok/s=22.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 86, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-29ddd378f685d885f6b86f2c863a9816 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=136 accepted=119/185 ttft=0.158 decode=7.290 elapsed=7.448 queued=0.000 tok/s=35.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 87, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 88, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-feba24a5dc394cf89e800eb42dbb4f8f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.160 decode=10.039 elapsed=10.199 queued=0.000 tok/s=25.4 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done chatcmpl-4c95541ac9a228b6993dc65ed332913c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.378 decode=9.869 elapsed=10.247 queued=0.220 tok/s=25.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 90, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-8ad867afedf263e53508fcd242c30cb9 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.160 decode=14.442 elapsed=14.602 queued=0.000 tok/s=17.7 +[tensorfold] done chatcmpl-90f38d28fd18b866ecdd83446b37e7e3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.377 decode=14.316 elapsed=14.693 queued=0.000 tok/s=17.8 +[tensorfold] done chatcmpl-a94192f7270c1977e4488dd1dcb8a93c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.619 decode=14.141 elapsed=14.759 queued=0.220 tok/s=18.0 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done chatcmpl-c39f7d2962e48aa5412b707cd2e1b80f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.884 decode=13.923 elapsed=14.807 queued=0.220 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 93, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0087f8b8332b4d4263e19b1060af378c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.159 decode=22.339 elapsed=22.497 queued=0.000 tok/s=11.4 +[tensorfold] done chatcmpl-41b3a7dcac0493b523f8d3c3c4950d1e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.375 decode=22.277 elapsed=22.653 queued=0.219 tok/s=11.4 +[tensorfold] done chatcmpl-e9ed3db78fbbe0fa413a2e91afd55519 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.618 decode=22.176 elapsed=22.794 queued=0.218 tok/s=11.5 +[tensorfold] done chatcmpl-a5a2f08a70ca84f3a309b7a537bd192b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.882 decode=22.041 elapsed=22.923 queued=0.217 tok/s=11.6 +[tensorfold] done chatcmpl-fd1010de34a40f96520dc4cb8e5729a8 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.159 decode=21.875 elapsed=23.033 queued=0.217 tok/s=11.7 +[tensorfold] done chatcmpl-1c7e4a9a057e37b44af1b103e7179b08 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.452 decode=21.671 elapsed=23.123 queued=0.216 tok/s=11.8 +[tensorfold] done chatcmpl-e3f5a0236c852d00f4d2d2492d215570 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.763 decode=21.427 elapsed=23.190 queued=0.216 tok/s=11.9 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done chatcmpl-9e56e29d3ef761b0f68b0a5052d1dd71 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=2.091 decode=21.148 elapsed=23.239 queued=0.216 tok/s=12.1 +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 0, kept 2; 2 states, 392 MiB and 15992 MiB spare of 16384 MiB (hits 0, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-b643a0437a53141c6f559aad7c34c93e prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=29 accepted=32/43 ttft=5.120 decode=1.630 elapsed=6.750 queued=0.000 tok/s=36.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 6144, kept 0; 2 states, 392 MiB and 15992 MiB spare of 16384 MiB (hits 1, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-c3789e28c1654e1de6433a3a7d33c95d prompt=7759 cached=6144 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=1.013 decode=2.748 elapsed=3.761 queued=0.000 tok/s=21.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 6144, kept 1; 3 states, 702 MiB and 15682 MiB spare of 16384 MiB (hits 2, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-5133601064687dd331d854cce61ed886 prompt=15550 cached=6144 thinking=True effort=high tokens=46 sha=539973cb8b13 finish=stop rounds=13 accepted=33/36 ttft=6.130 decode=0.849 elapsed=6.979 queued=0.000 tok/s=53.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 14336, kept 0; 3 states, 702 MiB and 15682 MiB spare of 16384 MiB (hits 3, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-1012f24b2499800e68456019acc9c198 prompt=15550 cached=14336 thinking=True effort=high tokens=46 sha=539973cb8b13 finish=stop rounds=45 accepted=0/0 ttft=0.830 decode=2.074 elapsed=2.904 queued=0.000 tok/s=21.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 19462 tokens, resumed at 14336, kept 1; 4 states, 1064 MiB and 15320 MiB spare of 16384 MiB (hits 4, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-fa55bb41b1cf330e8fc6d37b9d32d1af prompt=19462 cached=14336 thinking=True effort=high tokens=96 sha=8fc3c26162f1 finish=stop rounds=38 accepted=58/82 ttft=3.434 decode=2.308 elapsed=5.742 queued=0.000 tok/s=41.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 19462 tokens, resumed at 18432, kept 0; 4 states, 1064 MiB and 15320 MiB spare of 16384 MiB (hits 5, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-8fba426fa7bee0639bef1e062f548f18 prompt=19462 cached=18432 thinking=True effort=high tokens=96 sha=8fc3c26162f1 finish=stop rounds=95 accepted=0/0 ttft=0.738 decode=4.327 elapsed=5.065 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 21892 tokens, resumed at 18432, kept 1; 5 states, 1452 MiB and 14932 MiB spare of 16384 MiB (hits 6, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-7979f774e10c0f9e1eb9d91da7f4a7cd prompt=21892 cached=18432 thinking=True effort=high tokens=113 sha=dedd451779f3 finish=stop rounds=61 accepted=52/80 ttft=2.311 decode=3.350 elapsed=5.661 queued=0.000 tok/s=33.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 21892 tokens, resumed at 20480, kept 0; 5 states, 1452 MiB and 14932 MiB spare of 16384 MiB (hits 7, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-86ae8b82e3a4826675127b7ae9e34da7 prompt=21892 cached=20480 thinking=True effort=high tokens=113 sha=dedd451779f3 finish=stop rounds=112 accepted=0/0 ttft=0.923 decode=5.111 elapsed=6.034 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 24388 tokens, resumed at 20480, kept 1; 6 states, 1864 MiB and 14520 MiB spare of 16384 MiB (hits 8, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-af62c43e63f7700f172e47f1f7681cd1 prompt=24388 cached=20480 thinking=True effort=high tokens=160 sha=8bcbd355dc04 finish=length rounds=81 accepted=79/121 ttft=2.534 decode=4.584 elapsed=7.119 queued=0.000 tok/s=34.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 24388 tokens, resumed at 22528, kept 0; 6 states, 1864 MiB and 14520 MiB spare of 16384 MiB (hits 9, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-97cac8c8512509a560f7edd8fd4838f4 prompt=24388 cached=22528 thinking=True effort=high tokens=160 sha=8bcbd355dc04 finish=length rounds=159 accepted=0/0 ttft=1.131 decode=7.251 elapsed=8.382 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 26895 tokens, resumed at 22528, kept 1; 7 states, 2328 MiB and 14056 MiB spare of 16384 MiB (hits 10, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-2f6b5b996f5ad81ff811f9b988bd4b03 prompt=26895 cached=22528 thinking=True effort=high tokens=154 sha=c8e79afc5a10 finish=stop rounds=81 accepted=72/122 ttft=3.092 decode=4.572 elapsed=7.664 queued=0.000 tok/s=33.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 26895 tokens, resumed at 26624, kept 0; 7 states, 2328 MiB and 14056 MiB spare of 16384 MiB (hits 11, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-d125ef147c56345a880d59463025cd93 prompt=26895 cached=26624 thinking=True effort=high tokens=154 sha=c8e79afc5a10 finish=stop rounds=153 accepted=0/0 ttft=0.393 decode=6.943 elapsed=7.337 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 29798 tokens, resumed at 26624, kept 1; 8 states, 2818 MiB and 13566 MiB spare of 16384 MiB (hits 12, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-5c1d0f51899d777b99b95178a903fd77 prompt=29798 cached=26624 thinking=True effort=high tokens=136 sha=7629aa6bf5df finish=stop rounds=66 accepted=69/100 ttft=2.207 decode=3.727 elapsed=5.934 queued=0.000 tok/s=36.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 29798 tokens, resumed at 28672, kept 0; 8 states, 2818 MiB and 13566 MiB spare of 16384 MiB (hits 13, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-6982bfbffacc7e5caafb4aa716d6fc3a prompt=29798 cached=28672 thinking=True effort=high tokens=136 sha=7629aa6bf5df finish=stop rounds=135 accepted=0/0 ttft=0.815 decode=6.143 elapsed=6.958 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 35434 tokens, resumed at 28672, kept 1; 9 states, 3384 MiB and 13000 MiB spare of 16384 MiB (hits 14, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-b75c3529925d1a4070478d7d3933887d prompt=35434 cached=28672 thinking=True effort=high tokens=160 sha=1adac255ff32 finish=length rounds=60 accepted=99/148 ttft=4.629 decode=3.821 elapsed=8.450 queued=0.000 tok/s=41.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 35434 tokens, resumed at 34816, kept 0; 9 states, 3384 MiB and 13000 MiB spare of 16384 MiB (hits 15, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-63cff4ad1913d068fe9adce675465758 prompt=35434 cached=34816 thinking=True effort=high tokens=160 sha=1adac255ff32 finish=length rounds=159 accepted=0/0 ttft=0.576 decode=7.235 elapsed=7.811 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 38152 tokens, resumed at 34816, kept 1; 10 states, 3976 MiB and 12408 MiB spare of 16384 MiB (hits 16, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-de07ba08e795a6f5c5d608d057be4527 prompt=38152 cached=34816 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=34 accepted=125/133 ttft=2.293 decode=2.374 elapsed=4.668 queued=0.000 tok/s=67.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 38152 tokens, resumed at 36864, kept 0; 10 states, 3976 MiB and 12408 MiB spare of 16384 MiB (hits 17, misses 101, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-cab05ee680d36ce2e2d1b83729276e4b prompt=38152 cached=36864 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=159 accepted=0/0 ttft=0.890 decode=7.262 elapsed=8.153 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 2115 tokens, resumed at 0, kept 1; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 17, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-320dcac5f5491632b15448de24e91098 prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=7b1461bf71da finish=length rounds=75 accepted=84/126 ttft=1.483 decode=4.292 elapsed=5.775 queued=0.000 tok/s=37.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 18, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-023cf0834c43b2fdd1f9b246766b5bae prompt=7412 cached=6144 thinking=True effort=high tokens=90 sha=d0573eef0609 finish=stop rounds=35 accepted=55/81 ttft=0.896 decode=2.179 elapsed=3.075 queued=0.000 tok/s=40.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 19, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-4f1321b003b78aaeec9a019f8f8d2ece prompt=7412 cached=6144 thinking=True effort=high tokens=90 sha=d0573eef0609 finish=stop rounds=89 accepted=0/0 ttft=0.864 decode=4.045 elapsed=4.909 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 20, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-7279483dc7fe71a57af631d7efe4065e prompt=7474 cached=6144 thinking=True effort=high tokens=73 sha=9a2685e9905f finish=stop rounds=24 accepted=49/65 ttft=0.920 decode=1.571 elapsed=2.491 queued=0.000 tok/s=45.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 21, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-5e984615711534927724aa38eff2feae prompt=7474 cached=6144 thinking=True effort=high tokens=73 sha=9a2685e9905f finish=stop rounds=72 accepted=0/0 ttft=0.884 decode=3.285 elapsed=4.168 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12632 tokens, resumed at 6144, kept 1; 12 states, 4408 MiB and 11976 MiB spare of 16384 MiB (hits 22, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0d8ca74a4303d95c5a254808e63ae3a1 prompt=12632 cached=6144 thinking=True effort=high tokens=160 sha=e1f2511e60f3 finish=length rounds=75 accepted=84/122 ttft=4.392 decode=4.279 elapsed=8.672 queued=0.000 tok/s=37.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12632 tokens, resumed at 12288, kept 0; 12 states, 4408 MiB and 11976 MiB spare of 16384 MiB (hits 23, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-e64fc67a3ede5cbac8a2ceef1c5341d1 prompt=12632 cached=12288 thinking=True effort=high tokens=160 sha=e1f2511e60f3 finish=length rounds=159 accepted=0/0 ttft=0.428 decode=7.183 elapsed=7.611 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12721 tokens, resumed at 6144, kept 1; 13 states, 4694 MiB and 11690 MiB spare of 16384 MiB (hits 24, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-ec52bbeb8eef66ee9aa120af9ea6718d prompt=12721 cached=6144 thinking=True effort=high tokens=117 sha=f202ab6075ae finish=stop rounds=44 accepted=72/103 ttft=4.431 decode=2.738 elapsed=7.169 queued=0.000 tok/s=42.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12721 tokens, resumed at 12288, kept 0; 13 states, 4694 MiB and 11690 MiB spare of 16384 MiB (hits 25, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-4ec7b2ece8aa0a1ddf4f50903416d974 prompt=12721 cached=12288 thinking=True effort=high tokens=117 sha=f202ab6075ae finish=stop rounds=116 accepted=0/0 ttft=0.467 decode=5.288 elapsed=5.755 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15383 tokens, resumed at 12288, kept 1; 14 states, 5004 MiB and 11380 MiB spare of 16384 MiB (hits 26, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0f7de839ec045bce1f9b3483fe365191 prompt=15383 cached=12288 thinking=True effort=high tokens=137 sha=7695627844cd finish=stop rounds=70 accepted=67/97 ttft=2.144 decode=3.940 elapsed=6.084 queued=0.000 tok/s=34.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15383 tokens, resumed at 14336, kept 0; 14 states, 5004 MiB and 11380 MiB spare of 16384 MiB (hits 27, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-bf208b5c17e5fbeaadf2f1ca701eb328 prompt=15383 cached=14336 thinking=True effort=high tokens=137 sha=7695627844cd finish=stop rounds=136 accepted=0/0 ttft=0.768 decode=6.240 elapsed=7.008 queued=0.000 tok/s=21.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 17330 tokens, resumed at 12288, kept 1; 15 states, 5340 MiB and 11044 MiB spare of 16384 MiB (hits 28, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-70075222118f4355417848d235d3c7c3 prompt=17330 cached=12288 thinking=True effort=high tokens=160 sha=8b15f58457e8 finish=length rounds=69 accepted=90/124 ttft=3.387 decode=4.070 elapsed=7.457 queued=0.000 tok/s=39.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 17330 tokens, resumed at 16384, kept 0; 15 states, 5340 MiB and 11044 MiB spare of 16384 MiB (hits 29, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ca87bbbbe72aae24d01360e7765b193d prompt=17330 cached=16384 thinking=True effort=high tokens=160 sha=8b15f58457e8 finish=length rounds=159 accepted=0/0 ttft=0.730 decode=7.346 elapsed=8.076 queued=0.000 tok/s=21.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 40901 tokens, resumed at 6144, kept 1; 16 states, 5956 MiB and 10428 MiB spare of 16384 MiB (hits 30, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-0e02af30b0686b4c8cd9e2682b0a5f4a prompt=40901 cached=6144 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=22 accepted=137/137 ttft=22.722 decode=1.863 elapsed=24.585 queued=0.000 tok/s=85.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 40901 tokens, resumed at 38912, kept 0; 16 states, 5956 MiB and 10428 MiB spare of 16384 MiB (hits 31, misses 102, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ed7d5412ccbb07b69e7437837cdf6df6 prompt=40901 cached=38912 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=159 accepted=0/0 ttft=1.266 decode=7.326 elapsed=8.591 queued=0.000 tok/s=21.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 16 states, 5956 MiB and 10428 MiB spare of 16384 MiB (hits 31, misses 103, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-6e0286503428c881d5e2e27c01317279 prompt=1520 cached=0 thinking=False effort=none tokens=150 sha=033a197da586 finish=stop rounds=64 accepted=85/131 ttft=0.944 decode=3.833 elapsed=4.777 queued=0.000 tok/s=38.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 16 states, 5956 MiB and 10428 MiB spare of 16384 MiB (hits 31, misses 104, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-a52affa103eca2559c0ee7bfedf9073e prompt=1520 cached=0 thinking=False effort=none tokens=150 sha=033a197da586 finish=stop rounds=149 accepted=0/0 ttft=0.912 decode=6.693 elapsed=7.606 queued=0.000 tok/s=22.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 4202 tokens, resumed at 0, kept 1; 17 states, 6140 MiB and 10244 MiB spare of 16384 MiB (hits 31, misses 105, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-5ec50096b58ea810378260b3612472a6 prompt=4202 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.032 decode=0.245 elapsed=3.278 queued=0.000 tok/s=44.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 4202 tokens, resumed at 4096, kept 0; 17 states, 6140 MiB and 10244 MiB spare of 16384 MiB (hits 32, misses 105, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-6fa3667116bd3d902cf5386175da50d2 prompt=4202 cached=4096 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=0.270 decode=0.508 elapsed=0.778 queued=0.000 tok/s=21.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] ended chatcmpl-c3dd543cffae9a8bc2cf734371345da5 reason=client-left prompt=9702 tokens=0 after=4.08s +info: prompt cache: 9702 tokens, resumed at 6144, kept 1; 19 states, 6582 MiB and 9802 MiB spare of 16384 MiB (hits 33, misses 106, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-ccff8e5c61cd554c8dc167b3018b119a prompt=9702 cached=6144 thinking=True effort=high tokens=110 sha=88f98460968a finish=stop rounds=47 accepted=63/86 ttft=2.375 decode=2.761 elapsed=5.136 queued=0.059 tok/s=39.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 9702 tokens, resumed at 8192, kept 0; 19 states, 6582 MiB and 9802 MiB spare of 16384 MiB (hits 34, misses 106, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-b3412f6e13d4c57f2b4ea2fba2d35662 prompt=9702 cached=8192 thinking=True effort=high tokens=110 sha=88f98460968a finish=stop rounds=109 accepted=0/0 ttft=0.973 decode=4.963 elapsed=5.936 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15180 tokens, resumed at 8192, kept 1; 20 states, 6892 MiB and 9492 MiB spare of 16384 MiB (hits 35, misses 106, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-137e22ddf4a27aa6b1a545773e2a546f prompt=15180 cached=8192 thinking=True effort=high tokens=160 sha=5694b66a8995 finish=length rounds=76 accepted=83/115 ttft=4.668 decode=4.298 elapsed=8.966 queued=0.000 tok/s=37.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15180 tokens, resumed at 14336, kept 0; 20 states, 6892 MiB and 9492 MiB spare of 16384 MiB (hits 36, misses 106, evicted 0, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ac2cdabed31112defec11b9b7f60d274 prompt=15180 cached=14336 thinking=True effort=high tokens=160 sha=5694b66a8995 finish=length rounds=159 accepted=0/0 ttft=0.676 decode=7.196 elapsed=7.872 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 18246 tokens, resumed at 14336, kept 1; 21 states, 7228 MiB and 9156 MiB spare of 16384 MiB (hits 37, misses 106, evicted 0, refused 0, failed 0) +info: prompt cache: 19808 tokens, resumed at 16384, kept 2; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 106, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done chatcmpl-dbad07deb9cd4705c5838bbd625fb6cf prompt=19808 cached=16384 thinking=True effort=high tokens=141 sha=025c4856c697 finish=stop rounds=48 accepted=93/143 ttft=4.934 decode=4.651 elapsed=9.585 queued=0.787 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-b4db73a9e2a68db3b5ef1e2a13b9119b prompt=18246 cached=14336 thinking=True effort=high tokens=160 sha=9599ff0fa9b1 finish=length rounds=62 accepted=97/175 ttft=2.516 decode=7.827 elapsed=10.343 queued=0.000 tok/s=20.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 1023 tokens, resumed at 0, kept 0; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 107, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-9fb2c49e09f61a20f48d2bbe5d0c4d2d prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.712 decode=0.052 elapsed=0.764 queued=0.000 tok/s=19.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 108, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-2b0fdc9cb9bb59c0be166fea97367ef2 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.238 decode=0.053 elapsed=1.290 queued=0.000 tok/s=19.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 109, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-30a94529775f536b751db155b1ea5683 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.236 decode=0.052 elapsed=1.288 queued=0.000 tok/s=19.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 110, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-387733e69460beb2c60f51e03615d241 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.234 decode=0.053 elapsed=1.288 queued=0.000 tok/s=18.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 8191 tokens, resumed at 0, kept 1; 23 states, 7798 MiB and 8586 MiB spare of 16384 MiB (hits 38, misses 111, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-90fcc0382b3473269d92a756a92233fa prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=5.377 decode=0.052 elapsed=5.429 queued=0.000 tok/s=19.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 8192 tokens, resumed at 0, kept 1; 24 states, 8006 MiB and 8378 MiB spare of 16384 MiB (hits 38, misses 112, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-00a775aca26e26cbbedbaf82d587b72e prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=5.374 decode=0.053 elapsed=5.427 queued=0.000 tok/s=18.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 8191 tokens, resumed at 0, kept 1; 25 states, 8214 MiB and 8170 MiB spare of 16384 MiB (hits 38, misses 113, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-7ebdd8c6590ca95a186171e93641adcf prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=5.375 decode=0.054 elapsed=5.429 queued=0.000 tok/s=18.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 16384 tokens, resumed at 0, kept 1; 26 states, 8524 MiB and 7860 MiB spare of 16384 MiB (hits 38, misses 114, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-85f022ad285476a2af95009d6825f229 prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=10.637 decode=0.054 elapsed=10.690 queued=0.000 tok/s=18.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 16384 tokens, resumed at 0, kept 1; 27 states, 8834 MiB and 7550 MiB spare of 16384 MiB (hits 38, misses 115, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-2d045dd1bef19ec998cbcf07db0e2f3e prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=10.632 decode=0.054 elapsed=10.686 queued=0.000 tok/s=18.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 16383 tokens, resumed at 0, kept 1; 28 states, 9144 MiB and 7240 MiB spare of 16384 MiB (hits 38, misses 116, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-6f765b2bcd4ce60b0747417cb2ca69a3 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=10.863 decode=0.055 elapsed=10.918 queued=0.000 tok/s=18.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32768 tokens, resumed at 0, kept 1; 29 states, 9658 MiB and 6726 MiB spare of 16384 MiB (hits 38, misses 117, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-7105a1c567a15cc3b4a19f676d189111 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=21.434 decode=0.054 elapsed=21.487 queued=0.000 tok/s=18.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32766 tokens, resumed at 0, kept 1; 30 states, 10172 MiB and 6212 MiB spare of 16384 MiB (hits 38, misses 118, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-66a56aaa66c694e8327a13bb4c86f5d6 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=21.351 decode=0.054 elapsed=21.405 queued=0.000 tok/s=18.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32766 tokens, resumed at 0, kept 1; 31 states, 10686 MiB and 5698 MiB spare of 16384 MiB (hits 38, misses 119, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-c73379d87fd327f018546a711dcc4c89 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=21.300 decode=0.055 elapsed=21.355 queued=0.000 tok/s=18.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 32 states, 11608 MiB and 4776 MiB spare of 16384 MiB (hits 38, misses 120, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-7ed3eebc9ccefa7fbe73d9c0c9425f70 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=e52c3fd88437 finish=length rounds=1 accepted=0/1 ttft=43.266 decode=0.055 elapsed=43.321 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 33 states, 12530 MiB and 3854 MiB spare of 16384 MiB (hits 38, misses 121, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-73b97c4723d1102fffeb53d6f2eefe9b prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=43.166 decode=0.053 elapsed=43.219 queued=0.000 tok/s=18.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 122, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-f985922d38260f6308b447da2ef12c62 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=43.213 decode=0.053 elapsed=43.265 queued=0.000 tok/s=19.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 123, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-63597e44a157816c1b4508c0aa59d1aa prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=7d62171119f8 finish=length rounds=61 accepted=66/101 ttft=1.236 decode=3.530 elapsed=4.766 queued=0.000 tok/s=36.0 seed=1000 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 124, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-a5dbfe160aa2cb3b3e7d3a3124883342 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=c948b4bcbd58 finish=length rounds=70 accepted=57/88 ttft=1.244 decode=3.846 elapsed=5.090 queued=0.000 tok/s=33.0 seed=1001 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 125, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-89522ce839e4e0e4a90a6a9203239394 prompt=2047 cached=0 thinking=False effort=none tokens=128 sha=6a2fb209181c finish=length rounds=71 accepted=56/87 ttft=1.235 decode=3.865 elapsed=5.100 queued=0.000 tok/s=32.9 seed=1002 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 126, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-5eda6781114588d7bfe8abf2332307eb prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=f8aaeecc59d0 finish=length rounds=58 accepted=69/103 ttft=1.247 decode=3.398 elapsed=4.645 queued=0.000 tok/s=37.4 seed=1003 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 127, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-01b474e3aa4ac3f5e07a62de61baf51e prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=854ffaa5d759 finish=length rounds=76 accepted=51/94 ttft=1.248 decode=4.141 elapsed=5.389 queued=0.000 tok/s=30.7 seed=1004 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 128, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-9a4970b097d732baef446ce4657a70ae prompt=2047 cached=0 thinking=False effort=none tokens=128 sha=0effe3e2e261 finish=length rounds=52 accepted=75/106 ttft=1.240 decode=3.152 elapsed=4.392 queued=0.000 tok/s=40.3 seed=1005 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 129, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-fa72c66740af2371868f47f71616ba3d prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=2d30fcc66f99 finish=length rounds=49 accepted=78/99 ttft=1.247 decode=2.986 elapsed=4.233 queued=0.000 tok/s=42.5 seed=1006 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 63488, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 129, evicted 0, refused 0, failed 0) +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-5079c8c1b828834d085f3ee3816d9ae8 prompt=65536 cached=63488 thinking=False effort=none tokens=128 sha=37aa1a600539 finish=length rounds=50 accepted=77/140 ttft=1.436 decode=17.995 elapsed=19.431 queued=0.000 tok/s=7.1 seed=999 +[tensorfold] done chatcmpl-d8b325b524e3c315150aaa61d0d65e79 prompt=2047 cached=0 thinking=False effort=none tokens=128 sha=0effe3e2e261 finish=length rounds=44 accepted=83/131 ttft=2.562 decode=15.730 elapsed=18.292 queued=0.014 tok/s=8.1 seed=1005 +[tensorfold] done chatcmpl-1abe426a534fda37ea83d6dea028725c prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=2d30fcc66f99 finish=length rounds=44 accepted=83/130 ttft=8.126 decode=10.925 elapsed=19.051 queued=0.014 tok/s=11.6 seed=1006 +[tensorfold] done chatcmpl-c267a6a43435ca85cf45299b931c7acc prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=7d62171119f8 finish=length rounds=48 accepted=80/141 ttft=5.308 decode=14.058 elapsed=19.366 queued=0.016 tok/s=9.0 seed=1000 +[tensorfold] done chatcmpl-ddb63bd3012a2ea1296b8e7d61596dc6 prompt=2047 cached=0 thinking=False effort=none tokens=128 sha=6a2fb209181c finish=length rounds=52 accepted=75/152 ttft=1.253 decode=18.240 elapsed=19.493 queued=0.016 tok/s=7.0 seed=1002 +[tensorfold] done chatcmpl-bfa7c3a4ddee5e32b0a6aaf133694db8 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=f8aaeecc59d0 finish=length rounds=46 accepted=81/135 ttft=9.567 decode=9.925 elapsed=19.492 queued=0.014 tok/s=12.8 seed=1003 +[tensorfold] done chatcmpl-ae70aed079a0efacdb638b5c5338fb8f prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=c948b4bcbd58 finish=length rounds=56 accepted=71/167 ttft=3.934 decode=16.148 elapsed=20.082 queued=0.016 tok/s=7.9 seed=1001 +info: kv pool idle: promised 0, free 1284 + store 0 == total 1284 +[tensorfold] done chatcmpl-b0fa87778f85a65267c871bf31b39004 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=854ffaa5d759 finish=length rounds=66 accepted=61/175 ttft=6.704 decode=14.027 elapsed=20.731 queued=0.014 tok/s=9.1 seed=1004 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +CCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1777 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1778 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6804 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6805 pkey=0 pd=0xef989c053b70 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6806 pkey=0 pd=0xef989c053b70 +host-a:112:112 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/server-rank1.log new file mode 100644 index 0000000000..50c2fd029c --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/server-rank1.log @@ -0,0 +1,4891 @@ +[kvp-serve] 2026-10-22T16:54:18+00:00 rank 1 of 3 context 1048576 parallel 8 binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=unset TF_GLM_MM=unset args +info: rank 1: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-b:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:112:112 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:112:112 [0] NCCL INFO cudaDriverVersion 13030 +host-b:112:112 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:112:112 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:112:112 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:112:112 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:112:112 [0] NCCL INFO Count not open libibverbs.so +host-b:112:112 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:112:112 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:112:112 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:112:112 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x6682430 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:112:112 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x66c3b30 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:112:112 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x6705230 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:112:112 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x6746930 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:112:112 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:112:112 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:112:112 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:112:112 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:112:112 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:112:112 [0] NCCL INFO Initialized NET plugin IB +host-b:112:112 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:112:112 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:112:112 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:112:112 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:112:112 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:112:112 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:112:112 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:112:112 [0] NCCL INFO Using network IB +host-b:112:112 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:112:112 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x3a4ada0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xfa53eb0fed62d7d6 - Init START +host-b:112:112 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:112:112 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:112:112 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:112:112 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:112:112 [0] NCCL INFO Rank 1: 6 Net devices +host-b:112:112 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:112:112 [0] NCCL INFO comm 0x3a4ada0 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:112:112 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:112:112 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:112:112 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:112:112 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:112:112 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:112:128 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-b:112:127 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-b:112:127 [0] NCCL INFO proxy listening socket at <32769> +host-b:112:129 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-b:112:112 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:112:112 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:112:112 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:112:112 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:112:112 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:112:112 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:112:112 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:112:112 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a870010000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a870054000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a87009c000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a8700e0000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2020 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2021 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2020 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2020 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2021 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2021 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a87018d000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a8701d1000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a870219000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a87025d000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6568 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6569 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6570 pkey=0 pd=0xe0a87016bd10 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2020 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2021 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2148 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2149 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2148 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2148 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2149 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2149 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6600 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6601 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6602 pkey=0 pd=0xe0a8700dcc00 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2148 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2149 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2022 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2023 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2022 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2022 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2023 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2023 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6571 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6572 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6573 pkey=0 pd=0xe0a87016bd10 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2150 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2151 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2150 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de8e0 ctsFifoLkey=0x1de8e0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2150 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2151 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de8e0 ctsFifoLkey=0x1de8e0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2151 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2022 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2023 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2150 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2151 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6603 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6604 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6605 pkey=0 pd=0xe0a8700dcc00 +host-b:112:112 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:112:112 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:112:112 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:112:112 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:112:112 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a8705ab000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a8705f0000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a870638000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a870680000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2024 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2025 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2026 pkey=0 pd=0xe0a870053b70 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2152 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2153 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2154 pkey=0 pd=0xe0a87000f940 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2027 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2028 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2029 pkey=0 pd=0xe0a870053b70 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2155 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2156 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2157 pkey=0 pd=0xe0a87000f940 +host-b:112:112 [0] NCCL INFO Connected all trees +host-b:112:112 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:112:112 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:112:112 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:112:112 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a8708a8000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a8708e9000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6574 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6575 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6574 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1718d7 ctsFifoLkey=0x1718d7 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6574 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6575 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1718d7 ctsFifoLkey=0x1718d7 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6575 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6606 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6607 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6606 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x18e0df ctsFifoLkey=0x18e0df +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6606 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6607 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x18e0df ctsFifoLkey=0x18e0df +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6607 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a8709e0000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a870a24000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6574 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6575 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6606 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6607 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6576 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6577 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6576 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1826e3 ctsFifoLkey=0x1826e3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6576 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6577 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1826e3 ctsFifoLkey=0x1826e3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6577 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6608 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6609 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6608 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x198281 ctsFifoLkey=0x198281 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6608 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6609 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x198281 ctsFifoLkey=0x198281 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6609 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6576 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6577 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6608 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6609 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:112 [0] NCCL INFO Connected binomial trees +host-b:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:112:112 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:112:112 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:112:112 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:112:112 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-b:112:112 [0] NCCL INFO ncclCommInitRank comm 0x3a4ada0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xfa53eb0fed62d7d6 - Init COMPLETE +host-b:112:112 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 1.55 (kernels 0.25, alloc 0.03, bootstrap 0.96, allgathers 0.01, topo 0.01, graphs 0.00, connections 0.30, rest 0.00) +host-b:112:112 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:112:112 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:112:112 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:112:112 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:112:112 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:112:112 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:112:112 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:112:112 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a870b1a000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a870b5b000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a870ba0000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a870be4000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2030 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2031 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2030 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8481 ctsFifoLkey=0x1b8481 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2030 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2031 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8481 ctsFifoLkey=0x1b8481 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2031 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6578 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6579 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6580 pkey=0 pd=0xe0a87016bd10 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2158 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2159 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2158 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1decea ctsFifoLkey=0x1decea +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2158 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2159 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1decea ctsFifoLkey=0x1decea +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2159 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6610 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6611 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6612 pkey=0 pd=0xe0a8700dcc00 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a870dd6000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2030 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2031 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2158 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2159 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6613 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6614 pkey=0 pd=0xe0a8700dcc00 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6613 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19160b ctsFifoLkey=0x19160b +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6613 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6614 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19160b ctsFifoLkey=0x19160b +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6614 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a870e6d000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0a870eb5000) +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0a870ef9000) +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6613 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6614 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2160 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2161 pkey=0 pd=0xe0a87000f940 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2162 pkey=0 pd=0xe0a87000f940 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6581 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6582 pkey=0 pd=0xe0a87016bd10 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6581 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17d592 ctsFifoLkey=0x17d592 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6581 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6582 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17d592 ctsFifoLkey=0x17d592 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6582 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2032 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2033 pkey=0 pd=0xe0a870053b70 oooRq=0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2034 pkey=0 pd=0xe0a870053b70 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding deinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 1: rank 0 ended; leaving +v 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6581 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:127 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6582 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:112:112 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/server-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/server-rank2.log new file mode 100644 index 0000000000..6bef16d8c4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6auto/server-rank2.log @@ -0,0 +1,6519 @@ +[kvp-serve] 2026-10-22T16:54:17+00:00 rank 2 of 3 context 1048576 parallel 8 binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=unset TF_GLM_MM=unset args +info: rank 2: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:111:111 [0] NCCL INFO cudaDriverVersion 13030 +host-c:111:111 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:111:111 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:111:111 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:111:111 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:111:111 [0] NCCL INFO Count not open libibverbs.so +host-c:111:111 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:111:111 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:111:111 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:111:111 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x44e44430 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:111:111 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x44e85b30 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:111:111 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x44ec7230 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:111:111 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x44f08930 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:111:111 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:111:111 [0] NCCL INFO Initialized NET plugin IB +host-c:111:111 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:111:111 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:111:111 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:111:111 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:111:111 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:111:111 [0] NCCL INFO Using network IB +host-c:111:111 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:111:111 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x4220cda0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xfa53eb0fed62d7d6 - Init START +host-c:111:111 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:111:111 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:111:111 [0] NCCL INFO Rank 2: 6 Net devices +host-c:111:111 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:111:111 [0] NCCL INFO comm 0x4220cda0 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:111:111 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:111:111 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:111:111 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:111:111 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:111:127 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 1 +host-c:111:126 [0] NCCL INFO [Proxy Service] Device 0 CPU core 11 +host-c:111:126 [0] NCCL INFO proxy listening socket at <57237> +host-c:111:128 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-c:111:111 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe95540010000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe95540054000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe9554009c000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2020 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2021 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2020 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbcba ctsFifoLkey=0x1bbcba +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2020 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2021 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbcba ctsFifoLkey=0x1bbcba +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2021 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1764 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1765 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1766 pkey=0 pd=0xe955400f2420 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe955401c9000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2020 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2021 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2148 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2149 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2148 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2148 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2149 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2149 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1892 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1893 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1894 pkey=0 pd=0xe955400dcec0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe955402f5000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe95540339000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe95540381000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe955403c5000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2148 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2149 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2022 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2023 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2022 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcecb ctsFifoLkey=0x1bcecb +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2022 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2023 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcecb ctsFifoLkey=0x1bcecb +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2023 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1767 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1768 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1769 pkey=0 pd=0xe955400f2420 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1895 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1896 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1897 pkey=0 pd=0xe955400dcec0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2022 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2023 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2150 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2151 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2150 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5f4c ctsFifoLkey=0x1d5f4c +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2150 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2151 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5f4c ctsFifoLkey=0x1d5f4c +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2151 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2150 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2151 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:111 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:111:111 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe955405ab000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe955405ec000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe95540634000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1770 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1771 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1770 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1810cd ctsFifoLkey=0x1810cd +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1770 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1771 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1810cd ctsFifoLkey=0x1810cd +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1771 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1898 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1899 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1898 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197a79 ctsFifoLkey=0x197a79 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1898 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1899 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197a79 ctsFifoLkey=0x197a79 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1899 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe9554072e000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2024 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2025 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2026 pkey=0 pd=0xe95540053590 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2152 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2153 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2154 pkey=0 pd=0xe9554000f940 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe95540867000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1770 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1771 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1898 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1899 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1772 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1773 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1772 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1807c5 ctsFifoLkey=0x1807c5 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1772 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1773 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1807c5 ctsFifoLkey=0x1807c5 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1773 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe955408fe000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe95540946000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe9554098a000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2027 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2028 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2029 pkey=0 pd=0xe95540053590 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1900 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1901 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1900 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x198b88 ctsFifoLkey=0x198b88 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1900 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1901 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x198b88 ctsFifoLkey=0x198b88 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1901 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2155 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2156 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2157 pkey=0 pd=0xe9554000f940 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1772 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1773 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1900 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1901 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:111 [0] NCCL INFO Connected all trees +host-c:111:111 [0] NCCL INFO Connected binomial trees +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:111:111 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:111:111 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:111:111 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-c:111:111 [0] NCCL INFO ncclCommInitRank comm 0x4220cda0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xfa53eb0fed62d7d6 - Init COMPLETE +host-c:111:111 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.55 (kernels 0.25, alloc 0.03, bootstrap 0.00, allgathers 0.01, topo 0.01, graphs 0.00, connections 0.24, rest 0.00) +host-c:111:111 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:111:111 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:111:111 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:111:111 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:111:111 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:111:111 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:111:111 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:111:111 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe95540b1d000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe95540b5e000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe95540ba3000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe95540be4000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe95540c2c000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe95540c70000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe95540cb8000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe95540cfc000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2030 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2031 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2030 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bc0be ctsFifoLkey=0x1bc0be +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2030 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2031 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bc0be ctsFifoLkey=0x1bc0be +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2031 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1774 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1775 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1776 pkey=0 pd=0xe955400f2420 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2158 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2159 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2158 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d8382 ctsFifoLkey=0x1d8382 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2158 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2159 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d8382 ctsFifoLkey=0x1d8382 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2159 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1902 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1903 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1904 pkey=0 pd=0xe955400dcec0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2160 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2161 pkey=0 pd=0xe9554000f940 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2162 pkey=0 pd=0xe9554000f940 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2030 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2031 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2158 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2159 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1905 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1906 pkey=0 pd=0xe955400dcec0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1905 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19a6a4 ctsFifoLkey=0x19a6a4 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1905 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1906 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19a6a4 ctsFifoLkey=0x19a6a4 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1906 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1777 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1778 pkey=0 pd=0xe955400f2420 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1777 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x178242 ctsFifoLkey=0x178242 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1777 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1778 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x178242 ctsFifoLkey=0x178242 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1778 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2032 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2033 pkey=0 pd=0xe95540053590 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2034 pkey=0 pd=0xe95540053590 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1905 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1906 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 2: rank 0 ended; leaving +3:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1777 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1778 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:111 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/client.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/client.log new file mode 100644 index 0000000000..4a4984cafc --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/client.log @@ -0,0 +1,45 @@ +server up 2026-10-23 02:41:22 + 0 S1 t1 drafted prompt 7759 cached 0 ttft 5.11s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 1 S1 t1 plain prompt 7759 cached 0 ttft 5.07s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 2 S1 t2 drafted prompt 15550 cached 0 ttft 10.17s sha 539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b + 3 S1 t2 plain prompt 15550 cached 0 ttft 10.13s sha 539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b + 4 S1 t3 drafted prompt 19462 cached 0 ttft 12.75s sha 8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa + 5 S1 t3 plain prompt 19462 cached 0 ttft 12.74s sha 8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa + 6 S1 t4 drafted prompt 21892 cached 0 ttft 14.27s sha dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af + 7 S1 t4 plain prompt 21892 cached 0 ttft 14.24s sha dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af + 8 S1 t5 drafted prompt 24388 cached 0 ttft 15.83s sha 8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa + 9 S1 t5 plain prompt 24388 cached 0 ttft 15.78s sha 8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa + 10 S1 t6 drafted prompt 26895 cached 0 ttft 17.73s sha c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1 + 11 S1 t6 plain prompt 26895 cached 0 ttft 17.69s sha c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1 + 12 S1 t7 drafted prompt 29798 cached 0 ttft 19.50s sha 7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb + 13 S1 t7 plain prompt 29798 cached 0 ttft 19.47s sha 7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb + 14 S1 t8 drafted prompt 35434 cached 0 ttft 23.28s sha 1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e + 15 S1 t8 plain prompt 35434 cached 0 ttft 23.24s sha 1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e + 16 S1 t9 drafted prompt 38152 cached 0 ttft 24.95s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 + 17 S1 t9 plain prompt 38152 cached 0 ttft 24.94s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 + 18 X t1 drafted prompt 2115 cached 0 ttft 1.48s sha 7b1461bf71da95a2b74977aec25ecf7215b9f0fa044c82d2e1f4905ed66b798a + 19 A t1 drafted prompt 7412 cached 0 ttft 4.96s sha d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666 + 20 A t1 plain prompt 7412 cached 0 ttft 4.92s sha d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666 + 21 B t1 drafted prompt 7474 cached 0 ttft 4.99s sha 9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb + 22 B t1 plain prompt 7474 cached 0 ttft 4.94s sha 9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb + 23 A t2 drafted prompt 12632 cached 0 ttft 8.47s sha e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe + 24 A t2 plain prompt 12632 cached 0 ttft 8.45s sha e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe + 25 B t2 drafted prompt 12721 cached 0 ttft 8.52s sha f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2 + 26 B t2 plain prompt 12721 cached 0 ttft 8.48s sha f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2 + 27 A t3 drafted prompt 15383 cached 0 ttft 10.12s sha 7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8 + 28 A t3 plain prompt 15383 cached 0 ttft 10.10s sha 7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8 + 29 B t3 drafted prompt 17330 cached 0 ttft 11.43s sha 8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49 + 30 B t3 plain prompt 17330 cached 0 ttft 11.39s sha 8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49 + 31 E t1 drafted prompt 40901 cached 0 ttft 26.68s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 + 32 E t1 plain prompt 40901 cached 0 ttft 26.62s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 + 33 T t1 drafted prompt 1520 cached 0 ttft 0.93s sha 033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89 + 34 T t1 plain prompt 1520 cached 0 ttft 0.89s sha 033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89 + 35 T t2 drafted prompt 4202 cached 0 ttft 3.05s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 36 T t2 plain prompt 4202 cached 0 ttft 3.03s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 38 C t1 drafted prompt 9702 cached 0 ttft 6.39s sha 88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c + 39 C t1 plain prompt 9702 cached 0 ttft 6.35s sha 88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c + 40 C t2 drafted prompt 15180 cached 0 ttft 10.06s sha 5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8 + 41 C t2 plain prompt 15180 cached 0 ttft 10.02s sha 5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8 + 42 A t4 drafted prompt 18246 cached 0 ttft 11.85s sha 9599ff0fa9b19bbb5cb5218a5b6f538de1146ce12dad36c729686f12f666389e + 43 B t4 drafted prompt 19808 cached 0 ttft 26.47s sha 025c4856c697674513547e9be3c33a84537e8a4fcb74e731ea09263af3e6d80d +=== kvp-client r6off rc 0 diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/compare-auto.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/compare-auto.log new file mode 100644 index 0000000000..87a0a99bec --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/compare-auto.log @@ -0,0 +1,84 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.01 s vs fresh 5.07 s ( 5.0x) + S1 t2 drafted prompt 15550 cached 6144: ttft 6.13 s vs fresh 10.17 s ( 1.7x) + S1 t2 plain prompt 15550 cached 14336: ttft 0.83 s vs fresh 10.13 s ( 12.2x) + S1 t3 drafted prompt 19462 cached 14336: ttft 3.43 s vs fresh 12.75 s ( 3.7x) + S1 t3 plain prompt 19462 cached 18432: ttft 0.74 s vs fresh 12.74 s ( 17.3x) + S1 t4 drafted prompt 21892 cached 18432: ttft 2.31 s vs fresh 14.27 s ( 6.2x) + S1 t4 plain prompt 21892 cached 20480: ttft 0.92 s vs fresh 14.24 s ( 15.4x) + S1 t5 drafted prompt 24388 cached 20480: ttft 2.53 s vs fresh 15.83 s ( 6.2x) + S1 t5 plain prompt 24388 cached 22528: ttft 1.13 s vs fresh 15.78 s ( 14.0x) + S1 t6 drafted prompt 26895 cached 22528: ttft 3.09 s vs fresh 17.73 s ( 5.7x) + S1 t6 plain prompt 26895 cached 26624: ttft 0.39 s vs fresh 17.69 s ( 45.0x) + S1 t7 drafted prompt 29798 cached 26624: ttft 2.21 s vs fresh 19.50 s ( 8.8x) + S1 t7 plain prompt 29798 cached 28672: ttft 0.81 s vs fresh 19.47 s ( 23.9x) + S1 t8 drafted prompt 35434 cached 28672: ttft 4.63 s vs fresh 23.28 s ( 5.0x) + S1 t8 plain prompt 35434 cached 34816: ttft 0.58 s vs fresh 23.24 s ( 40.4x) + S1 t9 drafted prompt 38152 cached 34816: ttft 2.29 s vs fresh 24.95 s ( 10.9x) + S1 t9 plain prompt 38152 cached 36864: ttft 0.89 s vs fresh 24.94 s ( 28.0x) + A t1 drafted prompt 7412 cached 6144: ttft 0.90 s vs fresh 4.96 s ( 5.5x) + A t1 plain prompt 7412 cached 6144: ttft 0.86 s vs fresh 4.92 s ( 5.7x) + B t1 drafted prompt 7474 cached 6144: ttft 0.92 s vs fresh 4.99 s ( 5.4x) + B t1 plain prompt 7474 cached 6144: ttft 0.88 s vs fresh 4.94 s ( 5.6x) + A t2 drafted prompt 12632 cached 6144: ttft 4.39 s vs fresh 8.47 s ( 1.9x) + A t2 plain prompt 12632 cached 12288: ttft 0.43 s vs fresh 8.45 s ( 19.8x) + B t2 drafted prompt 12721 cached 6144: ttft 4.43 s vs fresh 8.52 s ( 1.9x) + B t2 plain prompt 12721 cached 12288: ttft 0.47 s vs fresh 8.48 s ( 18.2x) + A t3 drafted prompt 15383 cached 12288: ttft 2.14 s vs fresh 10.12 s ( 4.7x) + A t3 plain prompt 15383 cached 14336: ttft 0.77 s vs fresh 10.10 s ( 13.2x) + B t3 drafted prompt 17330 cached 12288: ttft 3.39 s vs fresh 11.43 s ( 3.4x) + B t3 plain prompt 17330 cached 16384: ttft 0.73 s vs fresh 11.39 s ( 15.6x) + E t1 drafted prompt 40901 cached 6144: ttft 22.72 s vs fresh 26.68 s ( 1.2x) + E t1 plain prompt 40901 cached 38912: ttft 1.27 s vs fresh 26.62 s ( 21.0x) + T t2 plain prompt 4202 cached 4096: ttft 0.27 s vs fresh 3.03 s ( 11.2x) + C t1 drafted prompt 9702 cached 6144: ttft 2.37 s vs fresh 6.39 s ( 2.7x) + C t1 plain prompt 9702 cached 8192: ttft 0.97 s vs fresh 6.35 s ( 6.5x) + C t2 drafted prompt 15180 cached 8192: ttft 4.67 s vs fresh 10.06 s ( 2.2x) + C t2 plain prompt 15180 cached 14336: ttft 0.68 s vs fresh 10.02 s ( 14.8x) + A t4 drafted prompt 18246 cached 14336: ttft 2.52 s vs fresh 11.85 s ( 4.7x) + B t4 drafted prompt 19808 cached 16384: ttft 4.93 s vs fresh 26.47 s ( 5.4x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/fresh.jsonl b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/fresh.jsonl new file mode 100644 index 0000000000..e021944ffb --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/fresh.jsonl @@ -0,0 +1,43 @@ +{"i": 0, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.106739524, "prefill": 5.101571059, "wall": 6.74185150093399, "finish": "tool_calls"}} +{"i": 1, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.065409336, "prefill": 5.064203014, "wall": 7.797273191972636, "finish": "tool_calls"}} +{"i": 2, "got": {"sha": "539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b", "cached": 0, "prompt": 15550, "tokens": 46, "ttft": 10.169653888, "prefill": 10.165197102, "wall": 11.017230866942555, "finish": "tool_calls"}} +{"i": 3, "got": {"sha": "539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b", "cached": 0, "prompt": 15550, "tokens": 46, "ttft": 10.133386326, "prefill": 10.131338423, "wall": 12.197452189982869, "finish": "tool_calls"}} +{"i": 4, "got": {"sha": "8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa", "cached": 0, "prompt": 19462, "tokens": 96, "ttft": 12.750480328, "prefill": 12.747779948, "wall": 15.052845261991024, "finish": "tool_calls"}} +{"i": 5, "got": {"sha": "8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa", "cached": 0, "prompt": 19462, "tokens": 96, "ttft": 12.739565459, "prefill": 12.736828529, "wall": 17.05206920602359, "finish": "tool_calls"}} +{"i": 6, "got": {"sha": "dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af", "cached": 0, "prompt": 21892, "tokens": 113, "ttft": 14.27129689, "prefill": 14.266641242, "wall": 17.61862372502219, "finish": "tool_calls"}} +{"i": 7, "got": {"sha": "dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af", "cached": 0, "prompt": 21892, "tokens": 113, "ttft": 14.243072289, "prefill": 14.239940556, "wall": 19.345604075933807, "finish": "tool_calls"}} +{"i": 8, "got": {"sha": "8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa", "cached": 0, "prompt": 24388, "tokens": 160, "ttft": 15.830798688, "prefill": 15.825888193, "wall": 20.40785416297149, "finish": "length"}} +{"i": 9, "got": {"sha": "8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa", "cached": 0, "prompt": 24388, "tokens": 160, "ttft": 15.781539902, "prefill": 15.777986059, "wall": 23.02545110799838, "finish": "length"}} +{"i": 10, "got": {"sha": "c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1", "cached": 0, "prompt": 26895, "tokens": 154, "ttft": 17.732951298, "prefill": 17.727435969, "wall": 22.294732098001987, "finish": "tool_calls"}} +{"i": 11, "got": {"sha": "c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1", "cached": 0, "prompt": 26895, "tokens": 154, "ttft": 17.69050103, "prefill": 17.686393809, "wall": 24.629891379969195, "finish": "tool_calls"}} +{"i": 12, "got": {"sha": "7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb", "cached": 0, "prompt": 29798, "tokens": 136, "ttft": 19.49591165, "prefill": 19.489516218, "wall": 23.21737304108683, "finish": "tool_calls"}} +{"i": 13, "got": {"sha": "7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb", "cached": 0, "prompt": 29798, "tokens": 136, "ttft": 19.466591442, "prefill": 19.461495039, "wall": 25.6158687309362, "finish": "tool_calls"}} +{"i": 14, "got": {"sha": "1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e", "cached": 0, "prompt": 35434, "tokens": 160, "ttft": 23.280516681, "prefill": 23.273209291, "wall": 27.088730115094222, "finish": "length"}} +{"i": 15, "got": {"sha": "1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e", "cached": 0, "prompt": 35434, "tokens": 160, "ttft": 23.242954623, "prefill": 23.237391003, "wall": 30.474726685904898, "finish": "length"}} +{"i": 16, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 0, "prompt": 38152, "tokens": 160, "ttft": 24.952070603, "prefill": 24.944173306, "wall": 27.326827075099573, "finish": "length"}} +{"i": 17, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 0, "prompt": 38152, "tokens": 160, "ttft": 24.941989731, "prefill": 24.936131116, "wall": 32.19846980494913, "finish": "length"}} +{"i": 18, "got": {"sha": "7b1461bf71da95a2b74977aec25ecf7215b9f0fa044c82d2e1f4905ed66b798a", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.477474181, "prefill": 1.475340978, "wall": 5.762198486016132, "finish": "length"}} +{"i": 19, "got": {"sha": "d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666", "cached": 0, "prompt": 7412, "tokens": 90, "ttft": 4.956722079, "prefill": 4.954822512, "wall": 7.1268069670768455, "finish": "tool_calls"}} +{"i": 20, "got": {"sha": "d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666", "cached": 0, "prompt": 7412, "tokens": 90, "ttft": 4.921061842, "prefill": 4.919596648, "wall": 8.964283806970343, "finish": "tool_calls"}} +{"i": 21, "got": {"sha": "9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb", "cached": 0, "prompt": 7474, "tokens": 73, "ttft": 4.992051868, "prefill": 4.989791009, "wall": 6.567046709940769, "finish": "tool_calls"}} +{"i": 22, "got": {"sha": "9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb", "cached": 0, "prompt": 7474, "tokens": 73, "ttft": 4.943812107, "prefill": 4.941956305, "wall": 8.224135098047554, "finish": "tool_calls"}} +{"i": 23, "got": {"sha": "e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe", "cached": 0, "prompt": 12632, "tokens": 160, "ttft": 8.472872705, "prefill": 8.469408033, "wall": 12.740358848008327, "finish": "length"}} +{"i": 24, "got": {"sha": "e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe", "cached": 0, "prompt": 12632, "tokens": 160, "ttft": 8.45310273, "prefill": 8.451130074, "wall": 15.634679945069365, "finish": "length"}} +{"i": 25, "got": {"sha": "f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2", "cached": 0, "prompt": 12721, "tokens": 117, "ttft": 8.515117838, "prefill": 8.511524197, "wall": 11.246910391957499, "finish": "tool_calls"}} +{"i": 26, "got": {"sha": "f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2", "cached": 0, "prompt": 12721, "tokens": 117, "ttft": 8.482100857, "prefill": 8.480382866, "wall": 13.728000696981326, "finish": "tool_calls"}} +{"i": 27, "got": {"sha": "7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8", "cached": 0, "prompt": 15383, "tokens": 137, "ttft": 10.124405283, "prefill": 10.120537625, "wall": 13.983000395004638, "finish": "tool_calls"}} +{"i": 28, "got": {"sha": "7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8", "cached": 0, "prompt": 15383, "tokens": 137, "ttft": 10.098925346, "prefill": 10.09631895, "wall": 16.26234921091236, "finish": "tool_calls"}} +{"i": 29, "got": {"sha": "8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49", "cached": 0, "prompt": 17330, "tokens": 160, "ttft": 11.426775331, "prefill": 11.422184393, "wall": 15.4629950519884, "finish": "length"}} +{"i": 30, "got": {"sha": "8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49", "cached": 0, "prompt": 17330, "tokens": 160, "ttft": 11.389627462, "prefill": 11.386779181, "wall": 18.59346097393427, "finish": "length"}} +{"i": 31, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 0, "prompt": 40901, "tokens": 160, "ttft": 26.684456625, "prefill": 26.675477202, "wall": 28.522890391061082, "finish": "length"}} +{"i": 32, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 0, "prompt": 40901, "tokens": 160, "ttft": 26.622384457, "prefill": 26.615406235, "wall": 33.90392732305918, "finish": "length"}} +{"i": 33, "got": {"sha": "033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89", "cached": 0, "prompt": 1520, "tokens": 150, "ttft": 0.929199678, "prefill": 0.927359526, "wall": 4.6956468189600855, "finish": "stop"}} +{"i": 34, "got": {"sha": "033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89", "cached": 0, "prompt": 1520, "tokens": 150, "ttft": 0.886366686, "prefill": 0.885652282, "wall": 7.481467461911961, "finish": "stop"}} +{"i": 35, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4202, "tokens": 12, "ttft": 3.047861305, "prefill": 3.045043045, "wall": 3.2976807670202106, "finish": "stop"}} +{"i": 36, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4202, "tokens": 12, "ttft": 3.030391404, "prefill": 3.028864656, "wall": 3.5406672669341788, "finish": "stop"}} +{"i": 38, "got": {"sha": "88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c", "cached": 0, "prompt": 9702, "tokens": 110, "ttft": 6.391096316, "prefill": 6.384822747, "wall": 9.140202198992483, "finish": "tool_calls"}} +{"i": 39, "got": {"sha": "88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c", "cached": 0, "prompt": 9702, "tokens": 110, "ttft": 6.351386014, "prefill": 6.349693536, "wall": 11.301413287059404, "finish": "tool_calls"}} +{"i": 40, "got": {"sha": "5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8", "cached": 0, "prompt": 15180, "tokens": 160, "ttft": 10.056047644, "prefill": 10.051997075, "wall": 14.335445799049921, "finish": "length"}} +{"i": 41, "got": {"sha": "5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8", "cached": 0, "prompt": 15180, "tokens": 160, "ttft": 10.022499964, "prefill": 10.020342273, "wall": 17.222023217007518, "finish": "length"}} +{"i": 42, "got": {"sha": "9599ff0fa9b19bbb5cb5218a5b6f538de1146ce12dad36c729686f12f666389e", "cached": 0, "prompt": 18246, "tokens": 160, "ttft": 11.846598759, "prefill": 11.841996732, "wall": 31.68811116402503, "finish": "length"}} +{"i": 43, "got": {"sha": "025c4856c697674513547e9be3c33a84537e8a4fcb74e731ea09263af3e6d80d", "cached": 0, "prompt": 19808, "tokens": 141, "ttft": 26.472744421, "prefill": 25.744492423, "wall": 31.285811885958537, "finish": "tool_calls"}} diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/hostwatch-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/hostwatch-rank0.log new file mode 100644 index 0000000000..9d38bf590a --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/hostwatch-rank0.log @@ -0,0 +1,451 @@ +02:37:22 114 +02:37:24 112 +02:37:26 112 +02:37:28 112 +02:37:30 110 +02:37:32 110 +02:37:34 110 +02:37:36 108 +02:37:39 108 +02:37:41 107 +02:37:43 107 +02:37:45 107 +02:37:47 106 +02:37:49 106 +02:37:51 105 +02:37:53 104 +02:37:55 104 +02:37:57 103 +02:37:59 103 +02:38:01 102 +02:38:03 101 +02:38:05 102 +02:38:07 100 +02:38:09 100 +02:38:11 99 +02:38:13 98 +02:38:15 98 +02:38:18 97 +02:38:20 97 +02:38:22 97 +02:38:24 95 +02:38:26 95 +02:38:28 95 +02:38:30 93 +02:38:32 93 +02:38:34 92 +02:38:36 92 +02:38:38 92 +02:38:40 91 +02:38:42 91 +02:38:44 90 +02:38:46 90 +02:38:48 90 +02:38:50 88 +02:38:52 88 +02:38:55 88 +02:38:57 87 +02:38:59 87 +02:39:01 85 +02:39:03 85 +02:39:05 85 +02:39:07 84 +02:39:09 84 +02:39:11 82 +02:39:13 82 +02:39:15 83 +02:39:17 81 +02:39:19 81 +02:39:21 80 +02:39:23 79 +02:39:25 79 +02:39:27 78 +02:39:29 78 +02:39:31 77 +02:39:34 76 +02:39:36 77 +02:39:38 76 +02:39:40 75 +02:39:42 75 +02:39:44 74 +02:39:46 74 +02:39:48 73 +02:39:50 72 +02:39:52 72 +02:39:54 72 +02:39:56 71 +02:39:58 71 +02:40:00 70 +02:40:02 70 +02:40:04 69 +02:40:06 68 +02:40:08 68 +02:40:10 67 +02:40:12 67 +02:40:15 67 +02:40:17 65 +02:40:19 66 +02:40:21 65 +02:40:23 64 +02:40:25 64 +02:40:27 64 +02:40:29 62 +02:40:31 62 +02:40:33 62 +02:40:35 61 +02:40:37 61 +02:40:39 59 +02:40:41 59 +02:40:43 59 +02:40:45 58 +02:40:47 58 +02:40:49 58 +02:40:52 56 +02:40:54 56 +02:40:56 54 +02:40:58 54 +02:41:00 54 +02:41:02 53 +02:41:04 54 +02:41:06 53 +02:41:08 50 +02:41:10 50 +02:41:12 50 +02:41:14 50 +02:41:16 50 +02:41:18 50 +02:41:20 36 +02:41:22 36 +02:41:24 36 +02:41:26 37 +02:41:29 37 +02:41:31 37 +02:41:33 37 +02:41:35 37 +02:41:37 37 +02:41:39 37 +02:41:41 37 +02:41:43 37 +02:41:45 37 +02:41:47 37 +02:41:49 37 +02:41:51 37 +02:41:53 37 +02:41:55 37 +02:41:57 37 +02:41:59 37 +02:42:02 37 +02:42:04 37 +02:42:06 37 +02:42:08 37 +02:42:10 37 +02:42:12 37 +02:42:14 37 +02:42:16 37 +02:42:18 37 +02:42:20 37 +02:42:22 37 +02:42:24 37 +02:42:26 37 +02:42:28 37 +02:42:30 37 +02:42:32 37 +02:42:35 37 +02:42:37 37 +02:42:39 37 +02:42:41 37 +02:42:43 37 +02:42:45 37 +02:42:47 37 +02:42:49 37 +02:42:51 37 +02:42:53 37 +02:42:55 37 +02:42:57 37 +02:42:59 37 +02:43:01 37 +02:43:03 37 +02:43:05 37 +02:43:07 37 +02:43:10 37 +02:43:12 37 +02:43:14 37 +02:43:16 37 +02:43:18 37 +02:43:20 37 +02:43:22 37 +02:43:24 37 +02:43:26 37 +02:43:28 37 +02:43:30 37 +02:43:32 37 +02:43:34 37 +02:43:36 37 +02:43:38 37 +02:43:40 37 +02:43:43 37 +02:43:45 37 +02:43:47 37 +02:43:49 37 +02:43:51 37 +02:43:53 37 +02:43:55 37 +02:43:57 37 +02:43:59 37 +02:44:01 37 +02:44:03 37 +02:44:05 37 +02:44:07 37 +02:44:09 37 +02:44:11 37 +02:44:14 37 +02:44:16 37 +02:44:18 37 +02:44:20 37 +02:44:22 37 +02:44:24 37 +02:44:26 37 +02:44:28 37 +02:44:30 37 +02:44:32 37 +02:44:34 37 +02:44:36 37 +02:44:38 37 +02:44:40 37 +02:44:42 37 +02:44:44 37 +02:44:46 37 +02:44:49 37 +02:44:51 37 +02:44:53 37 +02:44:55 37 +02:44:57 37 +02:44:59 37 +02:45:01 37 +02:45:03 37 +02:45:05 37 +02:45:07 37 +02:45:09 37 +02:45:11 37 +02:45:13 37 +02:45:15 37 +02:45:17 37 +02:45:19 37 +02:45:22 37 +02:45:24 37 +02:45:26 37 +02:45:28 37 +02:45:30 37 +02:45:32 37 +02:45:34 37 +02:45:36 37 +02:45:38 37 +02:45:40 37 +02:45:42 37 +02:45:44 37 +02:45:46 37 +02:45:48 37 +02:45:50 37 +02:45:52 37 +02:45:55 37 +02:45:57 37 +02:45:59 37 +02:46:01 37 +02:46:03 37 +02:46:05 37 +02:46:07 37 +02:46:09 37 +02:46:11 37 +02:46:13 37 +02:46:15 37 +02:46:17 37 +02:46:19 37 +02:46:21 37 +02:46:23 37 +02:46:25 37 +02:46:28 37 +02:46:30 37 +02:46:32 37 +02:46:34 37 +02:46:36 37 +02:46:38 37 +02:46:40 37 +02:46:42 37 +02:46:44 37 +02:46:46 37 +02:46:48 37 +02:46:50 37 +02:46:52 37 +02:46:54 37 +02:46:56 37 +02:46:58 37 +02:47:01 37 +02:47:03 37 +02:47:05 37 +02:47:07 37 +02:47:09 37 +02:47:11 37 +02:47:13 37 +02:47:15 37 +02:47:17 37 +02:47:19 37 +02:47:21 37 +02:47:23 37 +02:47:25 37 +02:47:27 37 +02:47:29 37 +02:47:31 37 +02:47:34 37 +02:47:36 37 +02:47:38 37 +02:47:40 37 +02:47:42 37 +02:47:44 37 +02:47:46 37 +02:47:48 37 +02:47:50 37 +02:47:52 37 +02:47:54 37 +02:47:56 37 +02:47:58 37 +02:48:00 37 +02:48:02 37 +02:48:04 37 +02:48:06 37 +02:48:09 37 +02:48:11 37 +02:48:13 37 +02:48:15 37 +02:48:17 37 +02:48:19 37 +02:48:21 37 +02:48:23 37 +02:48:25 37 +02:48:27 37 +02:48:29 37 +02:48:31 37 +02:48:33 37 +02:48:35 37 +02:48:37 37 +02:48:40 37 +02:48:42 37 +02:48:44 37 +02:48:46 37 +02:48:48 37 +02:48:50 37 +02:48:52 37 +02:48:54 37 +02:48:56 37 +02:48:58 37 +02:49:00 37 +02:49:02 37 +02:49:04 37 +02:49:06 37 +02:49:08 37 +02:49:10 37 +02:49:13 37 +02:49:15 37 +02:49:17 37 +02:49:19 37 +02:49:21 37 +02:49:23 37 +02:49:25 37 +02:49:27 37 +02:49:29 37 +02:49:31 37 +02:49:33 37 +02:49:35 37 +02:49:37 37 +02:49:39 37 +02:49:41 37 +02:49:43 37 +02:49:46 37 +02:49:48 37 +02:49:50 37 +02:49:52 37 +02:49:54 37 +02:49:56 37 +02:49:58 37 +02:50:00 37 +02:50:02 37 +02:50:04 37 +02:50:06 37 +02:50:08 37 +02:50:10 37 +02:50:12 37 +02:50:14 37 +02:50:16 37 +02:50:18 37 +02:50:21 37 +02:50:23 37 +02:50:25 37 +02:50:27 37 +02:50:29 37 +02:50:31 37 +02:50:33 37 +02:50:35 37 +02:50:37 37 +02:50:39 37 +02:50:41 37 +02:50:43 37 +02:50:45 37 +02:50:47 37 +02:50:49 37 +02:50:51 37 +02:50:54 37 +02:50:56 37 +02:50:58 37 +02:51:00 37 +02:51:02 37 +02:51:04 37 +02:51:06 37 +02:51:08 37 +02:51:10 37 +02:51:12 37 +02:51:14 37 +02:51:16 37 +02:51:18 37 +02:51:20 37 +02:51:22 37 +02:51:25 37 +02:51:27 37 +02:51:29 37 +02:51:31 37 +02:51:33 37 +02:51:35 37 +02:51:37 37 +02:51:39 37 +02:51:41 37 +02:51:43 37 +02:51:45 37 +02:51:47 37 +02:51:49 37 +02:51:51 37 +02:51:53 37 +02:51:56 37 +02:51:58 37 +02:52:00 37 +02:52:02 37 +02:52:04 37 +02:52:06 37 +02:52:08 37 +02:52:10 37 +02:52:12 37 +02:52:14 37 +02:52:16 37 +02:52:18 37 +02:52:20 37 +02:52:22 37 +02:52:24 37 +02:52:26 24 +02:52:29 24 +02:52:31 24 +02:52:33 24 +02:52:35 24 +02:52:37 24 +02:52:39 24 +02:52:41 24 +02:52:43 24 +02:52:45 37 +02:52:47 117 +02:52:49 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/hostwatch-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/hostwatch-rank1.log new file mode 100644 index 0000000000..16d178ffc1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/hostwatch-rank1.log @@ -0,0 +1,449 @@ +02:37:23 114 +02:37:25 112 +02:37:27 112 +02:37:29 112 +02:37:31 112 +02:37:33 110 +02:37:35 110 +02:37:37 109 +02:37:39 109 +02:37:41 109 +02:37:43 107 +02:37:45 107 +02:37:48 107 +02:37:50 105 +02:37:52 105 +02:37:54 104 +02:37:56 104 +02:37:58 102 +02:38:00 102 +02:38:02 102 +02:38:04 100 +02:38:06 100 +02:38:08 99 +02:38:10 99 +02:38:12 98 +02:38:14 98 +02:38:16 98 +02:38:18 97 +02:38:20 97 +02:38:22 97 +02:38:25 95 +02:38:27 95 +02:38:29 95 +02:38:31 94 +02:38:33 94 +02:38:35 93 +02:38:37 93 +02:38:39 92 +02:38:41 91 +02:38:43 91 +02:38:45 90 +02:38:47 89 +02:38:49 89 +02:38:51 88 +02:38:53 88 +02:38:55 87 +02:38:57 87 +02:38:59 85 +02:39:02 85 +02:39:04 85 +02:39:06 85 +02:39:08 84 +02:39:10 84 +02:39:12 83 +02:39:14 83 +02:39:16 82 +02:39:18 81 +02:39:20 80 +02:39:22 80 +02:39:24 80 +02:39:26 79 +02:39:28 78 +02:39:30 77 +02:39:32 77 +02:39:34 77 +02:39:36 75 +02:39:38 75 +02:39:41 74 +02:39:43 74 +02:39:45 74 +02:39:47 74 +02:39:49 72 +02:39:51 72 +02:39:53 72 +02:39:55 71 +02:39:57 71 +02:39:59 70 +02:40:01 70 +02:40:03 68 +02:40:05 68 +02:40:07 68 +02:40:09 67 +02:40:11 67 +02:40:13 67 +02:40:15 65 +02:40:18 65 +02:40:20 64 +02:40:22 64 +02:40:24 63 +02:40:26 63 +02:40:28 63 +02:40:30 61 +02:40:32 61 +02:40:34 61 +02:40:36 60 +02:40:38 60 +02:40:40 60 +02:40:42 59 +02:40:44 59 +02:40:46 58 +02:40:48 57 +02:40:50 57 +02:40:52 55 +02:40:54 55 +02:40:57 55 +02:40:59 54 +02:41:01 54 +02:41:03 54 +02:41:05 50 +02:41:07 50 +02:41:09 50 +02:41:11 50 +02:41:13 50 +02:41:15 49 +02:41:17 37 +02:41:19 37 +02:41:21 37 +02:41:23 37 +02:41:25 37 +02:41:27 37 +02:41:29 37 +02:41:31 37 +02:41:34 37 +02:41:36 37 +02:41:38 37 +02:41:40 37 +02:41:42 37 +02:41:44 37 +02:41:46 37 +02:41:48 37 +02:41:50 37 +02:41:52 37 +02:41:54 37 +02:41:56 37 +02:41:58 37 +02:42:00 37 +02:42:02 37 +02:42:04 37 +02:42:06 37 +02:42:09 37 +02:42:11 37 +02:42:13 37 +02:42:15 37 +02:42:17 37 +02:42:19 37 +02:42:21 37 +02:42:23 37 +02:42:25 37 +02:42:27 37 +02:42:29 37 +02:42:31 37 +02:42:33 37 +02:42:35 37 +02:42:37 37 +02:42:39 37 +02:42:41 37 +02:42:44 37 +02:42:46 37 +02:42:48 37 +02:42:50 37 +02:42:52 37 +02:42:54 37 +02:42:56 37 +02:42:58 37 +02:43:00 37 +02:43:02 37 +02:43:04 37 +02:43:06 37 +02:43:08 37 +02:43:10 37 +02:43:12 37 +02:43:15 37 +02:43:17 37 +02:43:19 37 +02:43:21 37 +02:43:23 37 +02:43:25 37 +02:43:27 37 +02:43:29 37 +02:43:31 37 +02:43:33 37 +02:43:35 37 +02:43:37 37 +02:43:39 37 +02:43:41 37 +02:43:43 37 +02:43:45 37 +02:43:47 37 +02:43:50 37 +02:43:52 37 +02:43:54 37 +02:43:56 37 +02:43:58 37 +02:44:00 37 +02:44:02 37 +02:44:04 37 +02:44:06 37 +02:44:08 37 +02:44:10 37 +02:44:12 37 +02:44:14 37 +02:44:16 37 +02:44:18 37 +02:44:21 37 +02:44:23 37 +02:44:25 37 +02:44:27 37 +02:44:29 37 +02:44:31 37 +02:44:33 37 +02:44:35 37 +02:44:37 37 +02:44:39 37 +02:44:41 37 +02:44:43 37 +02:44:45 37 +02:44:47 37 +02:44:49 37 +02:44:51 37 +02:44:53 37 +02:44:56 37 +02:44:58 37 +02:45:00 37 +02:45:02 37 +02:45:04 37 +02:45:06 37 +02:45:08 37 +02:45:10 37 +02:45:12 37 +02:45:14 37 +02:45:16 37 +02:45:18 37 +02:45:20 37 +02:45:22 37 +02:45:24 37 +02:45:26 37 +02:45:29 37 +02:45:31 37 +02:45:33 37 +02:45:35 37 +02:45:37 37 +02:45:39 37 +02:45:41 37 +02:45:43 37 +02:45:45 37 +02:45:47 37 +02:45:49 37 +02:45:51 37 +02:45:53 37 +02:45:55 37 +02:45:57 37 +02:45:59 37 +02:46:01 37 +02:46:04 37 +02:46:06 37 +02:46:08 37 +02:46:10 37 +02:46:12 37 +02:46:14 37 +02:46:16 37 +02:46:18 37 +02:46:20 37 +02:46:22 37 +02:46:24 37 +02:46:26 37 +02:46:28 37 +02:46:30 37 +02:46:32 37 +02:46:34 37 +02:46:37 37 +02:46:39 37 +02:46:41 37 +02:46:43 37 +02:46:45 37 +02:46:47 37 +02:46:49 37 +02:46:51 37 +02:46:53 37 +02:46:55 37 +02:46:57 37 +02:46:59 37 +02:47:01 37 +02:47:03 37 +02:47:05 37 +02:47:07 37 +02:47:10 37 +02:47:12 37 +02:47:14 37 +02:47:16 37 +02:47:18 37 +02:47:20 37 +02:47:22 37 +02:47:24 37 +02:47:26 37 +02:47:28 37 +02:47:30 37 +02:47:32 37 +02:47:34 37 +02:47:36 37 +02:47:38 37 +02:47:41 37 +02:47:43 37 +02:47:45 37 +02:47:47 37 +02:47:49 37 +02:47:51 37 +02:47:53 37 +02:47:55 37 +02:47:57 37 +02:47:59 37 +02:48:01 37 +02:48:03 37 +02:48:05 37 +02:48:07 37 +02:48:09 37 +02:48:11 37 +02:48:14 37 +02:48:16 37 +02:48:18 37 +02:48:20 37 +02:48:22 37 +02:48:24 37 +02:48:26 37 +02:48:28 37 +02:48:30 37 +02:48:32 37 +02:48:34 37 +02:48:36 37 +02:48:38 37 +02:48:40 37 +02:48:42 37 +02:48:44 37 +02:48:47 37 +02:48:49 37 +02:48:51 37 +02:48:53 37 +02:48:55 37 +02:48:57 37 +02:48:59 37 +02:49:01 37 +02:49:03 37 +02:49:05 37 +02:49:07 37 +02:49:09 37 +02:49:11 37 +02:49:13 37 +02:49:15 37 +02:49:17 37 +02:49:20 37 +02:49:22 37 +02:49:24 37 +02:49:26 37 +02:49:28 37 +02:49:30 37 +02:49:32 37 +02:49:34 37 +02:49:36 37 +02:49:38 37 +02:49:40 37 +02:49:42 37 +02:49:44 37 +02:49:46 37 +02:49:48 37 +02:49:50 37 +02:49:53 37 +02:49:55 37 +02:49:57 37 +02:49:59 37 +02:50:01 37 +02:50:03 37 +02:50:05 37 +02:50:07 37 +02:50:09 37 +02:50:11 37 +02:50:13 37 +02:50:15 37 +02:50:17 37 +02:50:19 37 +02:50:21 37 +02:50:23 37 +02:50:26 37 +02:50:28 37 +02:50:30 37 +02:50:32 37 +02:50:34 37 +02:50:36 37 +02:50:38 37 +02:50:40 37 +02:50:42 37 +02:50:44 37 +02:50:46 37 +02:50:48 37 +02:50:50 37 +02:50:52 37 +02:50:54 37 +02:50:56 37 +02:50:59 37 +02:51:01 37 +02:51:03 37 +02:51:05 37 +02:51:07 37 +02:51:09 37 +02:51:11 37 +02:51:13 37 +02:51:15 37 +02:51:17 37 +02:51:19 37 +02:51:21 37 +02:51:23 37 +02:51:25 37 +02:51:27 37 +02:51:29 37 +02:51:32 37 +02:51:34 37 +02:51:36 37 +02:51:38 37 +02:51:40 37 +02:51:42 37 +02:51:44 37 +02:51:46 37 +02:51:48 37 +02:51:50 37 +02:51:52 37 +02:51:54 37 +02:51:56 37 +02:51:58 37 +02:52:00 37 +02:52:02 37 +02:52:05 37 +02:52:07 37 +02:52:09 37 +02:52:11 37 +02:52:13 37 +02:52:15 37 +02:52:17 37 +02:52:19 37 +02:52:21 37 +02:52:23 37 +02:52:25 33 +02:52:27 24 +02:52:29 24 +02:52:31 24 +02:52:33 24 +02:52:36 24 +02:52:38 24 +02:52:40 24 +02:52:42 24 +02:52:44 24 +02:52:46 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/hostwatch-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/hostwatch-rank2.log new file mode 100644 index 0000000000..803783731b --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/hostwatch-rank2.log @@ -0,0 +1,449 @@ +02:37:24 114 +02:37:26 112 +02:37:28 112 +02:37:30 111 +02:37:32 110 +02:37:34 109 +02:37:36 109 +02:37:38 109 +02:37:40 109 +02:37:42 108 +02:37:44 107 +02:37:46 107 +02:37:48 107 +02:37:50 105 +02:37:52 105 +02:37:55 105 +02:37:57 104 +02:37:59 104 +02:38:01 102 +02:38:03 102 +02:38:05 102 +02:38:07 101 +02:38:09 101 +02:38:11 100 +02:38:13 100 +02:38:15 100 +02:38:17 98 +02:38:19 98 +02:38:21 97 +02:38:23 97 +02:38:25 96 +02:38:27 95 +02:38:29 96 +02:38:32 94 +02:38:34 94 +02:38:36 93 +02:38:38 93 +02:38:40 91 +02:38:42 91 +02:38:44 91 +02:38:46 90 +02:38:48 90 +02:38:50 89 +02:38:52 89 +02:38:54 88 +02:38:56 87 +02:38:58 86 +02:39:00 86 +02:39:02 86 +02:39:04 85 +02:39:06 85 +02:39:08 84 +02:39:11 84 +02:39:13 84 +02:39:15 82 +02:39:17 82 +02:39:19 81 +02:39:21 81 +02:39:23 80 +02:39:25 80 +02:39:27 80 +02:39:29 79 +02:39:31 79 +02:39:33 77 +02:39:35 77 +02:39:37 76 +02:39:39 76 +02:39:41 76 +02:39:43 76 +02:39:46 75 +02:39:48 74 +02:39:50 73 +02:39:52 73 +02:39:54 72 +02:39:56 72 +02:39:58 71 +02:40:00 71 +02:40:02 71 +02:40:04 70 +02:40:06 70 +02:40:08 69 +02:40:10 68 +02:40:12 68 +02:40:14 67 +02:40:16 67 +02:40:18 67 +02:40:20 66 +02:40:23 66 +02:40:25 64 +02:40:27 64 +02:40:29 64 +02:40:31 63 +02:40:33 63 +02:40:35 62 +02:40:37 62 +02:40:39 61 +02:40:41 60 +02:40:43 60 +02:40:45 59 +02:40:47 59 +02:40:49 58 +02:40:51 56 +02:40:53 55 +02:40:55 55 +02:40:57 55 +02:41:00 55 +02:41:02 55 +02:41:04 42 +02:41:06 42 +02:41:08 42 +02:41:10 42 +02:41:12 42 +02:41:14 42 +02:41:16 42 +02:41:18 42 +02:41:20 42 +02:41:22 42 +02:41:24 42 +02:41:26 42 +02:41:28 42 +02:41:30 42 +02:41:32 42 +02:41:35 42 +02:41:37 42 +02:41:39 42 +02:41:41 42 +02:41:43 42 +02:41:45 42 +02:41:47 42 +02:41:49 42 +02:41:51 42 +02:41:53 42 +02:41:55 42 +02:41:57 42 +02:41:59 42 +02:42:01 42 +02:42:03 42 +02:42:05 42 +02:42:08 42 +02:42:10 42 +02:42:12 42 +02:42:14 42 +02:42:16 42 +02:42:18 42 +02:42:20 42 +02:42:22 42 +02:42:24 42 +02:42:26 42 +02:42:28 42 +02:42:30 42 +02:42:32 42 +02:42:34 42 +02:42:36 42 +02:42:38 42 +02:42:41 42 +02:42:43 42 +02:42:45 42 +02:42:47 42 +02:42:49 42 +02:42:51 42 +02:42:53 42 +02:42:55 42 +02:42:57 42 +02:42:59 42 +02:43:01 42 +02:43:03 42 +02:43:05 42 +02:43:07 42 +02:43:09 42 +02:43:11 42 +02:43:14 42 +02:43:16 42 +02:43:18 42 +02:43:20 42 +02:43:22 42 +02:43:24 42 +02:43:26 42 +02:43:28 42 +02:43:30 42 +02:43:32 42 +02:43:34 42 +02:43:36 42 +02:43:38 42 +02:43:40 42 +02:43:42 42 +02:43:44 42 +02:43:47 42 +02:43:49 42 +02:43:51 42 +02:43:53 42 +02:43:55 42 +02:43:57 42 +02:43:59 42 +02:44:01 42 +02:44:03 42 +02:44:05 42 +02:44:07 42 +02:44:09 42 +02:44:11 42 +02:44:13 42 +02:44:16 42 +02:44:18 42 +02:44:20 42 +02:44:22 42 +02:44:24 42 +02:44:26 42 +02:44:28 42 +02:44:30 42 +02:44:32 42 +02:44:34 42 +02:44:36 42 +02:44:38 42 +02:44:40 42 +02:44:42 42 +02:44:44 42 +02:44:46 42 +02:44:49 42 +02:44:51 42 +02:44:53 42 +02:44:55 42 +02:44:57 42 +02:44:59 42 +02:45:01 42 +02:45:03 42 +02:45:05 42 +02:45:07 42 +02:45:09 42 +02:45:11 42 +02:45:13 42 +02:45:15 42 +02:45:17 42 +02:45:19 42 +02:45:22 42 +02:45:24 42 +02:45:26 42 +02:45:28 42 +02:45:30 42 +02:45:32 42 +02:45:34 42 +02:45:36 42 +02:45:38 42 +02:45:40 42 +02:45:42 42 +02:45:44 42 +02:45:46 42 +02:45:48 42 +02:45:50 42 +02:45:52 42 +02:45:55 42 +02:45:57 42 +02:45:59 42 +02:46:01 42 +02:46:03 42 +02:46:05 42 +02:46:07 42 +02:46:09 42 +02:46:11 42 +02:46:13 42 +02:46:15 42 +02:46:17 42 +02:46:19 42 +02:46:21 42 +02:46:23 42 +02:46:25 42 +02:46:28 42 +02:46:30 42 +02:46:32 42 +02:46:34 42 +02:46:36 42 +02:46:38 42 +02:46:40 42 +02:46:42 42 +02:46:44 42 +02:46:46 42 +02:46:48 42 +02:46:50 42 +02:46:52 42 +02:46:54 42 +02:46:56 42 +02:46:58 42 +02:47:01 42 +02:47:03 42 +02:47:05 42 +02:47:07 42 +02:47:09 42 +02:47:11 42 +02:47:13 42 +02:47:15 42 +02:47:17 42 +02:47:19 42 +02:47:21 42 +02:47:23 42 +02:47:25 42 +02:47:27 42 +02:47:29 42 +02:47:31 42 +02:47:34 42 +02:47:36 42 +02:47:38 42 +02:47:40 42 +02:47:42 42 +02:47:44 42 +02:47:46 42 +02:47:48 42 +02:47:50 42 +02:47:52 42 +02:47:54 42 +02:47:56 42 +02:47:58 42 +02:48:00 42 +02:48:02 42 +02:48:05 42 +02:48:07 42 +02:48:09 42 +02:48:11 42 +02:48:13 42 +02:48:15 42 +02:48:17 42 +02:48:19 42 +02:48:21 42 +02:48:23 42 +02:48:25 42 +02:48:27 42 +02:48:29 42 +02:48:31 42 +02:48:33 42 +02:48:35 42 +02:48:38 42 +02:48:40 42 +02:48:42 42 +02:48:44 42 +02:48:46 42 +02:48:48 42 +02:48:50 42 +02:48:52 42 +02:48:54 42 +02:48:56 42 +02:48:58 42 +02:49:00 42 +02:49:02 42 +02:49:04 42 +02:49:06 42 +02:49:09 42 +02:49:11 42 +02:49:13 42 +02:49:15 42 +02:49:17 42 +02:49:19 42 +02:49:21 42 +02:49:23 42 +02:49:25 42 +02:49:27 42 +02:49:29 42 +02:49:31 42 +02:49:33 42 +02:49:35 42 +02:49:37 42 +02:49:40 42 +02:49:42 42 +02:49:44 42 +02:49:46 42 +02:49:48 42 +02:49:50 42 +02:49:52 42 +02:49:54 42 +02:49:56 42 +02:49:58 42 +02:50:00 42 +02:50:02 42 +02:50:04 42 +02:50:06 42 +02:50:08 42 +02:50:11 42 +02:50:13 42 +02:50:15 42 +02:50:17 42 +02:50:19 42 +02:50:21 42 +02:50:23 42 +02:50:25 42 +02:50:27 42 +02:50:29 42 +02:50:31 42 +02:50:33 42 +02:50:35 42 +02:50:37 42 +02:50:39 42 +02:50:41 42 +02:50:43 42 +02:50:46 42 +02:50:48 42 +02:50:50 42 +02:50:52 42 +02:50:54 42 +02:50:56 42 +02:50:58 42 +02:51:00 42 +02:51:02 42 +02:51:04 42 +02:51:06 42 +02:51:08 42 +02:51:10 42 +02:51:12 42 +02:51:14 42 +02:51:16 42 +02:51:19 42 +02:51:21 42 +02:51:23 42 +02:51:25 42 +02:51:27 42 +02:51:29 42 +02:51:31 42 +02:51:33 42 +02:51:35 42 +02:51:37 42 +02:51:39 42 +02:51:41 42 +02:51:43 42 +02:51:45 42 +02:51:47 42 +02:51:50 42 +02:51:52 42 +02:51:54 42 +02:51:56 42 +02:51:58 42 +02:52:00 42 +02:52:02 42 +02:52:04 42 +02:52:06 42 +02:52:08 42 +02:52:10 42 +02:52:12 42 +02:52:14 42 +02:52:16 42 +02:52:18 42 +02:52:20 42 +02:52:23 42 +02:52:25 42 +02:52:27 29 +02:52:29 29 +02:52:31 29 +02:52:33 29 +02:52:35 29 +02:52:37 29 +02:52:39 29 +02:52:41 29 +02:52:43 29 +02:52:45 42 +02:52:47 done diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/server-rank0.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/server-rank0.log new file mode 100644 index 0000000000..7f723d4a64 --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/server-rank0.log @@ -0,0 +1,23668 @@ +[kvp-serve] 2026-10-22T17:37:12+00:00 rank 0 of 3 context 1048576 parallel auto binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=off TF_GLM_MM=unset args --prompt-cache-gib 0 +info: rank 0: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-a:7215:7215 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:7215:7215 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:7215:7215 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:7215:7215 [0] NCCL INFO cudaDriverVersion 13030 +host-a:7215:7215 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:7215:7215 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:7215:7215 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:7215:7215 [0] NCCL INFO Count not open libibverbs.so +host-a:7215:7215 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:7215:7215 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:7215:7215 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:7215:7215 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x2ec255c0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:7215:7215 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x2ec66cc0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:7215:7215 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x2eca83c0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:7215:7215 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x2ece9ac0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:7215:7215 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:7215:7215 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:7215:7215 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:7215:7215 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:7215:7215 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:7215:7215 [0] NCCL INFO Initialized NET plugin IB +host-a:7215:7215 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:7215:7215 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:7215:7215 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:7215:7215 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:7215:7215 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:7215:7215 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:7215:7215 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:7215:7215 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:7215:7215 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:7215:7215 [0] NCCL INFO Using network IB +host-a:7215:7215 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:7215:7215 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x2bfedf10 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xd5e2be80812c97fd - Init START +host-a:7215:7215 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:7215:7215 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:7215:7215 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:7215:7215 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:7215:7215 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:7215:7215 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:7215:7215 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:7215:7215 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:7215:7215 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:7215:7215 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:7215:7215 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:7215:7215 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:7215:7215 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:7215:7215 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:7215:7215 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:7215:7215 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:7215:7215 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:7215:7215 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:7215:7215 [0] NCCL INFO Rank 0: 6 Net devices +host-a:7215:7215 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:7215:7215 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:7215:7215 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:7215:7215 [0] NCCL INFO comm 0x2bfedf10 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:7215:7215 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:7215:7215 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:7215:7215 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:7215:7215 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:7215:7215 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:7215:7215 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:7215:7215 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:7215:7215 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:7215:7215 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:7215:7215 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:7215:7233 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-a:7215:7232 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-a:7215:7232 [0] NCCL INFO proxy listening socket at <52483> +host-a:7215:7215 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:7215:7234 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-a:7215:7215 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:7215:7215 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:7215:7215 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:7215:7215 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:7215:7215 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:7215:7215 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:7215:7215 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13fac010000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13fac054000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13fac09c000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6822 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6823 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6822 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b6866 ctsFifoLkey=0x1b6866 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6822 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6823 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b6866 ctsFifoLkey=0x1b6866 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6823 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6854 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6855 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6854 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ccccc ctsFifoLkey=0x1ccccc +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6854 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6855 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ccccc ctsFifoLkey=0x1ccccc +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6855 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13fac1aa000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13fac1f2000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13fac236000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13fac27e000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13fac2c2000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1794 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1795 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1796 pkey=0 pd=0xe13fac123da0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6822 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6823 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6854 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6855 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1922 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1923 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1924 pkey=0 pd=0xe13fac00f7d0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6824 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6825 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6824 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1be6e3 ctsFifoLkey=0x1be6e3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6824 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6825 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1be6e3 ctsFifoLkey=0x1be6e3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6825 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1797 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1798 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1799 pkey=0 pd=0xe13fac123da0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6856 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6857 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6856 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d8e88 ctsFifoLkey=0x1d8e88 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6856 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6857 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d8e88 ctsFifoLkey=0x1d8e88 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6857 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1925 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1926 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1927 pkey=0 pd=0xe13fac00f7d0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6824 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6825 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6856 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6857 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7215 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:7215:7215 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:7215:7215 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:7215:7215 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:7215:7215 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13fac5b6000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13fac5f7000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13fac638000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13fac679000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1800 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1801 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1800 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x177632 ctsFifoLkey=0x177632 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1800 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1801 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x177632 ctsFifoLkey=0x177632 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1801 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1800 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1801 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1928 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1929 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1928 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b0af ctsFifoLkey=0x19b0af +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1928 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1929 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b0af ctsFifoLkey=0x19b0af +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1929 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1928 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1929 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1802 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1803 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1802 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17ce8b ctsFifoLkey=0x17ce8b +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1802 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1803 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17ce8b ctsFifoLkey=0x17ce8b +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1803 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1930 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1931 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1930 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a2a0 ctsFifoLkey=0x19a2a0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1930 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1931 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a2a0 ctsFifoLkey=0x19a2a0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1931 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1802 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1803 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1930 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1931 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7215 [0] NCCL INFO Connected all trees +host-a:7215:7215 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:7215:7215 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:7215:7215 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:7215:7215 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13fac81e000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13fac863000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13fac8a8000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13fac8f0000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6826 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6827 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6828 pkey=0 pd=0xe13fac053590 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6858 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6859 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6860 pkey=0 pd=0xe13fac0f2420 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6829 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6830 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6831 pkey=0 pd=0xe13fac053590 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6861 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6862 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6863 pkey=0 pd=0xe13fac0f2420 +host-a:7215:7215 [0] NCCL INFO Connected binomial trees +host-a:7215:7215 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:7215:7215 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:7215:7215 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:7215:7215 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:7215:7215 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-a:7215:7215 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:7215:7215 [0] NCCL INFO ncclCommInitRank comm 0x2bfedf10 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xd5e2be80812c97fd - Init COMPLETE +host-a:7215:7215 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 5.57 (kernels 0.24, alloc 0.03, bootstrap 5.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.28, rest 0.00) +host-a:7215:7215 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:7215:7215 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:7215:7215 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:7215:7215 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:7215:7215 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:7215:7215 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:7215:7215 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:7215:7215 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13facb16000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13facb57000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13facb9f000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6832 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6833 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6832 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1bb4b2 ctsFifoLkey=0x1bb4b2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6832 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6833 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1bb4b2 ctsFifoLkey=0x1bb4b2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6833 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1804 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1805 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1806 pkey=0 pd=0xe13fac123da0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6864 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6865 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6864 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c2525 ctsFifoLkey=0x1c2525 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6864 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6865 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c2525 ctsFifoLkey=0x1c2525 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6865 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13facd13000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6832 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6833 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13facd5b000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13facd9c000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe13facde1000) +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe13face25000) +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6864 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6865 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1932 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1933 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1934 pkey=0 pd=0xe13fac00f7d0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1935 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1936 pkey=0 pd=0xe13fac00f7d0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1935 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a6a8 ctsFifoLkey=0x19a6a8 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1935 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1936 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a6a8 ctsFifoLkey=0x19a6a8 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1936 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6866 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6867 pkey=0 pd=0xe13fac0f2420 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6868 pkey=0 pd=0xe13fac0f2420 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1807 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1808 pkey=0 pd=0xe13fac123da0 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1807 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1807 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1808 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1808 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6834 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6835 pkey=0 pd=0xe13fac053590 oooRq=0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6836 pkey=0 pd=0xe13fac053590 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1935 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1936 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 78.03 GiB; 2 streams at once, 12.88 GiB each at a 1048576-token window, of 24.7 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 1048576; loaded in 248.5s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] done chatcmpl-31d1d8abcd2f2b6a19e71b8cedd3ed7f prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=29 accepted=32/43 ttft=5.107 decode=1.619 elapsed=6.726 queued=0.000 tok/s=37.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-92cb16e3cd1999f5557618b62cdf45bf prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=5.065 decode=2.730 elapsed=7.796 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-aa27b6381168baa319050e8bad608112 prompt=15550 cached=0 thinking=True effort=high tokens=46 sha=539973cb8b13 finish=stop rounds=13 accepted=33/36 ttft=10.170 decode=0.846 elapsed=11.015 queued=0.000 tok/s=53.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-47b1c650ec504721b55baef3a1df7f5a prompt=15550 cached=0 thinking=True effort=high tokens=46 sha=539973cb8b13 finish=stop rounds=45 accepted=0/0 ttft=10.133 decode=2.062 elapsed=12.196 queued=0.000 tok/s=21.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-067bca5d5c0be9586fa416ff438b88f5 prompt=19462 cached=0 thinking=True effort=high tokens=96 sha=8fc3c26162f1 finish=stop rounds=38 accepted=58/82 ttft=12.750 decode=2.300 elapsed=15.051 queued=0.000 tok/s=41.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-674cffe10f21789dd3e76a7afe6fbadc prompt=19462 cached=0 thinking=True effort=high tokens=96 sha=8fc3c26162f1 finish=stop rounds=95 accepted=0/0 ttft=12.740 decode=4.311 elapsed=17.050 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-283d2774e56d151d9d9968b682094dfa prompt=21892 cached=0 thinking=True effort=high tokens=113 sha=dedd451779f3 finish=stop rounds=61 accepted=52/80 ttft=14.271 decode=3.345 elapsed=17.617 queued=0.000 tok/s=33.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0e70adec524fd70183cd88d87773eeaf prompt=21892 cached=0 thinking=True effort=high tokens=113 sha=dedd451779f3 finish=stop rounds=112 accepted=0/0 ttft=14.243 decode=5.101 elapsed=19.344 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-4bdbbb5ca76cfc31cb159007242dc66e prompt=24388 cached=0 thinking=True effort=high tokens=160 sha=8bcbd355dc04 finish=length rounds=81 accepted=79/121 ttft=15.831 decode=4.575 elapsed=20.406 queued=0.000 tok/s=34.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-7e1d45a504aeff940b717bc122ab3323 prompt=24388 cached=0 thinking=True effort=high tokens=160 sha=8bcbd355dc04 finish=length rounds=159 accepted=0/0 ttft=15.782 decode=7.242 elapsed=23.024 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-541dd0f57fdb11bc551021d391833556 prompt=26895 cached=0 thinking=True effort=high tokens=154 sha=c8e79afc5a10 finish=stop rounds=81 accepted=72/122 ttft=17.733 decode=4.560 elapsed=22.293 queued=0.000 tok/s=33.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7e23e42aac108e9557bd61a0caf9906d prompt=26895 cached=0 thinking=True effort=high tokens=154 sha=c8e79afc5a10 finish=stop rounds=153 accepted=0/0 ttft=17.691 decode=6.937 elapsed=24.628 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fc45ef85901bea43a404a3a59e50768a prompt=29798 cached=0 thinking=True effort=high tokens=136 sha=7629aa6bf5df finish=stop rounds=66 accepted=69/100 ttft=19.496 decode=3.719 elapsed=23.215 queued=0.000 tok/s=36.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2689e220334b518bd4085901d70c7e58 prompt=29798 cached=0 thinking=True effort=high tokens=136 sha=7629aa6bf5df finish=stop rounds=135 accepted=0/0 ttft=19.467 decode=6.147 elapsed=25.614 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-1b1a69e35a9929f43f44cc09025dcd75 prompt=35434 cached=0 thinking=True effort=high tokens=160 sha=1adac255ff32 finish=length rounds=60 accepted=99/148 ttft=23.281 decode=3.806 elapsed=27.087 queued=0.000 tok/s=41.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-1f7f599f8b8db03ab8ea665495919ca7 prompt=35434 cached=0 thinking=True effort=high tokens=160 sha=1adac255ff32 finish=length rounds=159 accepted=0/0 ttft=23.243 decode=7.230 elapsed=30.473 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-c69a57e8dce4ff6eb460299914f65f60 prompt=38152 cached=0 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=34 accepted=125/133 ttft=24.952 decode=2.373 elapsed=27.325 queued=0.000 tok/s=67.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ab622be45b280124cce66276c0ed137e prompt=38152 cached=0 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=159 accepted=0/0 ttft=24.942 decode=7.255 elapsed=32.197 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-62c82e8a04a4bd82c81f032b8be25d08 prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=7b1461bf71da finish=length rounds=75 accepted=84/126 ttft=1.477 decode=4.283 elapsed=5.760 queued=0.000 tok/s=37.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-440b66cb4014cbe2bc865cffca3e2b03 prompt=7412 cached=0 thinking=True effort=high tokens=90 sha=d0573eef0609 finish=stop rounds=35 accepted=55/81 ttft=4.957 decode=2.169 elapsed=7.125 queued=0.000 tok/s=41.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2f0e9e827c8a452a5406b65b4d04e011 prompt=7412 cached=0 thinking=True effort=high tokens=90 sha=d0573eef0609 finish=stop rounds=89 accepted=0/0 ttft=4.921 decode=4.042 elapsed=8.963 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-420f7f37257d41c8d46cf38fce1c98f8 prompt=7474 cached=0 thinking=True effort=high tokens=73 sha=9a2685e9905f finish=stop rounds=24 accepted=49/65 ttft=4.992 decode=1.573 elapsed=6.565 queued=0.000 tok/s=45.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d79d1ef1734b5dfee3b1f88d6bbfe4f0 prompt=7474 cached=0 thinking=True effort=high tokens=73 sha=9a2685e9905f finish=stop rounds=72 accepted=0/0 ttft=4.944 decode=3.279 elapsed=8.223 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-8bf075fff491c6857c0117f0187702c7 prompt=12632 cached=0 thinking=True effort=high tokens=160 sha=e1f2511e60f3 finish=length rounds=75 accepted=84/122 ttft=8.473 decode=4.266 elapsed=12.739 queued=0.000 tok/s=37.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-08854682c1c939e28175a40665ffac4a prompt=12632 cached=0 thinking=True effort=high tokens=160 sha=e1f2511e60f3 finish=length rounds=159 accepted=0/0 ttft=8.453 decode=7.179 elapsed=15.633 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-dbe030a93da73c3821580202b67d2ca7 prompt=12721 cached=0 thinking=True effort=high tokens=117 sha=f202ab6075ae finish=stop rounds=44 accepted=72/103 ttft=8.515 decode=2.730 elapsed=11.245 queued=0.000 tok/s=42.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-bdbef35a95c9f9e870a8a9b7d9e27944 prompt=12721 cached=0 thinking=True effort=high tokens=117 sha=f202ab6075ae finish=stop rounds=116 accepted=0/0 ttft=8.482 decode=5.244 elapsed=13.726 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7f92f1b14725e634ab0cc51588f1a0aa prompt=15383 cached=0 thinking=True effort=high tokens=137 sha=7695627844cd finish=stop rounds=70 accepted=67/97 ttft=10.124 decode=3.856 elapsed=13.981 queued=0.000 tok/s=35.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-30a1e7bdc414a832127f17dcaab66eaa prompt=15383 cached=0 thinking=True effort=high tokens=137 sha=7695627844cd finish=stop rounds=136 accepted=0/0 ttft=10.099 decode=6.162 elapsed=16.261 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-6cc72027a9f2397182b15b078f15e1bf prompt=17330 cached=0 thinking=True effort=high tokens=160 sha=8b15f58457e8 finish=length rounds=69 accepted=90/124 ttft=11.427 decode=4.034 elapsed=15.461 queued=0.000 tok/s=39.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-2f7dc4236df7c4d2c36e1cdf480803aa prompt=17330 cached=0 thinking=True effort=high tokens=160 sha=8b15f58457e8 finish=length rounds=159 accepted=0/0 ttft=11.390 decode=7.202 elapsed=18.592 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-d45f7fef06d8176e70cd6ed664dd37ac prompt=40901 cached=0 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=22 accepted=137/137 ttft=26.684 decode=1.836 elapsed=28.520 queued=0.000 tok/s=86.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-4429af81ba5ede6c8892bce59e97b6bb prompt=40901 cached=0 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=159 accepted=0/0 ttft=26.622 decode=7.280 elapsed=33.902 queued=0.000 tok/s=21.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f79712b9af33865abc091b41c55495c6 prompt=1520 cached=0 thinking=False effort=none tokens=150 sha=033a197da586 finish=stop rounds=64 accepted=85/131 ttft=0.929 decode=3.765 elapsed=4.694 queued=0.000 tok/s=39.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1e0890ecf3c6a39d84e26b98da60eb1b prompt=1520 cached=0 thinking=False effort=none tokens=150 sha=033a197da586 finish=stop rounds=149 accepted=0/0 ttft=0.886 decode=6.593 elapsed=7.480 queued=0.000 tok/s=22.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-29986bf2e2a16057437d9d6e93ff9a41 prompt=4202 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.048 decode=0.248 elapsed=3.295 queued=0.000 tok/s=44.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-416c80aa97b91056cadc505cc6dd141d prompt=4202 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=3.030 decode=0.509 elapsed=3.539 queued=0.000 tok/s=21.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6e3f55b4e62bdce3363abe75724a10f0 prompt=9702 cached=0 thinking=True effort=high tokens=110 sha=88f98460968a finish=stop rounds=47 accepted=63/86 ttft=6.391 decode=2.747 elapsed=9.138 queued=0.000 tok/s=39.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-cf043bebba2aa06af1d3f61694a8e31c prompt=9702 cached=0 thinking=True effort=high tokens=110 sha=88f98460968a finish=stop rounds=109 accepted=0/0 ttft=6.351 decode=4.948 elapsed=11.300 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-251c692311322c451c02feedc912ad46 prompt=15180 cached=0 thinking=True effort=high tokens=160 sha=5694b66a8995 finish=length rounds=76 accepted=83/115 ttft=10.056 decode=4.278 elapsed=14.334 queued=0.000 tok/s=37.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-7eaec2ae9586dfd7d7a433c05dafe246 prompt=15180 cached=0 thinking=True effort=high tokens=160 sha=5694b66a8995 finish=length rounds=159 accepted=0/0 ttft=10.022 decode=7.198 elapsed=17.220 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-884abbd7379bcc14976410b7ec779bc3 prompt=19808 cached=0 thinking=True effort=high tokens=141 sha=025c4856c697 finish=stop rounds=48 accepted=93/143 ttft=26.473 decode=4.811 elapsed=31.284 queued=0.724 tok/s=29.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-d223f62e91f25a76c41c821feae4bfb7 prompt=18246 cached=0 thinking=True effort=high tokens=160 sha=9599ff0fa9b1 finish=length rounds=64 accepted=95/171 ttft=11.847 decode=19.839 elapsed=31.686 queued=0.000 tok/s=8.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1807 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7232 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1808 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7215:7215 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/server-rank1.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/server-rank1.log new file mode 100644 index 0000000000..563c73656a --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/server-rank1.log @@ -0,0 +1,5055 @@ +[kvp-serve] 2026-10-22T17:37:11+00:00 rank 1 of 3 context 1048576 parallel auto binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=off TF_GLM_MM=unset args --prompt-cache-gib 0 +info: rank 1: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-b:6303:6303 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:6303:6303 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:6303:6303 [0] NCCL INFO cudaDriverVersion 13030 +host-b:6303:6303 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:6303:6303 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:6303:6303 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:6303:6303 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:6303:6303 [0] NCCL INFO Count not open libibverbs.so +host-b:6303:6303 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:6303:6303 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:6303:6303 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:6303:6303 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x81d8430 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:6303:6303 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x8219b30 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:6303:6303 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x825b230 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:6303:6303 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x829c930 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:6303:6303 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:6303:6303 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:6303:6303 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:6303:6303 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:6303:6303 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:6303:6303 [0] NCCL INFO Initialized NET plugin IB +host-b:6303:6303 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:6303:6303 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:6303:6303 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:6303:6303 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:6303:6303 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:6303:6303 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:6303:6303 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:6303:6303 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:6303:6303 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:6303:6303 [0] NCCL INFO Using network IB +host-b:6303:6303 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:6303:6303 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x55a0da0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xd5e2be80812c97fd - Init START +host-b:6303:6303 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:6303:6303 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:6303:6303 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:6303:6303 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:6303:6303 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:6303:6303 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:6303:6303 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:6303:6303 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:6303:6303 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:6303:6303 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:6303:6303 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:6303:6303 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:6303:6303 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:6303:6303 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:6303:6303 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:6303:6303 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:6303:6303 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:6303:6303 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:6303:6303 [0] NCCL INFO Rank 1: 6 Net devices +host-b:6303:6303 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:6303:6303 [0] NCCL INFO comm 0x55a0da0 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:6303:6303 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:6303:6303 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:6303:6303 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:6303:6303 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:6303:6303 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:6303:6303 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:6303:6319 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-b:6303:6318 [0] NCCL INFO [Proxy Service] Device 0 CPU core 11 +host-b:6303:6318 [0] NCCL INFO proxy listening socket at <59047> +host-b:6303:6320 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-b:6303:6303 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:6303:6303 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:6303:6303 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:6303:6303 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:6303:6303 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:6303:6303 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:6303:6303 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:6303:6303 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a78010000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a78054000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a7809c000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a780e0000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a78128000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a7816c000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a781b4000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2050 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2051 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2050 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2050 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2051 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2051 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6598 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6599 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6600 pkey=0 pd=0xf01a7809bd50 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2178 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2179 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2178 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2178 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2179 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2179 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6630 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6631 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6632 pkey=0 pd=0xf01a782cf540 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2052 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2053 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2052 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b4f4c ctsFifoLkey=0x1b4f4c +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2052 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2053 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b4f4c ctsFifoLkey=0x1b4f4c +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2053 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2180 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2181 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2180 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1db2b1 ctsFifoLkey=0x1db2b1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2180 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2181 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1db2b1 ctsFifoLkey=0x1db2b1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2181 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a78481000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2050 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2051 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2178 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2179 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2052 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2053 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6601 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6602 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6603 pkey=0 pd=0xf01a7809bd50 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2180 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2181 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6633 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6634 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6635 pkey=0 pd=0xf01a782cf540 +host-b:6303:6303 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:6303:6303 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:6303:6303 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:6303:6303 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:6303:6303 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a785ae000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a785f3000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a78638000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a78680000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2054 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2055 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2056 pkey=0 pd=0xf01a78053e10 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2182 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2183 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2184 pkey=0 pd=0xf01a782a94c0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2057 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2058 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2059 pkey=0 pd=0xf01a78053e10 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2185 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2186 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2187 pkey=0 pd=0xf01a782a94c0 +host-b:6303:6303 [0] NCCL INFO Connected all trees +host-b:6303:6303 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:6303:6303 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:6303:6303 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:6303:6303 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a788a7000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a788e8000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6604 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6605 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6604 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6604 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6605 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6605 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6636 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6637 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6636 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x15f7f4 ctsFifoLkey=0x15f7f4 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6636 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6637 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x15f7f4 ctsFifoLkey=0x15f7f4 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6637 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a789df000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a78a23000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6604 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6605 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6636 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6637 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6606 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6607 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6606 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x164401 ctsFifoLkey=0x164401 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6606 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6607 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x164401 ctsFifoLkey=0x164401 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6607 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6638 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6639 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6638 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x18e0df ctsFifoLkey=0x18e0df +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6638 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6639 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x18e0df ctsFifoLkey=0x18e0df +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6639 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6606 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6607 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6638 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6639 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6303 [0] NCCL INFO Connected binomial trees +host-b:6303:6303 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:6303:6303 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:6303:6303 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:6303:6303 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:6303:6303 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-b:6303:6303 [0] NCCL INFO ncclCommInitRank comm 0x55a0da0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xd5e2be80812c97fd - Init COMPLETE +host-b:6303:6303 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 1.57 (kernels 0.28, alloc 0.05, bootstrap 0.95, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.28, rest 0.00) +host-b:6303:6303 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:6303:6303 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:6303:6303 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:6303:6303 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:6303:6303 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:6303:6303 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:6303:6303 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:6303:6303 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a78b19000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a78b5a000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a78b9f000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a78be3000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2060 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2061 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2060 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bd6d3 ctsFifoLkey=0x1bd6d3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2060 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2061 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bd6d3 ctsFifoLkey=0x1bd6d3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2061 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2188 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2189 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2188 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1deee4 ctsFifoLkey=0x1deee4 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2188 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2189 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1deee4 ctsFifoLkey=0x1deee4 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2189 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a78ce1000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6608 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6609 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6610 pkey=0 pd=0xf01a7809bd50 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a78d9f000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2060 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2061 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2188 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2189 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6640 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6641 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6642 pkey=0 pd=0xf01a782cf540 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6643 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6644 pkey=0 pd=0xf01a782cf540 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6643 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c7c4 ctsFifoLkey=0x19c7c4 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6643 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6644 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c7c4 ctsFifoLkey=0x19c7c4 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6644 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf01a78eb4000) +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf01a78ef8000) +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2190 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2191 pkey=0 pd=0xf01a782a94c0 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2192 pkey=0 pd=0xf01a782a94c0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6643 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6644 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6611 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6612 pkey=0 pd=0xf01a7809bd50 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6611 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x176d2b ctsFifoLkey=0x176d2b +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6611 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6612 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x176d2b ctsFifoLkey=0x176d2b +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6612 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2062 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2063 pkey=0 pd=0xf01a78053e10 oooRq=0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2064 pkey=0 pd=0xf01a78053e10 +host-b:6303:6318 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6611 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002,info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 1: rank 0 ended; leaving + comp_mask=0x0} +host-b:6303:6318 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6612 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6303:6303 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/server-rank2.log b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/server-rank2.log new file mode 100644 index 0000000000..274946f18c --- /dev/null +++ b/docs/receipts/glm-kv-pages-0b094b74/tp3/serve/r6off/server-rank2.log @@ -0,0 +1,1048 @@ +[kvp-serve] 2026-10-22T17:37:10+00:00 rank 2 of 3 context 1048576 parallel auto binary tensorfold-native 62649bbd8c2ce0d4 env TF_KV_PAGES=off TF_GLM_MM=unset args --prompt-cache-gib 0 +info: rank 2: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-c:6284:6284 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:6284:6284 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:6284:6284 [0] NCCL INFO cudaDriverVersion 13030 +host-c:6284:6284 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:6284:6284 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:6284:6284 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:6284:6284 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:6284:6284 [0] NCCL INFO Count not open libibverbs.so +host-c:6284:6284 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:6284:6284 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:6284:6284 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:6284:6284 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x1f9e9280 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:6284:6284 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x1fa2a980 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:6284:6284 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x1fa6c080 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:6284:6284 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x1faad780 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:6284:6284 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:6284:6284 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:6284:6284 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:6284:6284 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:6284:6284 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:6284:6284 [0] NCCL INFO Initialized NET plugin IB +host-c:6284:6284 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:6284:6284 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:6284:6284 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:6284:6284 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:6284:6284 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:6284:6284 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:6284:6284 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:6284:6284 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:6284:6284 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:6284:6284 [0] NCCL INFO Using network IB +host-c:6284:6284 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:6284:6284 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x1cdb1bd0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xd5e2be80812c97fd - Init START +host-c:6284:6284 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:6284:6284 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:6284:6284 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:6284:6284 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:6284:6284 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:6284:6284 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:6284:6284 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:6284:6284 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:6284:6284 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:6284:6284 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:6284:6284 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:6284:6284 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:6284:6284 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:6284:6284 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:6284:6284 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:6284:6284 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:6284:6284 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:6284:6284 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:6284:6284 [0] NCCL INFO Rank 2: 6 Net devices +host-c:6284:6284 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:6284:6284 [0] NCCL INFO comm 0x1cdb1bd0 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:6284:6284 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:6284:6284 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:6284:6284 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:6284:6284 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:6284:6284 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:6284:6284 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:6284:6299 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-c:6284:6299 [0] NCCL INFO proxy listening socket at <58953> +host-c:6284:6300 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-c:6284:6301 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-c:6284:6284 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:6284:6284 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:6284:6284 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:6284:6284 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:6284:6284 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:6284:6284 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:6284:6284 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:6284:6284 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e18010000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e18054000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e1809c000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e180e0000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e18128000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e1816c000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e181b4000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e181f8000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2050 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2051 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2050 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbcba ctsFifoLkey=0x1bbcba +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2050 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2051 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbcba ctsFifoLkey=0x1bbcba +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2051 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1794 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1795 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1796 pkey=0 pd=0xe99e1809bf70 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2178 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2179 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2178 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2178 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2179 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2179 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1922 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1923 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1924 pkey=0 pd=0xe99e182f1b30 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2052 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2053 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2052 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbfbc ctsFifoLkey=0x1bbfbc +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2052 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2053 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bbfbc ctsFifoLkey=0x1bbfbc +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2053 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1797 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1798 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1799 pkey=0 pd=0xe99e1809bf70 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2180 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2181 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2180 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d1c1b ctsFifoLkey=0x1d1c1b +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2180 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2181 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d1c1b ctsFifoLkey=0x1d1c1b +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2181 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1925 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1926 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1927 pkey=0 pd=0xe99e182f1b30 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2050 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2051 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2178 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2179 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2052 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2053 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2180 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2181 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6284 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:6284:6284 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:6284:6284 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:6284:6284 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:6284:6284 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:6284:6284 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:6284:6284 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:6284:6284 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:6284:6284 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e185bd000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e185fe000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e18643000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1800 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1801 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1800 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1807c5 ctsFifoLkey=0x1807c5 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1800 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1801 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1807c5 ctsFifoLkey=0x1807c5 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1801 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2054 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2055 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2056 pkey=0 pd=0xe99e18053eb0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1928 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1929 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1928 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19ebe8 ctsFifoLkey=0x19ebe8 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1928 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1929 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19ebe8 ctsFifoLkey=0x19ebe8 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1929 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e187a8000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1800 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1801 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1928 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1929 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2182 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2183 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2184 pkey=0 pd=0xe99e18283e70 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e18862000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e188a6000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e188ee000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1802 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1803 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1802 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17e4a1 ctsFifoLkey=0x17e4a1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1802 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1803 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17e4a1 ctsFifoLkey=0x17e4a1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1803 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1930 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1931 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1930 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1930 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1931 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1931 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e189e8000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1802 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1803 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2057 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2058 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2059 pkey=0 pd=0xe99e18053eb0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1930 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1931 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2185 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2186 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2187 pkey=0 pd=0xe99e18283e70 +host-c:6284:6284 [0] NCCL INFO Connected all trees +host-c:6284:6284 [0] NCCL INFO Connected binomial trees +host-c:6284:6284 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:6284:6284 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:6284:6284 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:6284:6284 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:6284:6284 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-c:6284:6284 [0] NCCL INFO ncclCommInitRank comm 0x1cdb1bd0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0xd5e2be80812c97fd - Init COMPLETE +host-c:6284:6284 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.53 (kernels 0.24, alloc 0.04, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.23, rest 0.00) +host-c:6284:6284 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:6284:6284 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:6284:6284 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:6284:6284 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:6284:6284 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:6284:6284 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:6284:6284 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:6284:6284 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e18b1a000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e18b5b000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e18ba0000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e18be4000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2060 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2061 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2060 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1b9795 ctsFifoLkey=0x1b9795 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2060 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2061 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1b9795 ctsFifoLkey=0x1b9795 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2061 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1804 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1805 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1806 pkey=0 pd=0xe99e1809bf70 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1932 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1933 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1934 pkey=0 pd=0xe99e182f1b30 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e18d79000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2060 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2061 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2188 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2189 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2188 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d9190 ctsFifoLkey=0x1d9190 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2188 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2189 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d9190 ctsFifoLkey=0x1d9190 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2189 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1935 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1936 pkey=0 pd=0xe99e182f1b30 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1935 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1935 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 1936 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 1936 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e18e6d000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99e18eb5000) +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99e18ef9000) +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2188 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2189 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1935 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 1936 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2190 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2191 pkey=0 pd=0xe99e18283e70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2192 pkey=0 pd=0xe99e18283e70 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1807 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1808 pkey=0 pd=0xe99e1809bf70 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1807 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1807 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1808 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1808 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2062 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2063 pkey=0 pd=0xe99e18053eb0 oooRq=0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2064 pkey=0 pd=0xe99e18053eb0 +host-c:6284:6299 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 2: rank 0 ended; leaving +ev 4 with dev 0 +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1807 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6299 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1808 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6284:6284 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/README.md b/docs/receipts/glm-kv-pages-8e705835/README.md new file mode 100644 index 0000000000..107b015fc4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/README.md @@ -0,0 +1,95 @@ +# GLM-5.3-Flash on CUDA: the engine's own sequence on KV pages, receipt + +Tested implementation: the binaries built from `8e705835…`, commit 20: #1163's 14 commits and the first six of the +pull request that stacks on it (commit.txt has the tree and the commits). The receipt's commit adds this folder and +changes no source. TP=3's ranks ran commit 20's engine source with the two local commits for the ring that #1163's +receipt describes (`tp3/ring.patch`): rank 0's NCCL id handed to the other ranks in a file (#1178 is the upstream +way), and a heat ceiling (`TF_HEAT_CEILING`: a prompt chunk also waits while its expected rise would pass the +ceiling), which changes when chunks run, not what they compute. +Base: #1163's head `d7ff6494…`, on main `f8d254c5…` (after 1.1.2); main's binary for the replay and the speed A/B +was built the same way (`bin-f8d254c5`). +Test date: 2026-10-23. Platform: MSI EdgeXpert (GB10, DGX Spark-compatible; sm_121, 128 GB shared memory), two at +TP=2 over a direct RoCE link, three at TP=3 over a ring of direct links; driver 580.178.04. Ranks in Ubuntu 24.04 +containers pinned to the efficiency cores (`--cpuset-cpus=0-4,10-14`), NCCL 2.30.7. +Toolchain: Zig 0.17.0, nvcc 13.3 (V13.3.73) for sm_121. binaries.txt has every binary's sha256. +Checkpoint: `nvidia/GLM-5.3-Flash-NVFP4` at `423acf37583782c51c142d145aef733d72943d93`. + +## Scope + +This qualifies the engine's own sequence (slot 0) on KV pages at TP=2 and TP=3: slot 0 on pages from its init +against its whole-window twin byte for byte, the ranks' agreement on `TF_KV_PAGES`, served exactness with every +stream on pages (concurrent = alone, drafted = `"draft": false`, resumed = fresh, images), the replies against main's, +the floor (one window of pages planned before the rest) holding a long request while short ones share the pool, +admission when the pool is down to its floor, speed against main at TP=2, the no-GPU CI and the mutations. The +kernels are #1163's, unchanged, so the kernel contract and gate 5's records are in #1163's receipt. It does not +qualify FP8 caches on pages (refused by name, as in #1163), DFlash2 drafts (no weights here), Metal or MLX. + +## What changed for these runs + +- `TF_KV_PAGES=all` is now `on`: every stream, the engine's own among them, is on pages. So `r2all` and `r2on` serve + the same configuration twice, and `r6all` is the concurrency check at a 96k window with all eight streams on pages. +- Every rank resolves `TF_KV_PAGES` and the ranks agree on it; the chains pass the same environment to every rank. +- The floor is shown two ways: `r6admit` serves a 96k window with the pool down to its floor and a page a stream + (64k requests: one fits, the next waits), and `r6auto` serves seven 2k prompts beside a 64k conversation at a 1M + window. No request was sent at 1M tokens (below). + +## Results + +| Check | Result | +| --- | --- | +| `glm-paged-check`, TP=2, first five layers (`tp2/check/`) | Rank 0: on the engine's own sequence, on pages from its init, 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their whole-window twins' logits, tokens and drafts byte for byte, the pages out of order (12 jumps) and NaN before use, the pool idle after; rank 1 made rank 0's pool and mapped its pages through every step | +| `glm-paged-check`, one GPU (`single/glm-paged-check.log`) | The same 7 prompts and 91 rows on the engine's own sequence, byte for byte | +| Synthetic checkpoint, `TF_GLM_LAYERS=2` (`single/reg/`) | `glm-ref`, `glm-synth`, `glm-lanes-cover`, `glm-lanes-concurrent`, `glm-shared-check` and `glm-paged-check` pass, 0 failed | +| TP=2 `all` and `on` (`tp2/serve/r2all`, `r2on`) | The same configuration twice: "4 streams at once, the engine's own and 3 more taking 803 pages of 512 rows (5.00 GiB) as they need them (enough for one at a 131072-token window), 180 MiB fixed each, of 9.5 GiB left after a 12.2 GiB reserve" (757 pages under `on`, whose serve had 9.2 GiB left); #1163's had 504 pages with the engine's own sequence idle under `all`, and 509 beside its window under `on`. `bench_concurrent.py`: 12 cells each, every reply equal to its alone run (28/28) and to `"draft": false` (30/30); each reuse record's 44 requests all cached as planned; every idle census `free 803 + store 0 == total 803` (757 under `on`) | +| The `on` record replayed on windows and on main (`tp2/serve/r2auto`, `r2main`, `tp2/compare/`) | `auto` keeps windows at TP=2 ("4 streams at once, 1.77 GiB each at a 131072-token window"), as main does; both replays against the `on` and the `all` records: 43/43 replies equal and as cached as planned in all four, drafted == plain on the same messages 23/23, no resumed reply slower than its fresh one | +| Images on pages (`tp2/serve/r2vis`) | 561 pages (3.49 GiB; 340 MiB fixed each with `--vision`); `media_http.py`: drafted, repeated, `"draft": false` and three at once give the same tokens (prompt 2,050); `media_resume_http.py`: resumed past the image (6,144 of 7,603 tokens cached) = cold, as are `"draft": false` and two at once; idle `free 561 + store 0 == total 561` | +| Speed at TP=2 against main (`tp2/speed/`) | Serves in turn: this head (windows, as `auto` keeps them at two ranks), main, this head, main, then this head with `all` (every stream on pages, the engine's own among them). Decode, the 32k chat and cold prompts of 1k to 64k: this head's faster serve over main's slower 0.990 to 1.000, `all` 0.994 to 1.005; at 64k main's own two serves differ by 0.7%. The same tokens in all five | +| `glm-paged-check`, TP=3, first five layers (`tp3/check/`) | Rank 0: on the engine's own sequence, on pages from its init, 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their whole-window twins' logits, tokens and drafts byte for byte, the pages out of order (12 jumps) and NaN before use, the pool idle after; ranks 1 and 2 made rank 0's pool and mapped its pages through every step | +| TP=3, pages by default, a 1M window (`tp3/serve/r6auto`) | `TF_KV_PAGES` unset pages at three ranks: "8 streams at once, the engine's own and 7 more taking 3355 pages of 512 rows (20.89 GiB) as they need them (enough for one at a 1048576-token window), 131 MiB fixed each, of 37.8 GiB left after a 12.2 GiB reserve" (#1163's: the engine's own 12.88 GiB window in the model's and 1,284 pages for the other seven); `bench_concurrent.py`: 16 cells, every reply equal to its alone run (60/60) and to `"draft": false` (72/72); the reuse record's 44 requests all cached as planned; cold prompts of 1k to 64k tokens 0.73 s to 43.4 s (`cold.json`); seven 2k prompts while a 64k conversation held its rows on pages (63,488 of its 65,536 tokens resumed from the cold run's kept state): all seven equal their alone runs and none waited for another request to end (`long-beside-short.json`); every idle census `free 3355 + store 0 == total 3355` | +| TP=3 `all`, a 96k window (`tp3/serve/r6all`) | `all` is `on`: "8 streams at once, the engine's own and 7 more taking 1544 pages of 512 rows (9.61 GiB) as they need them (enough for one at a 98304-token window)", the bound of eight windows' pages; 16 cells, 60/60 and 72/72; in the cells of eight no request waited (the longest first token 1.69 to 2.15 s; #1163's `all`, with the engine's own sequence idle, had the eighth wait, 12.0 to 21.2 s); every idle census `free 1544 + store 0 == total 1544` | +| TP=3 `off`, the `auto` record replayed (`tp3/serve/r6off`) | A window a stream with `--parallel auto`: "2 streams at once, 12.88 GiB each at a 1048576-token window, of 24.4 GiB left after a 12.2 GiB reserve"; against the `auto` record (`compare-auto.log`): 43/43 replies equal and as cached as planned, drafted == plain 23/23, no resumed reply slower than its fresh one | +| The pool at its floor (`tp3/serve/r6admit`) | `TF_KV_PAGES=all` at a 96k window with a 37.2 GiB prompt cache asked: the plan trims the cache to 35.9 GiB to keep the floor, so the pool is the floor and a page a stream, 200 pages (1.25 GiB); `kvp_admit.py --fit` took 64k prompts, 129 pages each, so the pool holds one and not two. Three alone (cold, each kept in the prompt cache), then together: 3/3 equal their alone runs, each resumed 63,488 of its 65,536 tokens, and they ran one at a time, each starting after the one before ended; idle `free 200 + store 0 == total 200` | +| Builds on a GB10 (`tp2/build.log`, `tp2/build/`, `tp3/build.log`) | Commit 20, nvcc 13.3.73 `-Dsm=121`: `zig fmt --check` on the changed files, the build, the native server, `test-cuda-host` (155/155 steps, 2,516/2,522 tests, 6 skipped), `test-kv-pool` (17/17) and `test` (276/276 steps, 2,023/2,037 tests, 14 skipped) pass. Their `failed command` lines are zig's build runner showing the stderr of tests that print on purpose (a comparator's FAIL line for a planted NaN, a rank's stop message), and `lean rc=1` is the lean list's exit with main's 438 problems, none new. The ring's binaries came from the TP=3 tree's first build, commit 20 with `tp3/ring.patch` (`tp3/build.log`), whose steps pass the same way; the TP=2 chain built the same commit again later (`runs/tp2-chain.txt`, its binary another sha), and the ring ran the first | +| No-GPU CI under WSL2 on commit 20 (`ci/`) | `zig build test` 3,470/3,486 (15 skipped; the failure is upstream's `cuda_cost_identity` mtime test, WSL2's coarse file times), `test-kernel-contract` 72/72, `test-golden`, the server and pull security tests 20/20, `fuzz-cortheon` 14/14, the served Flash Next unittest (8), the GLM snapshot, KV pool and CUDA host tests 72/73 (the same failure); lean 438, as on main | +| Mutations (`ci/mutations/`) | Thirty guards, each broken alone on a copy of commit 20: #1163's fifteen and fifteen for the engine's own pages (the floor, its bound, `holds`, the flag to `plan` and `holds`, `window_rows`, `Plan.size`, slot 0 made and placed on pages, a pool refused beside a window, admission by the pool alone, the ranks' agreement and fingerprint). 29 fail a test; #1163's `bytes-on-paged-seq` no longer applies, as its line became `Plan.size`, which two of the new fifteen break. The unchanged copy fails no test (`head.log`) | + +## Reproduction + +Every serve: `tensorfold-native serve CKPT --name glm-tf --host 127.0.0.1 --port 8097 --max-tokens 32768 +--reasoning-effort high --parallel P --context C`, drafts on, one rank a host (`TF_GLM_RANK`, `TF_GLM_WORLD`), +`TF_HEAT_HIGH=92 TF_HEAT_LOW=88 TF_HEAT_CEILING=93`, the same `TF_KV_PAGES` and `TF_KV_POISON` on every rank. +TP=2: `--context 131072 --parallel 4`; `r2all`, `r2on` and `r2vis` add `--prompt-cache-gib 4` and `TF_KV_POISON=1` +with `TF_KV_PAGES` `all` or `on`, `r2vis` `--vision`; `r2auto` and `r2main` and every speed serve +`--prompt-cache-gib 0`. TP=3: `--parallel 8` and `TF_KV_POISON=1`; `r6auto` `--context 1048576` with `TF_KV_PAGES` +unset and the default prompt cache (16 GiB); `r6all` `--context 98304 TF_KV_PAGES=all`; `r6admit` `--context 98304 +TF_KV_PAGES=all` with a 37.2 GiB prompt cache asked (the `all` serve's room less its streams' fixed parts), +which the plan trims to keep the floor; `r6off` `--context 1048576 TF_KV_PAGES=off --prompt-cache-gib 0 --parallel +auto`; NCCL over the ring's direct links with `NCCL_DEBUG=INFO`, and each TP=3 host's memory guard (v3's +`hostwatch.sh`: the server stopped below 5 GiB free). The checks are the repository's tools: +`bench_concurrent.py --alone --serial --tokens 256 --reps 1`, `reuse_served.py` (`record --planned --min-prompt +2048`, `replay`, `compare`, `cold`, `ids`), `bench_openai.py`, `bench_cache.py --require-cold`; here `media_http.py`, +`media_resume_http.py`, `kvp_admit.py` and `kvp_sum.py` (tools/; `kvp_sum.py conc` ends non-zero when a request +failed or a reply differed). `tf-cuda-test glm-paged-check CKPT RANK WORLD IDFILE` with `TF_GLM_LAYERS=5`. runs/ has +the chains' own output. + +## Remaining coverage limits + +- No request reached a 1M-token window's length: the floor's pages for one such request are in each 1M startup line + ("enough for one at a 1048576-token window"), and `r6admit` shows the floor holding a 64k request at a 96k window. +- TP=3's ranks ran commit 20's engine source with `tp3/ring.patch` (above); TP=2's and one GPU's ran commit 20 alone. +- Ranks 1 and 2 of the TP=3 check print their PASS lines in the middle of NCCL's INFO lines (`NCCL_DEBUG=INFO`). +- Speed against main was measured at TP=2 only. +- `--kv-cache fp8` keeps windows: `auto` does not page and `on` and `all` are refused (`Fp8OnPages`). +- DFlash2 (`--drafter`): no weights here; the drafts on pages are the MTP head's. +- Metal and MLX were not run (no Mac). + +## Publication handling + +CRLF line ends made LF, trailing blanks dropped and an indent with a space before a tab made spaces (as `git diff +--check` asks); machine names, internal addresses, home and local paths, and the work's session, plan, records, +container and image names replaced with placeholders (`redaction-report.json` counts them a file; host-a to host-c +run TP=3's ranks 0 to 2, host-d and host-e TP=2's ranks 0 and 1); result values, token ids and token hashes kept. +The TP=3 server logs keep NCCL's INFO lines as they were written. The reuse records keep their rows without the +request bodies: each row has its body's sha256 and size, and `records-sha256.txt` the originals'. The speed runs' +prompt files are not included: `reuse_served.py cold` and `ids` make them again. SHA256SUMS pins every published +file except itself. diff --git a/docs/receipts/glm-kv-pages-8e705835/SHA256SUMS b/docs/receipts/glm-kv-pages-8e705835/SHA256SUMS new file mode 100644 index 0000000000..f5cdd2afd4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/SHA256SUMS @@ -0,0 +1,165 @@ +84fa594cde35cea7e998726c9c85fe2015d752e2bcf86219aa2cad0ad50c9274 README.md +9f4fcb9bbe91a29b7601ff2e7b2bc0f0673a59ace24ff00c0c86e096da0fce81 binaries.txt +deb0a20d69499c6a8442d3df7c8b1397aff5532285cb7011d60b96ed21ef04f9 ci/flashnext-served.log +74231526718f4e53c9c68356dcfd3e8934bfc63f6963084ba44a7a6d892cf220 ci/fuzz-cortheon.log +b75cf194e0b674b1cbbec2a6a9c7bc9910856f93e1300660173b31802a4b5674 ci/glm-host.log +5346a387792d47a2d7988b569e011afc223a871635a6399763485557ceb3121f ci/lean.txt +b6dd7018e0413f5063c0abec89b0abe1c45e6dd121c09379c0fbcc5d529ea4c4 ci/mutations/auto-at-two.log +c0735da6b956a20de037d1b71e2fec17805e62e287343ca014c24d8e314f67ce ci/mutations/bind-before-pages.log +7339f661768a46fa9f2e42caaa7e45b4c2bd71e6d84581821ab7cd70f70d3252 ci/mutations/close-keeps-pages.log +0d763d6d53dc47e76ab2e53c395e2d291e1b99a2df62e9d95cb67115281d5047 ci/mutations/fits-flag-dropped.log +f401d6f7ade1d370e5f09d1fc67a6d029d6ccf913b361699496ae95fc3b3cdf6 ci/mutations/fits-own-free.log +c79b6e33256256600580a46c3205ff9fbc1cc1e611ffd1b7af3b1885fd5966c7 ci/mutations/fp8-kernel-table.log +7f135db8a839ff452f088e2510a476263fb83d58b36b41106811d1b51e949d75 ci/mutations/fp8-on-pages.log +308c3529a78d472c3b7ab20bc8374dc85732e9cb4356a7f4b73329231a3e85c2 ci/mutations/head.log +5f9efbe80026c26019ee0da082853cdc2092964aa4af12b843036c5134fc1b20 ci/mutations/holds-no-floor.log +d4e68cb1f7bdf528d35580cb9765b4ab828c2bcfbe27027144b77fed640a4265 ci/mutations/kept-window-cache.log +fdc6a153b54753f7ffc204ad22b4d6ff1f0d91ba63013beab621cbea57b77d91 ci/mutations/map-any-id.log +3c72a9c1b98f94ecd1bdd611b9bffb86ce8e24241d9f00bad4fb2440506c0867 ci/mutations/no-poison.log +88c5ea15386ad304cbade897af64096ff6bf7a05421377fe8ac10d8cc4526db9 ci/mutations/no-verify-rows.log +9dbea4284c7a7cb3ba477b7882daa28b21e7581eeb9c922b89091cd865eb8c97 ci/mutations/own-as-window.log +77f4f7517b857dac087eb472d7f72718b84941f81085fcdec3de53b4872b770f ci/mutations/own-no-floor.log +30e307df3892d8f3bd66ade85f33e82cae07e2234a7dee7acf7c798a653c2dce ci/mutations/own-not-bounded.log +a4c8cfe2dcaf53ff7480d87bda591d49af86c97ee73037b20b81d23db151bc77 ci/mutations/own-not-placed.log +5ebe7074c387e1539380ea21551606d8ae1c7286799b8a9d9c961fb9eabf4659 ci/mutations/page-bytes-unchecked.log +f1d4fe2cd3693b00551c8adc63bccee580d1f473473fc12dfaaa7db7ce1f3797 ci/mutations/paged-first-at-cache.log +4fe5f7d0820f99069899042aa5037a35676232aec5ea9d9ae6aa05ffe0d6045b ci/mutations/paged-table-kept.log +60a47c7fc8160cba117ca8c8251a87496901bcfa00ac290717bcc0297782eb96 ci/mutations/pages-not-agreed.log +7a4d85eaf6b44c1afb7fb9e0fdb9d7b0cb1dacfa67fbc51a6d02b2a200bb57a9 ci/mutations/pages-not-fingerprinted.log +f7c25596d0f0adc60914af6901533f0eab717404c2232c154e1dd9a02a875ab3 ci/mutations/piece-first-page.log +38fa856084533b282159dc7064378322b20d9bb158a6fbd44e1e6816d5b875e4 ci/mutations/piece-no-offset.log +3c49e08b8f9185783863a0a8944de41e255e3a987ad65f41febd73521e27201d ci/mutations/plan-flag-dropped.log +24ef3ab80b69166d68eea313268a23d6b6f82bbd72061072f510f37aeb9ffc68 ci/mutations/pool-on-window.log +2ce37ee98489758aea2776bc689b898edac4afaf0222e9a5cc76b0f41684d8a2 ci/mutations/size-at-held.log +15e9417773e44c6fc3c7f58f211208f92ef7fe3c75a07e01d281a90618c91ea0 ci/mutations/size-mapped-only.log +a0c4d7f7cea2ef235feaad621b7282c750629c89ee616f46302f358a759745b1 ci/mutations/slot-zero-unpaged.log +1409497b9f29aac7c20c066a0550157ef766e56fd6c0348f571f51fb8e19a90c ci/mutations/split-no-slots.log +a6d8a4a58e25c55b3189d396e8044d83a8ca9a8d0aae0cd8b21f7548d78c8695 ci/security.log +36d559695f5e16c3ef2de61a69acc9eed9056278580931e7ce5d350cd6406f41 ci/test-golden.log +ec8634ffe3001ce3f1b4403d11193b860fb464375f9782853c7153e30763b109 ci/test-kernel-contract.log +ac9e1ea69c3d8c9fef0350e1a279ae9bd4d405996e1bb76933c47fde3f1014fb ci/test.log +ca09fbbec73068ebfb150d0d81676058eb09243015cc9edd4de2eaa164c03b82 commit.txt +f40986f9d9260e0ac2ef34986eef230f6a9e9a142862cb1406929f6c862933e0 records-sha256.txt +f744278d513e0893774705e61018d03d575f8e3ee94c3f774ea70eb7b40e8d20 redaction-report.json +7d8c4585176ac5440c98d20aa066863eecdabbde0ff70c4dbbd71053352eaaf2 runs/tp2-chain.txt +8a7641bfda1a7f1690eaffab3c70f64282bc560d876016e2a974ac4be65eacdf single/glm-paged-check.log +adbf33709a37fbc4324ff069209cfee21365875cdd1f406bbbcf5a6b40c43cb5 single/reg/glm-lanes-concurrent.log +dbd448fe0919788628aba3969e1604ec34be032e4e65e48b9ef8dd1dd6b4adeb single/reg/glm-lanes-cover.log +b21a34a1250dc015643d1f936fd2f8adbfd2c94638af9efbe967c144a263ea60 single/reg/glm-shared-check.log +f9c012dcbfbcf6812b5868c4c3d8df59ed3bd179927b1e266b9d32434b9fe83f single/reg/paged.log +cce62e664ca901f5bb9f7fd149c901be4dbcab00a16d55997d10bab4a456a73a single/reg/ref.log +3b4647b14a4b406949b8be39e71615b8ee911af57d7315c055ca41c7f64274b6 single/reg/summary.log +f5592213010036b87e5d313c612cd157866e910d30bca53cd1fba3db2c22a1e0 single/reg/synth.log +2e47bb5e010aaeef606b482811b0c7e15bd0b288e2ba86dc6f2c93c03dccdcf6 tools/kvp_admit.py +de2fb053f9cb118ba8f4e828944231bec917a4bc4aa049db8c26364f17883bb2 tools/kvp_sum.py +45166e626e21f39a16b5d399476f0a402df19f7993b669fde4838a3542878761 tools/media_http.py +16b709bd8b79d660f03b62ef0f45a1438413cc8516158b7851e06837207dab0b tools/media_resume_http.py +fda5d3a62817f8a6886382e5f8bbfb616fc309ddde46892a0fce2e372027d6ee tp2/build.log +f08cf8d12cdf11384a9cb56449ac372d9ae0a6fa3c92f2f69cc34321c7d1ce35 tp2/build/build.log +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 tp2/build/fmt.log +d85adaef029c65b6cc79d5c27ebbef72a764707539470f11154e559756b4a854 tp2/build/host.log +ac9ff3b7cbe66732afbaaca84d87116d2ea3aa79dbfb9a722191434bd52b6071 tp2/build/kvpool.log +5346a387792d47a2d7988b569e011afc223a871635a6399763485557ceb3121f tp2/build/lean.log +d48282c1bd3ce423d030b439b7ecdb29bd70691ca7849ab83be25d19cbaadb93 tp2/build/native.log +8f0cb253b4006ac9906b8a48c13730f528bf18ab4924c33f06cf44a5e964c634 tp2/build/test.log +0a40142df0411f0df44a359cbcfe2218f96da069e84022a1c8b2b65eef55a34b tp2/check/glm-paged-check-rank0.log +27a3c10399e022ee115c4c1157977803b7be564ab02b1dfa030a030a41fd5bd0 tp2/check/glm-paged-check-rank1.log +ee02ff837aa9dfd34e5e4d0cf00f98fca55915434d0096ee5b1e1917372197e5 tp2/compare/auto-against-all.log +14de5cce8337dc87a0c53fb6d7273e3d5b7e629acd4d10ac6abee4bf212c0535 tp2/compare/auto-against-on.log +c1d1407551c2485ebd1dd1478304dfac188f010b0bf04849b30101a3e618864d tp2/compare/main-against-all.log +ac3f4b33ed6dcd5c2200d60a228a6dface94237a61023d67a554fee3dfd7b2d1 tp2/compare/main-against-on.log +112f23cb5988b2dc56c235105b1336ef61de2cdc0bb5434a52c71aa1d38f4381 tp2/serve/r2all/client.log +83fd4d62a768b5958d63745d74c590a3eb2fe698cb55f3609da400db833ad498 tp2/serve/r2all/conc.json +5c1ef4923ffe3d84cceee27ed91371266d27d202230deae2bb47f11ae729d93e tp2/serve/r2all/rows-without-bodies.jsonl +f6485167d35d02cc5f139944005987efa79ebfcdf5b7adc05940f669aec7034e tp2/serve/r2all/server-rank0.log +43d59b4728bd6364cb55a87396254db80d468b00443e1f2a897c55e6f58798c2 tp2/serve/r2all/server-rank1.log +b87cc69063d46f8c2ad3791a708d2113bfb890e28d9b2cae9acc486f773eecf1 tp2/serve/r2auto/client.log +f4eb60391557be1ad3dd176756a183a685210a13ecc1b432ae92d404cd2139bc tp2/serve/r2auto/fresh.jsonl +98a43e55261544d7325cee5e11299707d25098f8b771d6bab51386db248d82ac tp2/serve/r2auto/server-rank0.log +b602ada1d7f3e37ab8aef8048bb89580854cd927e5d6adc9fc0d0fac1d831694 tp2/serve/r2auto/server-rank1.log +b9a2cb4236d0117f23fbc56c1dc96cc1e81a8970fca6ddf939a31a58c7700644 tp2/serve/r2main/client.log +3852491fd4e2cf58ae23ef4ec129dc85d689ad1bee5b9404288d29e277d31431 tp2/serve/r2main/fresh.jsonl +ba3b3df684006813273652d5d682904998898fbcb48bc528c5ac6337cd88c5e2 tp2/serve/r2main/server-rank0.log +16026c9d303b7c6f6d4ac35ce37005e4e0ed485bb474627adb0eafe84e930b69 tp2/serve/r2main/server-rank1.log +d0cdf9812128b2de2fbc036fc9d3a602e970180e26144c8b4798b7f13f3a660b tp2/serve/r2on/client.log +ce833f3efad0f518b1dcdcf8f7fae5e2404f8432b28dd7a8343c2f0810984223 tp2/serve/r2on/conc.json +eed4389f529af9ff5e1a98274355a520f6c622286833414f53998cc3fb83e9cf tp2/serve/r2on/rows-without-bodies.jsonl +1089146b2404683096ea6397ddfa6d0e0569201f1af47574706be49a373279db tp2/serve/r2on/server-rank0.log +50d66ff1495572a8292c787d0914d35443df34a69ba6891079489b8bf377d3ce tp2/serve/r2on/server-rank1.log +ddb26bab144afb8b2fa2a8918f8885ca418b22fd8e11fde82ab8b0188b764485 tp2/serve/r2vis/client.log +2b8fd0e9adc1e1e7f622aa301116443c63a2219b9528ea9aee56bf396c17717d tp2/serve/r2vis/media-resume.json +c5ff34f4f3d2cd1425da1e8740f8acefccb922bd308e11efdb751f71a5ff0f55 tp2/serve/r2vis/media.json +92cdde3d43a454f7a63bdb24e0b1381659bbddbc6f2c4fce2b0cc1853ee02f97 tp2/serve/r2vis/server-rank0.log +21ed58d8a4fd8bb57e63729bd8011c7082f3b283523809f49aae79f42efab2e8 tp2/serve/r2vis/server-rank1.log +138ebd4ec39794a74f883a5d88ea32a300afda46a4aa5da74ac4ed79225277f5 tp2/speed/r3mainr9/cache.json +ea44be4af64dcf3d467a7edd52bcd049a265b9afee3f54cabff373459924260c tp2/speed/r3mainr9/client.log +8a42a6bbc6788b5656685b7ae2258ff3b039a4613ec73a5126b1442b9760c1ec tp2/speed/r3mainr9/cold.json +bcf72259135e3acbf48fbcfeaff8ef91963a4b77cf04e2165b496295efe382d8 tp2/speed/r3mainr9/decode.json +68d459646608eae21ad948da9877eadd95607b8c9363b47053f65f2001e017a8 tp2/speed/r3mainr9/server-rank0.log +848079b2ae038ff9b6a96cb55eff1cfdfbb44cc3185e1d3426c96caa50ce4d19 tp2/speed/r3mainr9/server-rank1.log +6ccc510902f7d55b397d6f685231926f64fb7c811239b495f089dd879ee1a81e tp2/speed/r3mainr92/cache.json +242ef6741f2ba8f420de58bb96db691dfe6dbbaebbdd3a86ab5f67db297c38b6 tp2/speed/r3mainr92/client.log +3443a8a974f13c17bac0cea5c825dab8042939f2acbc45f99f9e518f1f3b6344 tp2/speed/r3mainr92/cold.json +db36124271afb708f4b40f1c6224e80f59dfa65b787a82aa081d3d8b3854855b tp2/speed/r3mainr92/decode.json +86eb748ed752b4e16ae322539b60be64a2f9ba7e425a982fa0dc90c21125b25b tp2/speed/r3mainr92/server-rank0.log +709c39969c8417c6ad5e11d6d2d1eab1d892189544844733e0799fe2fe684fc6 tp2/speed/r3mainr92/server-rank1.log +a9997dd1db94bfe05c669ce7b9aa60acf5bc91183c365032a0dd2588ebecf01e tp2/speed/r3r9a/cache.json +e115461cd1c2602b3439844b76bf143fd83c8db9386aaf749f615f87d5ce4939 tp2/speed/r3r9a/client.log +ce03d9b3938f310ebb33b17132a8d67381b53f4c0192375c486223cbc6e8874a tp2/speed/r3r9a/cold.json +46a6f1f1f370e621c618cf9a93d0337d623c2adb77b9ce152e0b95c93427bf2c tp2/speed/r3r9a/decode.json +4d9b3505c14a1bd5b15ea68e16ebc79178b775a963e1a657b5ffef8038b5ae7b tp2/speed/r3r9a/server-rank0.log +788b68c0b4139b3d8c590672624f2155e8bf198a6afc7a5d72105d2fd2b76f5b tp2/speed/r3r9a/server-rank1.log +6ed7df3f996a478c84bbc07412f938a30c294b09c77b0bf96ffef32183905711 tp2/speed/r3r9all/cache.json +2e2f944b71a7a728277db7788aa2a1c4111cf66cb434c6ce62dc3492dba90203 tp2/speed/r3r9all/client.log +5d66cb945eb8f1856875e2d477e3b1eb142fc3906ad44e9a8f925683da3c3e53 tp2/speed/r3r9all/cold.json +56a1497ef507c4a4002928eca780607e1741a59d341ada643cf3115395837a65 tp2/speed/r3r9all/decode.json +b2b5f0ed11ef3912ca9d9e4bcedfbac2830ce76504a924bf9445be28ce5e6b1a tp2/speed/r3r9all/server-rank0.log +c1f92ef7026657cc35468a7ed48f88ffb88dad1675ffe0afcd2a816696b89390 tp2/speed/r3r9all/server-rank1.log +df1f23c5f1201000505f367be74d9b98295586386668ef260eb7dc05bbc776ef tp2/speed/r3r9b/cache.json +f50351248f2ecb8fee76b83fe7ad52325dfc45368261e2aa5c172b63e10a68fd tp2/speed/r3r9b/client.log +5a1f5edc88af315f2ec014f90e623e7cd352b5422592bd45e9c3abf49aa1c767 tp2/speed/r3r9b/cold.json +4716fa6799ce165896189e4259967fb3fccf7483341e865c6bb01140940710d8 tp2/speed/r3r9b/decode.json +a0efd82494bf82f479839f4f28ff7bdc0c066c3fa07b511f6cc470a7931e0703 tp2/speed/r3r9b/server-rank0.log +b4f63f1c2004a761f5ac3f306eab4caa8fcffe451b6b40b7ba6738c9ec4b1d47 tp2/speed/r3r9b/server-rank1.log +e38345389fcc47f19c5ac76c39966242f790caa87ae7dfc6ff77929eac161658 tp3/build.log +66314b0477ec337318dbfdb972b04e4f64ab6743d53777d066238346a8fc9e3f tp3/chain.log +48703db44884c67b04cad7010e703703f7bfa53ae3308722ee10b2aeaa1f2baf tp3/check/glm-paged-check-rank0.log +8ad1401683e76ee32e8854a9c9ed8f735a89fe856bd02a389d9ac9c0efdb6b3e tp3/check/glm-paged-check-rank1.log +c8ba89c986aa541d2835d3491bce815ce4cc8805896d4a479545b015444669ac tp3/check/glm-paged-check-rank2.log +32ec884f53e91113dbf60e338729882701bbdc1951db21eda73796fc8a8d61d7 tp3/ring.patch +9aa551ecc2f7551fe7027b6b687a76a50104f620d2249603f05fb28558f9f10c tp3/serve/r6admit/admit.json +b740a60e6a116ce58f63b491531028227248850cc97d35285c218aba2ba14b2a tp3/serve/r6admit/client.log +c5544e9e34a6be02568600779d2926705605a8c5aeb59aa7f1bfdfdefd057c6e tp3/serve/r6admit/hostwatch-rank0.log +efab252857c21e6055946abfe9d8ffeb7c3d839de1b93fa1191b03303a14574b tp3/serve/r6admit/hostwatch-rank1.log +132cdbddc3abb774a5751df5cc43de8bdb6af52888d5068b15f5c884c9e64ab1 tp3/serve/r6admit/hostwatch-rank2.log +25bf2f912a29062e3a3e4ea401eea67e77195a4d0365eb0c684d882f7bc5789b tp3/serve/r6admit/server-rank0.log +edec660aeabe17a9bc2d893e649005186406846a51e8bcb90e7858c6fc8c8573 tp3/serve/r6admit/server-rank1.log +14edf8f8e1b885bb74d72f6e666a61050bd97d41318dfcc874a0e2296667640b tp3/serve/r6admit/server-rank2.log +0462b0f8f27287925da12ebdc625e41ffd4cd502d1ffd16d3ad11cffeff5e115 tp3/serve/r6all/client.log +1fb023123eadcb609d6723933ef9650e155310e94ad0560d1ffd616423a8e7e5 tp3/serve/r6all/conc.json +f932d7b3e290a00759429e309732f4fbe444f5c4513d28d3de1cfca296fefd41 tp3/serve/r6all/hostwatch-rank0.log +59c69afb6fe96cb1c640795e14341935551bd13aba38f0de93d1331cd81dabd2 tp3/serve/r6all/hostwatch-rank1.log +5761c732f2f12e16f43e15e44800c534b4413c596e2e45e476be92f6932a77ed tp3/serve/r6all/hostwatch-rank2.log +d3fe2b29a7b569dd9e5218f1e55eec04d31997b06c9b36bd44a2d5699f5f116d tp3/serve/r6all/server-rank0.log +12bf10e607679f5c323bd226df208025f08273d89fb734087b9042950ca0fe16 tp3/serve/r6all/server-rank1.log +50f115baeae91ab23d547935843d19baf251d251fda2fc19145cba8946e2238e tp3/serve/r6all/server-rank2.log +e323e896eac2fd573120785fbd3bc2329370135aaa611556a8ec24e403ff9c6b tp3/serve/r6auto/client.log +a03527822d42c1c3815d8e2293b044ea567bcc7b7278f9d64b3365b1668dff00 tp3/serve/r6auto/cold.json +f9abe17ca6002c4fa1030db44ba6bd0c43944e3176eea30b3d8bfb9a49fbd09b tp3/serve/r6auto/conc.json +97e95f444a7e1afb7b624aeba2f6e824e0660fcd43ad192be7a8cf82bcdf9246 tp3/serve/r6auto/hostwatch-rank0.log +15c4e5b89a68622f48cc2116ed08efe7d81b01a60f81701c57bbccfcb9a6a289 tp3/serve/r6auto/hostwatch-rank1.log +4b61df4e84ac6613144e39240ac9707a416535dc75689d0cd827a49e7b7049c4 tp3/serve/r6auto/hostwatch-rank2.log +5b7bf2a0360425b99d5c54a36f057557bb67177530309c50b3150723b5a3102b tp3/serve/r6auto/long-beside-short.json +dd40046108770e7a33f4ec67d7187f87be9ae4251fccb35d05cace20dc945509 tp3/serve/r6auto/rows-without-bodies.jsonl +f19ed8069314400e92b719a8bcc810d3e3b4f436b73823f62127334c8fc6e5fc tp3/serve/r6auto/server-rank0.log +0724cb273e8726ecfa95ad9930e2bd71428b1fc96d54c6014db66fd7ab04824e tp3/serve/r6auto/server-rank1.log +4abf6180eca5a22a2431ca92f0d117f9bf68e32f91728c222fa2981117cca18c tp3/serve/r6auto/server-rank2.log +c8a5068c7ae7ce0d9352c9bd70e270d524fb25d1771c5d52087d9c22f1941416 tp3/serve/r6off/client.log +49be40f981eed8159466ce10c585c65d8d454d2eb1c22cc21f0148ef65238f45 tp3/serve/r6off/compare-auto.log +a5183eddb01a2c6f0841279ed0f95326b06e4281545f2f59065c0e91f77f2212 tp3/serve/r6off/fresh.jsonl +25eece414a8554ae94c271704f2f734bf6c47dc8b4e39967292d43cb5c51ff72 tp3/serve/r6off/hostwatch-rank0.log +68983ccfbd86bc825a56d317aa32ca04bda2294342f43c15da03fd8df6bb0b57 tp3/serve/r6off/hostwatch-rank1.log +e45e9903e667b12d393ff900a2902c1edc15ef1d3801550ffe42d2adfa1d749b tp3/serve/r6off/hostwatch-rank2.log +9933e6889d1640f0d08d797d746079991ac1734cdac5f1e28cb8957ec695bc0c tp3/serve/r6off/server-rank0.log +48f253aae75831f17fdb16247bf970aa3dd6b34638f0c868d0ecefc67f0d9aa4 tp3/serve/r6off/server-rank1.log +08f61577d6c2a408e6478475a85068245a0ec03750a1fe7730ad62216388174d tp3/serve/r6off/server-rank2.log diff --git a/docs/receipts/glm-kv-pages-8e705835/binaries.txt b/docs/receipts/glm-kv-pages-8e705835/binaries.txt new file mode 100644 index 0000000000..eb7bd844d1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/binaries.txt @@ -0,0 +1,11 @@ +bin-8e705835: 8e705835 built with Zig 0.17.0 and nvcc 13.3 (V13.3.73) for sm_121; TP=2's ranks and the checks +0c7727776fda6e2838ce71bdde2cc4192cec7ef30b073634b529bb21e4bf84d2 tensorfold-native +0c7727776fda6e2838ce71bdde2cc4192cec7ef30b073634b529bb21e4bf84d2 tensorfold-native-r9 +15eb214d024fafbcf1b102b436672f1dfcda0a3e21b77f7e5165a42cd91bc791 tf-cuda-test + +bin-f8d254c5: main f8d254c5 built the same way; the replay on main and the A/B's main serves +d9181ac075f36ee5c118ef0b38513f6ebf61ab9fba312dbe592f2c434bd02a39 tensorfold-native + +bin-10698886: 8e705835 with tp3/ring.patch above it, built the same way; TP=3's ranks and check +eed66a8b83e10eb3190fa7b86babb8881b4390a9158904e39e5bbce3dc5ec18e tensorfold-native +6530f1e42385bf43e015a6db68abdbe9fc7f1f84b1aef3a9b16f5f63a1384418 tf-cuda-test diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/flashnext-served.log b/docs/receipts/glm-kv-pages-8e705835/ci/flashnext-served.log new file mode 100644 index 0000000000..61474265fe --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/flashnext-served.log @@ -0,0 +1,5 @@ +........ +---------------------------------------------------------------------- +Ran 8 tests in 0.062s + +OK diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/fuzz-cortheon.log b/docs/receipts/glm-kv-pages-8e705835/ci/fuzz-cortheon.log new file mode 100644 index 0000000000..4ea8380438 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/fuzz-cortheon.log @@ -0,0 +1,4 @@ +Build Summary: 3/3 steps succeeded; 14/14 tests passed +fuzz-cortheon success ++- run test 14 pass (14 total) 18s MaxRSS:7M + +- compile test safe native success 27s MaxRSS:483M diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/glm-host.log b/docs/receipts/glm-kv-pages-8e705835/ci/glm-host.log new file mode 100644 index 0000000000..74d9bdb308 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/glm-host.log @@ -0,0 +1,418 @@ +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0xeb20131d --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test 0 pass, 1 fail (1 total) +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-r9c/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql(u8, &first, &changed)); + ^ +failed command: ./.zig-cache/o/50d121a9ab9d688f31bd98df2c908ad4/test --cache-dir=./.zig-cache --seed=0xeb20131d --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/ba389278848dd62bdeff7f7a7b458416/test --cache-dir=./.zig-cache --seed=0xeb20131d --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/8a659e07ef41ab76477b7d9e749b71ca/test --cache-dir=./.zig-cache --seed=0xeb20131d --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 72/73 tests passed (1 failed) +test-cuda-host transitive failure ++- run test cached +| +- compile test debug native cached 192ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 145ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 173ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 696ms MaxRSS:39M ++- run exe runtime-discovery-exit success 23ms +| +- compile exe runtime-discovery-exit debug native cached 104ms MaxRSS:39M +| +- options cached +| +- compile lib runtime-discovery-mock debug native cached 112ms MaxRSS:38M ++- run exe rpc-warning-check success 18ms +| +- compile exe rpc-warning-check debug native cached 462ms MaxRSS:39M +| +- options cached +| +- compile lib rpc-mock-complete debug native cached 461ms MaxRSS:39M +| +- compile lib rpc-mock-abi debug native cached 337ms MaxRSS:39M +| +- compile lib rpc-mock-wait debug native cached 338ms MaxRSS:39M +| +- compile lib rpc-mock-store debug native cached 563ms MaxRSS:39M +| +- compile lib rpc-mock-wrong debug native cached 337ms MaxRSS:38M ++- run test cached +| +- compile test debug native cached 463ms MaxRSS:39M +| +- options (+5 more reused dependencies) ++- run test cached +| +- compile test debug native cached 349ms MaxRSS:39M ++- run test 30 pass (30 total) 61ms MaxRSS:10M +| +- compile test debug native cached 187ms MaxRSS:39M +| +- WriteFile h3-compile-only.fatbin cached +| +- options cached ++- run test cached +| +- compile test debug native cached 286ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 157ms MaxRSS:40M +| +- WriteFile attn_train.fatbin cached +| +- WriteFile dsv41_exl3_experts.fatbin cached +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- options cached +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- options cached +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) ++- run test cached +| +- compile test debug native cached 1s MaxRSS:40M +| +- options cached +| | +- compile exe qwen-round-timing-probe debug native cached 814ms MaxRSS:40M +| | +- WriteFile draft_ids.txt cached +| | +- options (reused) +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 403ms MaxRSS:40M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 425ms MaxRSS:39M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run test cached +| +- compile test debug native cached 302ms MaxRSS:39M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run test cached +| +- compile test debug native cached 101ms MaxRSS:39M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 524ms MaxRSS:39M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- compile obj exl3_host_compile debug native cached 115ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 235ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 126ms MaxRSS:38M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 293ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 115ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 173ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 641ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 204ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 108ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 514ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 1s MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 204ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test fast native cached 233ms MaxRSS:40M +| +- options (reused) ++- run test cached +| +- compile test fast native cached 679ms MaxRSS:41M +| +- options (reused) ++- compile obj bolt_host_compile debug x86_64-linux-gnu cached 16ms MaxRSS:39M +| +- options cached +| +- WriteFile draft_ids.txt (reused) ++- run test 26 pass (26 total) 3s MaxRSS:8M +| +- compile test debug native cached 126ms MaxRSS:38M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 497ms MaxRSS:40M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 401ms MaxRSS:39M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run exe qwen-cost-key-process success 1m +| +- compile exe qwen-cost-key-process debug native cached 238ms MaxRSS:40M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- WriteFile target/config.json cached ++- run test 0 pass, 1 fail (1 total) +| +- compile test debug native cached 628ms MaxRSS:39M ++- compile obj drama_cuda_native_host debug native cached 286ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 644ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 124ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 521ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 626ms MaxRSS:40M ++- run test cached +| +- compile test debug native cached 165ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 530ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 146ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 1s MaxRSS:39M ++- run test cached +| +- compile test debug native cached 104ms MaxRSS:39M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 132ms MaxRSS:39M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test cached +| +- compile test debug native cached 173ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 117ms MaxRSS:40M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 255ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 485ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 107ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 81ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 172ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 536ms MaxRSS:40M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run test cached +| +- compile test debug native cached 657ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 213ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 271ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 238ms MaxRSS:40M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 242ms MaxRSS:40M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 301ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 273ms MaxRSS:39M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 281ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 195ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 448ms MaxRSS:39M ++- run test cached +| +- compile test debug native cached 209ms MaxRSS:39M ++- run test cached + +- compile test debug native cached 497ms MaxRSS:40M + +- options cached + +- options (reused) + +- WriteFile draft_ids.txt (reused) + +- WriteFile exlr8_alias.zig cached +test-kv-pool success ++- run test cached +| +- compile test debug native cached 123ms MaxRSS:39M ++- run test cached + +- compile test debug native cached 167ms MaxRSS:39M +test-glm-snapshots-host success ++- run test 16 pass (16 total) 6s MaxRSS:65M + +- compile test debug native success 2s MaxRSS:184M + +- options (reused) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/lean.txt b/docs/receipts/glm-kv-pages-8e705835/ci/lean.txt new file mode 100644 index 0000000000..3666d6027b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/lean.txt @@ -0,0 +1,439 @@ +zig/build/hip.zig:1: comment past 120 columns +zig/kernels/cuda/experts.cu:1: comment past 120 columns +zig/kernels/cuda/experts_pack.cu:1: comment past 120 columns +zig/kernels/cuda/experts_prefill.cu:1: comment past 120 columns +zig/kernels/cuda/gdn.cu:1: comment past 120 columns +zig/kernels/cuda/qmm_frag.cuh:1: comment past 120 columns +zig/kernels/cuda/qmm_group.cu:1: comment past 120 columns +zig/kernels/cuda/qmm_prefill.cu:1: comment past 120 columns +zig/kernels/hip/affine.hip:1: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:1: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:368: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:473: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:503: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:521: block comment +zig/kernels/metal/core/hc_boundary.metal:5: comment past 120 columns +zig/kernels/metal/core/hc_boundary.metal:13: comment past 120 columns +zig/kernels/metal/core/hc_boundary.metal:82: comment past 120 columns +zig/kernels/metal/core/hc_boundary.metal:109: comment past 120 columns +zig/kernels/metal/core/moe_route.metal:1: comment past 120 columns +zig/kernels/metal/core/moe_route.metal:11: comment past 120 columns +zig/kernels/metal/core/moe_route.metal:68: comment past 120 columns +zig/kernels/metal/core/row_projection.metal:8: comment past 120 columns +zig/kernels/metal/core/row_projection.metal:72: comment past 120 columns +zig/kernels/metal/core/row_projection.metal:161: comment past 120 columns +zig/kernels/metal/decode/fn_gdn.metal:1: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:2: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:3: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:4: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:5: multi-line comment +zig/kernels/metal/glm_absorb_nax.metal:1: comment past 120 columns +zig/kernels/metal/glm_absorb_nax.metal:13: comment past 120 columns +zig/kernels/metal/glm_attn.metal:1: comment past 120 columns +zig/kernels/metal/glm_attn.metal:12: comment past 120 columns +zig/kernels/metal/glm_attn.metal:54: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:1: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:27: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:107: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:116: comment past 120 columns +zig/kernels/metal/glm_kda_prompt.metal:2: comment past 120 columns +zig/kernels/metal/glm_kda_prompt.metal:4: comment past 120 columns +zig/kernels/metal/glm_kda_prompt.metal:65: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:1: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:41: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:170: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:201: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:235: comment past 120 columns +zig/kernels/metal/kimi/dense.metal:10: comment past 120 columns +zig/kernels/metal/kimi/experts.metal:86: comment past 120 columns +zig/kernels/metal/nemotron_experts.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_experts.metal:97: comment past 120 columns +zig/kernels/metal/nemotron_glue.metal:172: comment past 120 columns +zig/kernels/metal/nemotron_head.metal:20: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:7: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:189: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:298: comment past 120 columns +zig/kernels/metal/nemotron_sample.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_sample.metal:145: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:8: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:134: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:149: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:313: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:1: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:28: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:64: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:198: comment past 120 columns +zig/kernels/metal/prefill/fn_prompt.metal:1: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:2: comment past 120 columns +zig/kernels/metal/prefill/fn_prompt.metal:128: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:137: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:200: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:201: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:253: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:254: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:255: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:318: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:319: multi-line comment +zig/kernels/metal/prefill/fn_select.metal:1: comment past 120 columns +zig/kernels/metal/prefill/fn_select.metal:14: comment past 120 columns +zig/kernels/metal/prefill/fn_select.metal:39: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:1: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:44: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:117: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:159: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:1: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:47: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:111: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:271: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:311: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:334: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax_b.metal:102: comment past 120 columns +zig/kernels/metal/prefill/sort.metal:52: comment past 120 columns +zig/kernels/metal/sources.zig:57: multi-line comment +zig/src/cli/cuda_segments.zig:1: comment past 120 columns +zig/src/cli/cuda_segments.zig:22: comment past 120 columns +zig/src/cli/info.zig:13: multi-line comment +zig/src/cli/pull.zig:232: multi-line comment +zig/src/cluster/admission.zig:1: comment past 120 columns +zig/src/cluster/admission_test.zig:1: comment past 120 columns +zig/src/cluster/backend.zig:1: comment past 120 columns +zig/src/cluster/backend.zig:11: comment past 120 columns +zig/src/cluster/backend.zig:101: comment past 120 columns +zig/src/cluster/backend_test.zig:1: comment past 120 columns +zig/src/cluster/budget.zig:1: comment past 120 columns +zig/src/cluster/canon.zig:1: comment past 120 columns +zig/src/cluster/canon.zig:14: comment past 120 columns +zig/src/cluster/canon.zig:75: comment past 120 columns +zig/src/cluster/cli_model.zig:98: comment past 120 columns +zig/src/cluster/cluster.zig:1: comment past 120 columns +zig/src/cluster/config.zig:1: comment past 120 columns +zig/src/cluster/cost.zig:1: comment past 120 columns +zig/src/cluster/estimate.zig:1: comment past 120 columns +zig/src/cluster/exchange.zig:1: comment past 120 columns +zig/src/cluster/exchange.zig:179: comment past 120 columns +zig/src/cluster/fabric_link.zig:1: comment past 120 columns +zig/src/cluster/fabric_link_test.zig:1: comment past 120 columns +zig/src/cluster/launch.zig:1: comment past 120 columns +zig/src/cluster/loader.zig:1: comment past 120 columns +zig/src/cluster/loader.zig:55: comment past 120 columns +zig/src/cluster/loader_test.zig:1: comment past 120 columns +zig/src/cluster/membership.zig:1: comment past 120 columns +zig/src/cluster/membership_test.zig:1: comment past 120 columns +zig/src/cluster/node.zig:83: comment past 120 columns +zig/src/cluster/plan.zig:1: comment past 120 columns +zig/src/cluster/plan.zig:12: comment past 120 columns +zig/src/cluster/plan.zig:15: comment past 120 columns +zig/src/cluster/plan.zig:116: comment past 120 columns +zig/src/cluster/plan.zig:176: comment past 120 columns +zig/src/cluster/probe_fixtures.zig:1: comment past 120 columns +zig/src/cluster/pull.zig:1: comment past 120 columns +zig/src/cluster/pull_test.zig:1: comment past 120 columns +zig/src/cluster/round.zig:1: comment past 120 columns +zig/src/cluster/status.zig:1: comment past 120 columns +zig/src/cluster/toy.zig:1: comment past 120 columns +zig/src/cluster/toy.zig:93: comment past 120 columns +zig/src/cluster/toy.zig:193: comment past 120 columns +zig/src/cluster/traffic.zig:1: comment past 120 columns +zig/src/cluster/traffic.zig:18: comment past 120 columns +zig/src/cluster/traffic.zig:38: comment past 120 columns +zig/src/cluster/traffic.zig:49: comment past 120 columns +zig/src/cluster/wire.zig:1: comment past 120 columns +zig/src/core/affine_mm.zig:1: comment past 120 columns +zig/src/core/affine_mm.zig:25: comment past 120 columns +zig/src/core/affine_mm.zig:33: comment past 120 columns +zig/src/core/affine_mm.zig:54: comment past 120 columns +zig/src/core/affine_mm.zig:60: comment past 120 columns +zig/src/core/affine_mm.zig:75: comment past 120 columns +zig/src/core/affine_mm.zig:99: comment past 120 columns +zig/src/core/checkpoint.zig:1: comment past 120 columns +zig/src/core/checkpoint.zig:11: comment past 120 columns +zig/src/core/checkpoint_metal.zig:65: comment past 120 columns +zig/src/core/checkpoint_metal.zig:178: comment past 120 columns +zig/src/core/copy_index.zig:1: comment past 120 columns +zig/src/core/hc.zig:1: comment past 120 columns +zig/src/core/hc.zig:30: comment past 120 columns +zig/src/core/lanes/fill.zig:26: comment past 120 columns +zig/src/core/lanes/fill.zig:54: comment past 120 columns +zig/src/core/lanes/gpu_full.zig:40: comment past 120 columns +zig/src/core/lanes/gpu_full.zig:114: comment past 120 columns +zig/src/core/lanes/gpu_full_test.zig:1: comment past 120 columns +zig/src/core/moe_route.zig:1: comment past 120 columns +zig/src/core/moe_route.zig:33: comment past 120 columns +zig/src/core/paced_read.zig:5: comment past 120 columns +zig/src/core/prompt_cache.zig:26: comment past 120 columns +zig/src/core/prompt_cache.zig:176: comment past 120 columns +zig/src/core/prompt_cache.zig:300: comment past 120 columns +zig/src/core/prompt_cache.zig:349: comment past 120 columns +zig/src/core/prompt_cache_test.zig:1: comment past 120 columns +zig/src/core/row_projection.zig:21: comment past 120 columns +zig/src/core/row_projection.zig:24: comment past 120 columns +zig/src/core/row_projection.zig:96: comment past 120 columns +zig/src/core/safetensors.zig:64: comment past 120 columns +zig/src/core/segments.zig:1: comment past 120 columns +zig/src/core/segments.zig:26: comment past 120 columns +zig/src/core/segments.zig:61: comment past 120 columns +zig/src/core/stagger.zig:7: comment past 120 columns +zig/src/core/stagger.zig:22: comment past 120 columns +zig/src/core/stagger.zig:144: comment past 120 columns +zig/src/core/template/template.zig:103: comment past 120 columns +zig/src/core/tokenizer/regex.zig:1: comment past 120 columns +zig/src/core/tokenizer/regex.zig:272: comment past 120 columns +zig/src/core/tokenizer/tokenizer.zig:1: comment past 120 columns +zig/src/core/tokenizer/unicode.zig:224: comment past 120 columns +zig/src/cuda/cublaslt.zig:1: comment past 120 columns +zig/src/cuda/segments.zig:142: comment past 120 columns +zig/src/cuda/triton.zig:72: comment past 120 columns +zig/src/fabric/collective.zig:1: comment past 120 columns +zig/src/fabric/collective.zig:130: comment past 120 columns +zig/src/fabric/collective.zig:183: comment past 120 columns +zig/src/fabric/cost.zig:34: comment past 120 columns +zig/src/fabric/cost.zig:43: comment past 120 columns +zig/src/fabric/link_fake.zig:8: comment past 120 columns +zig/src/fabric/mailbox.zig:59: comment past 120 columns +zig/src/fabric/manifest.zig:1: comment past 120 columns +zig/src/fabric/mcdma.zig:160: comment past 120 columns +zig/src/fabric/packet.zig:1: comment past 120 columns +zig/src/fabric/prefill_source.zig:18: comment past 120 columns +zig/src/fabric/verbs_abi.zig:1: comment past 120 columns +zig/src/fabric/verbs_link.zig:1: comment past 120 columns +zig/src/fabric/words.zig:142: comment past 120 columns +zig/src/families/flashnext/call_log.zig:1: comment past 120 columns +zig/src/families/flashnext/follow.zig:51: multi-line comment +zig/src/families/flashnext/follow.zig:51: comment past 120 columns +zig/src/families/flashnext/follow.zig:136: comment past 120 columns +zig/src/families/flashnext/follow.zig:147: comment past 120 columns +zig/src/families/flashnext/follow.zig:150: comment past 120 columns +zig/src/families/flashnext/gdn.zig:1: multi-line comment +zig/src/families/flashnext/gdn.zig:13: multi-line comment +zig/src/families/flashnext/gdn.zig:32: multi-line comment +zig/src/families/flashnext/marks.zig:1: comment past 120 columns +zig/src/families/flashnext/marks.zig:13: comment past 120 columns +zig/src/families/flashnext/marks.zig:40: comment past 120 columns +zig/src/families/flashnext/marks.zig:62: comment past 120 columns +zig/src/families/flashnext/marks.zig:72: comment past 120 columns +zig/src/families/flashnext/marks.zig:103: comment past 120 columns +zig/src/families/flashnext/marks.zig:121: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:17: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:20: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:25: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:49: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:108: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:155: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:205: comment past 120 columns +zig/src/families/flashnext/split.zig:1: comment past 120 columns +zig/src/families/flashnext/split.zig:8: comment past 120 columns +zig/src/families/flashnext/split.zig:62: comment past 120 columns +zig/src/families/flashnext/tp.zig:19: comment past 120 columns +zig/src/families/flashnext/tp.zig:216: multi-line comment +zig/src/families/flashnext/tp.zig:216: comment past 120 columns +zig/src/families/flashnext/tp.zig:247: multi-line comment +zig/src/families/flashnext/tp.zig:263: multi-line comment +zig/src/families/flashnext/tp.zig:283: comment past 120 columns +zig/src/families/flashnext/tp.zig:319: multi-line comment +zig/src/families/flashnext/tp.zig:320: multi-line comment +zig/src/families/flashnext/tp.zig:341: comment past 120 columns +zig/src/families/flashnext/tp.zig:371: comment past 120 columns +zig/src/families/flashnext/tp.zig:392: comment past 120 columns +zig/src/families/flashnext/tp.zig:407: comment past 120 columns +zig/src/families/flashnext/tp.zig:449: multi-line comment +zig/src/families/flashnext/tp.zig:450: multi-line comment +zig/src/families/flashnext/tp_settings.zig:8: comment past 120 columns +zig/src/families/flashnext/tp_settings.zig:39: comment past 120 columns +zig/src/families/glm/attn_check.zig:1: comment past 120 columns +zig/src/families/glm/attn_check.zig:127: comment past 120 columns +zig/src/families/glm/backend.zig:178: comment past 120 columns +zig/src/families/glm/backend.zig:326: comment past 120 columns +zig/src/families/glm/checks.zig:37: comment past 120 columns +zig/src/families/glm/checks.zig:72: comment past 120 columns +zig/src/families/glm/checks.zig:105: comment past 120 columns +zig/src/families/glm/checks.zig:153: comment past 120 columns +zig/src/families/glm/checks.zig:190: comment past 120 columns +zig/src/families/glm/config.zig:222: comment past 120 columns +zig/src/families/glm/engine.zig:1: comment past 120 columns +zig/src/families/glm/engine.zig:42: comment past 120 columns +zig/src/families/glm/engine.zig:105: comment past 120 columns +zig/src/families/glm/engine.zig:238: comment past 120 columns +zig/src/families/glm/engine.zig:455: comment past 120 columns +zig/src/families/glm/ep.zig:1: comment past 120 columns +zig/src/families/glm/ep.zig:25: comment past 120 columns +zig/src/families/glm/ep.zig:50: comment past 120 columns +zig/src/families/glm/ep.zig:204: comment past 120 columns +zig/src/families/glm/ep.zig:218: comment past 120 columns +zig/src/families/glm/ep.zig:309: comment past 120 columns +zig/src/families/glm/ep_control.zig:1: comment past 120 columns +zig/src/families/glm/ep_control.zig:120: comment past 120 columns +zig/src/families/glm/forward.zig:1: comment past 120 columns +zig/src/families/glm/forward.zig:197: comment past 120 columns +zig/src/families/glm/forward.zig:218: comment past 120 columns +zig/src/families/glm/forward.zig:286: comment past 120 columns +zig/src/families/glm/forward.zig:330: comment past 120 columns +zig/src/families/glm/forward.zig:393: comment past 120 columns +zig/src/families/glm/forward.zig:406: comment past 120 columns +zig/src/families/glm/forward.zig:536: comment past 120 columns +zig/src/families/glm/kept.zig:29: comment past 120 columns +zig/src/families/glm/kept.zig:81: comment past 120 columns +zig/src/families/glm/kept.zig:105: comment past 120 columns +zig/src/families/glm/kept.zig:218: comment past 120 columns +zig/src/families/glm/kernels.zig:1: comment past 120 columns +zig/src/families/glm/load_plan.zig:1: comment past 120 columns +zig/src/families/glm/load_plan.zig:99: comment past 120 columns +zig/src/families/glm/moe.zig:18: comment past 120 columns +zig/src/families/glm/moe.zig:150: comment past 120 columns +zig/src/families/glm/mtp.zig:1: comment past 120 columns +zig/src/families/glm/mtp.zig:17: comment past 120 columns +zig/src/families/glm/plain_rounds.zig:14: comment past 120 columns +zig/src/families/glm/plain_rounds.zig:53: comment past 120 columns +zig/src/families/glm/prompt.zig:1: comment past 120 columns +zig/src/families/glm/prompt.zig:21: comment past 120 columns +zig/src/families/glm/prompt.zig:134: comment past 120 columns +zig/src/families/glm/prompt.zig:156: comment past 120 columns +zig/src/families/glm/prompt.zig:218: comment past 120 columns +zig/src/families/glm/prompt.zig:246: comment past 120 columns +zig/src/families/glm/prompt.zig:342: comment past 120 columns +zig/src/families/glm/prompt.zig:393: comment past 120 columns +zig/src/families/glm/prompt.zig:417: comment past 120 columns +zig/src/families/glm/prompt.zig:432: comment past 120 columns +zig/src/families/glm/prompt.zig:553: comment past 120 columns +zig/src/families/glm/snapshot.zig:1: comment past 120 columns +zig/src/families/glm/snapshot.zig:59: comment past 120 columns +zig/src/families/glm/snapshot.zig:91: comment past 120 columns +zig/src/families/glm/weights.zig:1: comment past 120 columns +zig/src/families/kimi_k3/kernels.zig:283: comment past 120 columns +zig/src/families/kimi_k3/prepare.zig:1: comment past 120 columns +zig/src/families/nemotron/backend.zig:301: comment past 120 columns +zig/src/families/nemotron/backend.zig:421: comment past 120 columns +zig/src/families/nemotron/copy_lanes.zig:1: comment past 120 columns +zig/src/families/nemotron/copy_lanes.zig:66: comment past 120 columns +zig/src/families/nemotron/cuda_mtp.zig:372: comment past 120 columns +zig/src/families/nemotron/cuda_segments.zig:1: comment past 120 columns +zig/src/families/nemotron/cuda_segments.zig:64: comment past 120 columns +zig/src/families/nemotron/cuda_triton.zig:109: comment past 120 columns +zig/src/families/nemotron/cuda_triton.zig:116: comment past 120 columns +zig/src/families/nemotron/forward.zig:35: comment past 120 columns +zig/src/families/nemotron/head_block.zig:21: comment past 120 columns +zig/src/families/nemotron/layers.zig:61: comment past 120 columns +zig/src/families/nemotron/mtp.zig:193: comment past 120 columns +zig/src/families/nemotron/prefill_launch.zig:59: comment past 120 columns +zig/src/families/nemotron/prefill_launch.zig:137: comment past 120 columns +zig/src/families/nemotron/prefill_mixers.zig:1: comment past 120 columns +zig/src/families/nemotron/round_depth.zig:1: comment past 120 columns +zig/src/families/nemotron/simd_attention.zig:1: comment past 120 columns +zig/src/families/nemotron/simd_attention.zig:5: comment past 120 columns +zig/src/families/nemotron/simd_attention.zig:88: comment past 120 columns +zig/src/families/qwen3_5/config.zig:1: comment past 120 columns +zig/src/families/qwen3_5/qualify.zig:1: comment past 120 columns +zig/src/families/qwen3_5/weights.zig:1: comment past 120 columns +zig/src/hip/device_arch.c:1: block comment +zig/src/hip/driver.zig:1: comment past 120 columns +zig/src/hip/memory.zig:1: comment past 120 columns +zig/src/main.zig:226: comment past 120 columns +zig/src/native/flashnext_host.zig:445: comment past 120 columns +zig/src/native/metal.zig:318: comment past 120 columns +zig/src/server/chunk_plan.zig:1: comment past 120 columns +zig/src/server/hf_text.zig:23: comment past 120 columns +zig/src/server/log.zig:106: comment past 120 columns +zig/src/server/messages.zig:25: comment past 120 columns +zig/src/server/messages.zig:151: comment past 120 columns +zig/src/server/metrics.zig:325: multi-line comment +zig/src/server/model_text.zig:103: comment past 120 columns +zig/src/server/tool_params.zig:120: comment past 120 columns +zig/src/server/tool_parse.zig:1: comment past 120 columns +zig/src/server/tool_stream.zig:417: comment past 120 columns +zig/src/server/tool_streaming.zig:1: comment past 120 columns +zig/tests/cuda/bench.zig:17: comment past 120 columns +zig/tests/cuda/box/run.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/box/build.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/box/capture.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/box/zrun.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/capture.py:2: docstring past 120 columns +zig/tests/cuda/nemotron/test_aot_manifest.py:2: multi-line docstring +zig/tests/cuda/oracle/oracle.py:1: docstring past 120 columns +zig/tests/cuda/oracle/oracle.py:82: docstring past 120 columns +zig/tests/cuda/oracle/sass_compare.py:1: docstring past 120 columns +zig/tests/cuda/runtime_tests.zig:276: comment past 120 columns +zig/tests/dispatch_bench.zig:1: comment past 120 columns +zig/tests/fabric_handoff.zig:1: comment past 120 columns +zig/tests/fabric_library.zig:1: comment past 120 columns +zig/tests/fabric_vectors.zig:1: comment past 120 columns +zig/tests/flashnext_bench.zig:69: comment past 120 columns +zig/tests/flashnext_bench.zig:119: multi-line comment +zig/tests/flashnext_bench.zig:169: multi-line comment +zig/tests/flashnext_reuse.zig:93: comment past 120 columns +zig/tests/flashnext_reuse.zig:183: comment past 120 columns +zig/tests/flashnext_run.zig: 1690 lines (split past 600) +zig/tests/flashnext_run.zig:1: multi-line comment +zig/tests/flashnext_run.zig:2: multi-line comment +zig/tests/flashnext_run.zig:762: multi-line comment +zig/tests/flashnext_run.zig:828: multi-line comment +zig/tests/flashnext_run.zig:1308: multi-line comment +zig/tests/flashnext_run.zig:1393: multi-line comment +zig/tests/flashnext_run.zig:1441: multi-line comment +zig/tests/flashnext_run.zig:1442: multi-line comment +zig/tests/glm_run.zig:1: comment past 120 columns +zig/tests/grid_sync_bench.zig:1: comment past 120 columns +zig/tests/hip_mock.c:1: block comment +zig/tests/linktest_suite.zig:1: comment past 120 columns +zig/tests/mm_bench.zig:1: comment past 120 columns +zig/tests/moe_bench.zig:1: comment past 120 columns +zig/tests/nemotron_dense.zig:1: comment past 120 columns +zig/tests/nemotron_dense.zig:11: comment past 120 columns +zig/tests/nemotron_dense.zig:100: comment past 120 columns +zig/tests/nemotron_dense.zig:292: comment past 120 columns +zig/tests/nemotron_experts.zig:1: comment past 120 columns +zig/tests/nemotron_experts.zig:133: comment past 120 columns +zig/tests/nemotron_prefill_check.zig:1: comment past 120 columns +zig/tests/nemotron_sample_check.zig:1: comment past 120 columns +zig/tests/nemotron_wide.zig:1: comment past 120 columns +zig/tests/nemotron_wide.zig:244: comment past 120 columns +zig/tests/ordering.zig:1: comment past 120 columns +zig/tests/ordering.zig:29: comment past 120 columns +zig/tests/row_projection_check.zig:1: comment past 120 columns +zig/tests/row_projection_check.zig:19: comment past 120 columns +zig/tests/row_projection_check.zig:163: comment past 120 columns +zig/tests/server/build.sh:2: comment past 120 columns +zig/tests/server/freeze_cases.py:1: docstring past 120 columns +zig/tests/server/freeze_cases.py:150: comment past 120 columns +zig/tests/server/golden_check.zig:79: comment past 120 columns +zig/tests/server/golden_check.zig:173: comment past 120 columns +zig/tests/server/wire.py:52: docstring past 120 columns +zig/tests/server/zig_test.sh:2: comment past 120 columns +zig/tests/tp2_bench.zig:1: comment past 120 columns +zig/tests/wide_heads.zig:46: comment past 120 columns +zig/tests/wide_probe.zig:1: comment past 120 columns +zig/tests/wide_probe.zig:263: comment past 120 columns +zig/tests/wide_trees.zig:1: comment past 120 columns +zig/tests/wide_trees.zig:126: comment past 120 columns +zig/tests/wide_trees.zig:156: comment past 120 columns +tools/zig/check_prefill_layers.py:1: docstring past 120 columns +tools/zig/check_prefill_model.py:1: docstring past 120 columns +tools/zig/check_prefill_ops.py:1: docstring past 120 columns +tools/zig/flashnext_dump.py:1: multi-line docstring +tools/zig/flashnext_hc_coalesce.py:1: multi-line docstring +tools/zig/gen_glm_kernels.py:43: comment past 120 columns +tools/zig/glm_compare.py:1: docstring past 120 columns +tools/zig/glm_ref.py:1: docstring past 120 columns +tools/zig/glm_ref.py:96: docstring past 120 columns +tools/zig/glm_ref.py:122: docstring past 120 columns +tools/zig/oracle_nemotron_rows.py:38: docstring past 120 columns +tools/zig/prefill_dump.py:1: docstring past 120 columns +tools/zig/prefill_glue.py:1: docstring past 120 columns +tools/zig/prefill_kernels.py:34: docstring past 120 columns +tools/zig/prefill_launch.py:1: docstring past 120 columns +tools/zig/prefill_launch.py:173: docstring past 120 columns +tools/zig/prefill_swap.py:1: docstring past 120 columns +tools/zig/reuse_served.py:133: docstring past 120 columns +tools/zig/reuse_served.py:232: comment past 120 columns +tools/zig/reuse_served.py:254: docstring past 120 columns +tools/zig/reuse_served.py:290: docstring past 120 columns +tools/zig/reuse_served.py:375: multi-line docstring +tools/zig/reuse_served.py:401: docstring past 120 columns +tools/zig/reuse_served.py:423: docstring past 120 columns +tools/zig/reuse_served.py:440: docstring past 120 columns +tools/zig/test_fabric.sh:2: comment past 120 columns +tools/zig/test_lanes.sh:2: comment past 120 columns +tools/zig/tokenizer_parity.py:1: docstring past 120 columns +tools/zig/triton_aot_manifest.py:2: docstring past 120 columns +tools/zig/unicode_tables.py:1: docstring past 120 columns +3599 files, 438 problems diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/auto-at-two.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/auto-at-two.log new file mode 100644 index 0000000000..691d08b796 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/auto-at-two.log @@ -0,0 +1,94 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xb47a2d84 --listen=- + +test-cuda-host ++- run test 87 pass, 1 fail (88 total) +error: 'cuda_native.test.TF_KV_PAGES: auto pages from three ranks, off and 0 give windows, all and other values page like on' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x1270da9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_native.zig:374:9: 0x12e3a84 in test.TF_KV_PAGES: auto pages from three ranks, off and 0 give windows, all and other values page like on (cuda.zig) + try std.testing.expect(!try paging(null, w, false)); + ^ +failed command: ./.zig-cache/o/6ea58775dfbc40e9dc25299590ad0624/test --cache-dir=./.zig-cache --seed=0xb47a2d84 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/a86904f42eba2657d0e1a14b7a9f07a8/test --cache-dir=./.zig-cache --seed=0xb47a2d84 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0xb47a2d84 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/9a7efdcf6dea67fb10d1dfb9a878907a/test --cache-dir=./.zig-cache --seed=0xb47a2d84 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 260/261 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 87 pass, 1 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/bind-before-pages.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/bind-before-pages.log new file mode 100644 index 0000000000..40544e50ef --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/bind-before-pages.log @@ -0,0 +1,98 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/50271aedb58fefd8dfe4d418dc1b89d1/test --cache-dir=./.zig-cache --seed=0x8acf1e57 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0x8acf1e57 --listen=- + +test-cuda-host ++- run test 87 pass, 1 fail (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected 3, found 4 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1275e09 in expectEqualInner__func_954 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x13fee6d in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:210:5: 0x13fef15 in test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window (cuda.zig) + try expectEqual(live, F.live); // nothing allocated for it + ^ +failed command: ./.zig-cache/o/63f2e7a09bfb0bfc3ccf3693aaece1bb/test --cache-dir=./.zig-cache --seed=0x8acf1e57 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/24537ef161639000e9a463a0115fdc0d/test --cache-dir=./.zig-cache --seed=0x8acf1e57 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x8acf1e57 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 87 pass, 1 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/close-keeps-pages.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/close-keeps-pages.log new file mode 100644 index 0000000000..f3d34b0288 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/close-keeps-pages.log @@ -0,0 +1,111 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x4bf25543 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0x4bf25543 --listen=- + +test-cuda-host ++- run test 85 pass, 3 fail (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + /zt/kvp-mut/zig/src/core/kv_pool.zig:212:32: 0x13fb3b4 in promise (root.zig) + if (p.available() < n) return error.LaneMemory; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:139:9: 0x13f8660 in open (cuda.zig) + try p.pool.promise(l, n); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:126:56: 0x1554ae7 in test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back (cuda.zig) + try std.testing.expectEqualSlices(u32, &.{ 1, 0 }, try p.open(1, 600, 100)); // the last page given back, first + ^ +error: 'cuda_pages_test.test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x1270da9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:151:5: 0x1553697 in test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others (cuda.zig) + try expect(own.st.capacity == 0 and p.pool.idle() and p.pool.census().free == 8); + ^ +error: 'cuda_pages_test.test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x1270da9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:236:5: 0x13fd412 in test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not (cuda.zig) + try expect(ln.fits(1000, 100)); + ^ +failed command: ./.zig-cache/o/677bf8f1d93e2938fb878af4eed738f6/test --cache-dir=./.zig-cache --seed=0x4bf25543 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/633be87be374488bab20be4529d50134/test --cache-dir=./.zig-cache --seed=0x4bf25543 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/9a723d8a2a691f018b28aeb49857df5d/test --cache-dir=./.zig-cache --seed=0x4bf25543 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 257/260 tests passed (3 failed) +test-cuda-host transitive failure ++- run test 85 pass, 3 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fits-flag-dropped.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fits-flag-dropped.log new file mode 100644 index 0000000000..13e0fa6bbb --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fits-flag-dropped.log @@ -0,0 +1,71 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x8b90ea8e --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0x8b90ea8e --listen=- + +test-cuda-host ++- run test 72 pass, 1 fail (73 total) +error: 'cuda_kv_pool.test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x134a739 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:197:5: 0x18b1d58 in test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so (cuda.zig) + try std.testing.expect(!fits(Owned, loaded, o, floor + 3 * each + (4 << 30) - 1, 8192, 0)); + ^ +failed command: ./.zig-cache/o/3dbe0d8b8396e35f6d7192bd6dc9a842/test --cache-dir=./.zig-cache --seed=0x8b90ea8e --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 132/133 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 72 pass, 1 fail (73 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fits-own-free.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fits-own-free.log new file mode 100644 index 0000000000..683080e184 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fits-own-free.log @@ -0,0 +1,94 @@ +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x791ee95 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/ade6ac48df685f5fe805ced5ee114f6e/test --cache-dir=./.zig-cache --seed=0x791ee95 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test 87 pass, 1 fail (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x1270da9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:235:5: 0x13fd33b in test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not (cuda.zig) + try expect(!ln.fits(3000, 100)); // the engine's own is free, but only 3 pages are + ^ +failed command: ./.zig-cache/o/d0a8e0f306af60efb6e48ca807dd92db/test --cache-dir=./.zig-cache --seed=0x791ee95 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0x791ee95 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/17dc9e8651e737d8be094c313a004702/test --cache-dir=./.zig-cache --seed=0x791ee95 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 87 pass, 1 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fp8-kernel-table.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fp8-kernel-table.log new file mode 100644 index 0000000000..6887ca666f --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fp8-kernel-table.log @@ -0,0 +1,104 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test 87 pass, 1 fail (88 total) +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_kv8_test.test.GLM FP8 KV: an FP8 cache takes no page table, so its writes and reads refuse one before they launch' failed: + expected error.Fp8OnPages, found void + /.local/share/zig/0.17.0/lib/std/testing.zig:634:9: 0x159b93b in expectError__func_2273 (std.zig) + return error.TestExpectedError; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_kv8_test.zig:143:5: 0x159c71b in test.GLM FP8 KV: an FP8 cache takes no page table, so its writes and reads refuse one before they launch (cuda.zig) + try t.expectError(error.Fp8OnPages, latent.latentWriteFormat(set, stream, 100, 512, 200, 400, paged, 300, 3, 512)); + ^ +failed command: ./.zig-cache/o/efb910fa3bd0ecea1d36abc0e8087684/test --cache-dir=./.zig-cache --seed=0xf3d39b84 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xf3d39b84 --listen=- + +test-cuda-host ++- run test 11 pass, 1 fail (12 total) +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +error: 'cuda_kv8_test.test.GLM FP8 KV: an FP8 cache takes no page table, so its writes and reads refuse one before they launch' failed: + expected error.Fp8OnPages, found void + /.local/share/zig/0.17.0/lib/std/testing.zig:634:9: 0x127089b in expectError__func_1068 (std.zig) + return error.TestExpectedError; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_kv8_test.zig:143:5: 0x127167b in test.GLM FP8 KV: an FP8 cache takes no page table, so its writes and reads refuse one before they launch (cuda.zig) + try t.expectError(error.Fp8OnPages, latent.latentWriteFormat(set, stream, 100, 512, 200, 400, paged, 300, 3, 512)); + ^ +failed command: ./.zig-cache/o/44a83b526cb92156cf203cc509651417/test --cache-dir=./.zig-cache --seed=0xf3d39b84 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0xf3d39b84 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/b408ba0fed42c84384e4a96b798f513b/test --cache-dir=./.zig-cache --seed=0xf3d39b84 --listen=- + +Build Summary: 160/163 steps succeeded (2 failed); 258/260 tests passed (2 failed) +test-cuda-host transitive failure ++- run test 11 pass, 1 fail (12 total) ++- run test 87 pass, 1 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fp8-on-pages.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fp8-on-pages.log new file mode 100644 index 0000000000..153db37d62 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/fp8-on-pages.log @@ -0,0 +1,95 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test 87 pass, 1 fail (88 total) +error: 'cuda_native.test.TF_KV_PAGES with --kv-cache fp8: auto and off keep windows at every rank count, on and all are refused by name' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + expected error.Fp8OnPages, found true + /.local/share/zig/0.17.0/lib/std/testing.zig:634:9: 0x12e2563 in expectError__func_1183 (std.zig) + return error.TestExpectedError; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_native.zig:396:53: 0x12e294e in test.TF_KV_PAGES with --kv-cache fp8: auto and off keep windows at every rank count, on and all are refused by name (cuda.zig) + for ([_][]const u8{ "on", "1", "all" }) |v| try std.testing.expectError(error.Fp8OnPages, paging(v, w, true)); + ^ +failed command: ./.zig-cache/o/926d1afbfc6d4c32420f3791af06f728/test --cache-dir=./.zig-cache --seed=0x61a53e8b --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/02640b643fe1497f991b263f2d2fd298/test --cache-dir=./.zig-cache --seed=0x61a53e8b --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0x61a53e8b --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/44d76c1aa06bd5e86609369fe9812ad0/test --cache-dir=./.zig-cache --seed=0x61a53e8b --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x61a53e8b --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 87 pass, 1 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/head.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/head.log new file mode 100644 index 0000000000..573292c69c --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/head.log @@ -0,0 +1,83 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x204829ed --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/46ab26c0f957a0f6c71df9e01db15533/test --cache-dir=./.zig-cache --seed=0x204829ed --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0x204829ed --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/b0c61eb08475fe745302286326f77b73/test --cache-dir=./.zig-cache --seed=0x204829ed --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/f1b7dd59cf21e84f4db4b59615b85a48/test --cache-dir=./.zig-cache --seed=0x204829ed --listen=- diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/holds-no-floor.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/holds-no-floor.log new file mode 100644 index 0000000000..af6b78e8cd --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/holds-no-floor.log @@ -0,0 +1,309 @@ +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/ecd2dc12c06efa85870e4f998fe3cc2e/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-kv-pool ++- run test 10 pass, 1 fail (11 total) +error: 'kv_pool_test.test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x1237889 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:97:5: 0x124c4c3 in test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest (kv_pool.zig) + try expect(!kv.holds(glm, .{ .room = room - 1, .engine = 0, .lanes = 4, .cache = 16 * g, .window = window, .engine_paged = true })); + ^ +failed command: ./.zig-cache/o/4995c7d53919f9bd88d7e3d07d95dc6c/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/1692dddd29db20325d9f760dc6345579/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test 351 pass, 1 fail (352 total) +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +error: 'kv_pool_test.test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x12cd979 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:97:5: 0x13a7fe3 in test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest (root.zig) + try expect(!kv.holds(glm, .{ .room = room - 1, .engine = 0, .lanes = 4, .cache = 16 * g, .window = window, .engine_paged = true })); + ^ +failed command: ./.zig-cache/o/8d3f18fca6122c2928f3ac89c85dbe3a/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/de86649483c7bd7be22b038413f982b0/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/4e5451292ae24df36b4a79d4192b80dd/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/708bca239af536b3e5b38bd37c0fffd9/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test 72 pass, 1 fail (73 total) +error: 'cuda_kv_pool.test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x134a739 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:197:5: 0x18b1d58 in test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so (cuda.zig) + try std.testing.expect(!fits(Owned, loaded, o, floor + 3 * each + (4 << 30) - 1, 8192, 0)); + ^ +failed command: ./.zig-cache/o/cd0c76a960fe8b0f793c6518acc35994/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/8ed093dfadbb7a412d5e48e632ba34bf/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/be1081222eb237b82db872920842fc90/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/23bdb3dcae00843d2d5d8e4137b25d87/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/3ae61573c8bf1bbc067c5126911b692b/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/573a4187ceb17971c6a976dee2a93470/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/7ec2846eb4fd8025e059abc078e298bb/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/4e4f78618dbf66930026ddb8fc7bdf23/test --cache-dir=./.zig-cache --seed=0x3e5db7e3 --listen=- + +Build Summary: 158/163 steps succeeded (3 failed); 1051/1058 tests passed (4 skipped, 3 failed) +test-cuda-host transitive failure ++- run test 351 pass, 1 fail (352 total) ++- run test 72 pass, 1 fail (73 total) +test-kv-pool transitive failure ++- run test 10 pass, 1 fail (11 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/kept-window-cache.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/kept-window-cache.log new file mode 100644 index 0000000000..05a1c8d2e8 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/kept-window-cache.log @@ -0,0 +1,305 @@ +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/f1b7dd59cf21e84f4db4b59615b85a48/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/46ab26c0f957a0f6c71df9e01db15533/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/1f63417bb0368e42ba6481d0a040a07b/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/da15d3c41ea5d2153781660b047e67e6/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/f9419dba1a4fe6ebd6e84eb94f7dcebf/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test 0 pass, 1 fail (1 total) +ectedResult; + ^ +/zt/kvp-mut/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql(u8, &first, &changed)); + ^ +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexp +failed command: ./.zig-cache/o/50d121a9ab9d688f31bd98df2c908ad4/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/f4ba094fe6dadc4538f8c6e38d953163/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +failed command: ./.zig-cache/o/92af39c21a90673092679820f4bb048b/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/300e9d9bc5cc2177d3c6380f0e8a7c36/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/e4ba30d29a5143ca4e05e92e95d45ebe/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test 72 pass, 1 fail (73 total) +error: 'cuda_kv_pool.test.kv pages: a family without snapshot slots keeps the paged plan's cache, not what window-sized streams leave' failed: + expected 17179869184, found 4294967296 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x18982b3 in expectEqualInner__func_3089 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x18bcc5f in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:169:5: 0x18bccb8 in test.kv pages: a family without snapshot slots keeps the paged plan's cache, not what window-sized streams leave (cuda.zig) + try std.testing.expectEqual(@as(u64, 16 << 30), k.copied); + ^ +failed command: ./.zig-cache/o/7b831bfbe58f6d66348149fa10e1b4e6/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/826bec6bc03435dcc4d3f85f6a062ce2/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/8a659e07ef41ab76477b7d9e749b71ca/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/24a9f3e50fc977a68a7fb2ada6585356/test --cache-dir=./.zig-cache --seed=0xb78465aa --listen=- + +Build Summary: 160/163 steps succeeded (2 failed); 1052/1058 tests passed (4 skipped, 2 failed) +test-cuda-host transitive failure ++- run test 0 pass, 1 fail (1 total) ++- run test 72 pass, 1 fail (73 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/map-any-id.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/map-any-id.log new file mode 100644 index 0000000000..360dfc519f --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/map-any-id.log @@ -0,0 +1,95 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/ab562d14c48e33660d7302ec9376bb8a/test --cache-dir=./.zig-cache --seed=0x8743759f --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0x8743759f --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/9aa9bc24f579cfcfb2600e097e5cbbf6/test --cache-dir=./.zig-cache --seed=0x8743759f --listen=- + +test-cuda-host ++- run test 87 pass, 1 fail (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected error.BadStep, found void + /.local/share/zig/0.17.0/lib/std/testing.zig:634:9: 0x1550a5b in expectError__func_2144 (std.zig) + return error.TestExpectedError; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:175:5: 0x1551171 in test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged (cuda.zig) + try std.testing.expectError(error.BadStep, p.map(1, &.{8})); + ^ +failed command: ./.zig-cache/o/193a5ecb2e1906ee4e603f1ef614a9ec/test --cache-dir=./.zig-cache --seed=0x8743759f --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x8743759f --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 87 pass, 1 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/no-poison.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/no-poison.log new file mode 100644 index 0000000000..385c1ff983 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/no-poison.log @@ -0,0 +1,98 @@ +test-cuda-host ++- run test 87 pass, 1 fail (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected 3, found 0 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1275e09 in expectEqualInner__func_954 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x13f8994 in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:247:5: 0x13f8a0f in test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written (cuda.zig) + try expectEqual(@as(usize, 3), F.fills); + ^ +failed command: ./.zig-cache/o/859e998fbd2ca0ebff200509a13485b8/test --cache-dir=./.zig-cache --seed=0x40e6505b --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/aecf7ecc87e2d0afd29c1a6481a298e6/test --cache-dir=./.zig-cache --seed=0x40e6505b --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0x40e6505b --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x40e6505b --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/cba679e163611d653fe3fd6108909728/test --cache-dir=./.zig-cache --seed=0x40e6505b --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 87 pass, 1 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/no-verify-rows.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/no-verify-rows.log new file mode 100644 index 0000000000..eb451bbed8 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/no-verify-rows.log @@ -0,0 +1,98 @@ +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xdf2de247 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/58226c593f2b38486e4ade3e789a91f1/test --cache-dir=./.zig-cache --seed=0xdf2de247 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0xdf2de247 --listen=- + +test-cuda-host ++- run test 87 pass, 1 fail (88 total) +error: 'cuda_native.test.the plan's window, the context and a verify window, takes the pages of a stream's table at that capacity' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + expected 17, found 16 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1275e09 in expectEqualInner__func_954 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x128974a in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_native.zig:410:9: 0x12897a3 in test.the plan's window, the context and a verify window, takes the pages of a stream's table at that capacity (cuda.zig) + try std.testing.expectEqual(pages_mod.tableLen(capacity(context)), l.pages(context + window_rows)); + ^ +failed command: ./.zig-cache/o/bd1413d9ee4c3b6853fd786e726cb5c7/test --cache-dir=./.zig-cache --seed=0xdf2de247 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/8c57c3ebd7dcc53c9383cd3e1f931571/test --cache-dir=./.zig-cache --seed=0xdf2de247 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 87 pass, 1 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-as-window.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-as-window.log new file mode 100644 index 0000000000..2888d593a7 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-as-window.log @@ -0,0 +1,985 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xff383845 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/93b9fc96b23df867c9ca96a57392a2eb/test --cache-dir=./.zig-cache --seed=0xff383845 --listen=- + +test-cuda-host ++- run test 80 pass, 8 fail (88 total); 35 leaks +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:58:49: 0x13eba62 in init (cuda.zig) + if (own == null or own.?.st.table == 0) return error.OwnNotPaged; // slot 0 holds a window, or none was made + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:103:9: 0x13f7af4 in pool__func_1708 (cuda.zig) + return pages_mod.Pages.init(f.e.gpa, &f.e, l, streams, pages, leader); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:112:15: 0x1553f29 in test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back (cuda.zig) + const p = try f.pool(l, 3, 8, true); + ^ + [SafeAllocator] (err): leaked [addr: 7d510cb1a8a0, len: 32 (0x20) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:380:36: 0x13e7d1b in make (cuda.zig) + const rings = try gpa.alloc(u64, 2 * (nd + 1)); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:112:25: 0x1553ee9 in test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back (cuda.zig) + const p = try f.pool(l, 3, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d510cb1a7d8, len: 64 (0x40) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:377:37: 0x13e734c in make (cuda.zig) + const caches = try gpa.alloc(dsa_block.Caches, nd); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:112:25: 0x1553ee9 in test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back (cuda.zig) + const p = try f.pool(l, 3, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d510cb1a748, len: 2 (0x2) align: 1] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:374:34: 0x13e714b in make (cuda.zig) + const cur = try gpa.alloc(u1, nk); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:112:25: 0x1553ee9 in test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back (cuda.zig) + const p = try f.pool(l, 3, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d510cb1a270, len: 488 (0x1e8) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:31:31: 0x13eaa9f in ownSequence (cuda.zig) + const q = try e.gpa.create(types.Seq); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:112:25: 0x1553ee9 in test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back (cuda.zig) + const p = try f.pool(l, 3, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7d510ce2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d510cb1a160, len: 136 (0x88) align: 8] allocated at: + /.local/share/zig/0.17.0/lib/std/array_list.zig:1375:56: 0x13f2c31 in ensureTotalCapacityPrecise (std.zig) + const new_memory = try gpa.alignedAlloc(T, alignment, new_capacity); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1349:51: 0x13f2859 in ensureTotalCapacity (std.zig) + return self.ensureTotalCapacityPrecise(gpa, growCapacity(new_capacity)); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1406:41: 0x13f671d in addOne (std.zig) + try self.ensureTotalCapacity(gpa, newlen); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1027:49: 0x13f64bc in append (std.zig) + const new_item_ptr = try self.addOne(gpa); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:29:49: 0x13ea9a5 in ownSequence (cuda.zig) + if (e.seqs.items.len == 0) try e.seqs.append(e.gpa, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:112:25: 0x1553ee9 in test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back (cuda.zig) + const p = try f.pool(l, 3, 8, true); + ^ + (additional stack frames may have been skipped...) +error: 'cuda_pages_test.test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x1270da9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:140:5: 0x1552d6d in test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others (cuda.zig) + try expect(f.e.cur == own and f.e.slot == 0 and own.st.table != 0 and own.st.capacity == 0); // no window made + ^ + [SafeAllocator] (err): leaked [addr: 7d5109f168a0, len: 32 (0x20) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:380:36: 0x13e7d1b in make (cuda.zig) + const rings = try gpa.alloc(u64, 2 * (nd + 1)); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:138:30: 0x1552b99 in test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f167d8, len: 64 (0x40) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:377:37: 0x13e734c in make (cuda.zig) + const caches = try gpa.alloc(dsa_block.Caches, nd); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:138:30: 0x1552b99 in test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f16748, len: 2 (0x2) align: 1] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:374:34: 0x13e714b in make (cuda.zig) + const cur = try gpa.alloc(u1, nk); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:138:30: 0x1552b99 in test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f16270, len: 488 (0x1e8) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:31:31: 0x13eaa9f in ownSequence (cuda.zig) + const q = try e.gpa.create(types.Seq); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:138:30: 0x1552b99 in test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7d510ce2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x7d510ce2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f16160, len: 136 (0x88) align: 8] allocated at: + /.local/share/zig/0.17.0/lib/std/array_list.zig:1375:56: 0x13f2c31 in ensureTotalCapacityPrecise (std.zig) + const new_memory = try gpa.alignedAlloc(T, alignment, new_capacity); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1349:51: 0x13f2859 in ensureTotalCapacity (std.zig) + return self.ensureTotalCapacityPrecise(gpa, growCapacity(new_capacity)); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1406:41: 0x13f671d in addOne (std.zig) + try self.ensureTotalCapacity(gpa, newlen); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1027:49: 0x13f64bc in append (std.zig) + const new_item_ptr = try self.addOne(gpa); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:29:49: 0x13ea9a5 in ownSequence (cuda.zig) + if (e.seqs.items.len == 0) try e.seqs.append(e.gpa, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:138:30: 0x1552b99 in test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + (additional stack frames may have been skipped...) +error: 'cuda_pages_test.test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:58:49: 0x13eba62 in init (cuda.zig) + if (own == null or own.?.st.table == 0) return error.OwnNotPaged; // slot 0 holds a window, or none was made + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:103:9: 0x13f7af4 in pool__func_1708 (cuda.zig) + return pages_mod.Pages.init(f.e.gpa, &f.e, l, streams, pages, leader); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:170:15: 0x1550d58 in test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged (cuda.zig) + const p = try f.pool(l, 3, 8, false); + ^ + [SafeAllocator] (err): leaked [addr: 7d5109f0e8a0, len: 32 (0x20) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:380:36: 0x13e7d1b in make (cuda.zig) + const rings = try gpa.alloc(u64, 2 * (nd + 1)); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:170:25: 0x1550d18 in test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged (cuda.zig) + const p = try f.pool(l, 3, 8, false); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f0e7d8, len: 64 (0x40) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:377:37: 0x13e734c in make (cuda.zig) + const caches = try gpa.alloc(dsa_block.Caches, nd); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:170:25: 0x1550d18 in test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged (cuda.zig) + const p = try f.pool(l, 3, 8, false); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f0e748, len: 2 (0x2) align: 1] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:374:34: 0x13e714b in make (cuda.zig) + const cur = try gpa.alloc(u1, nk); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:170:25: 0x1550d18 in test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged (cuda.zig) + const p = try f.pool(l, 3, 8, false); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f0e270, len: 488 (0x1e8) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:31:31: 0x13eaa9f in ownSequence (cuda.zig) + const q = try e.gpa.create(types.Seq); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:170:25: 0x1550d18 in test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged (cuda.zig) + const p = try f.pool(l, 3, 8, false); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7d510ce2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f0e160, len: 136 (0x88) align: 8] allocated at: + /.local/share/zig/0.17.0/lib/std/array_list.zig:1375:56: 0x13f2c31 in ensureTotalCapacityPrecise (std.zig) + const new_memory = try gpa.alignedAlloc(T, alignment, new_capacity); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1349:51: 0x13f2859 in ensureTotalCapacity (std.zig) + return self.ensureTotalCapacityPrecise(gpa, growCapacity(new_capacity)); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1406:41: 0x13f671d in addOne (std.zig) + try self.ensureTotalCapacity(gpa, newlen); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1027:49: 0x13f64bc in append (std.zig) + const new_item_ptr = try self.addOne(gpa); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:29:49: 0x13ea9a5 in ownSequence (cuda.zig) + if (e.seqs.items.len == 0) try e.seqs.append(e.gpa, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:170:25: 0x1550d18 in test.a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged (cuda.zig) + const p = try f.pool(l, 3, 8, false); + ^ + (additional stack frames may have been skipped...) +error: 'cuda_pages_test.test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:58:49: 0x13eba62 in init (cuda.zig) + if (own == null or own.?.st.table == 0) return error.OwnNotPaged; // slot 0 holds a window, or none was made + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:101:9: 0x1400f56 in follow (cuda.zig) + return init(gpa, e, l, streams, pages, false); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:190:15: 0x1401620 in test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool (cuda.zig) + const p = try pages_mod.Pages.follow(std.testing.allocator, &f.e, 2, 8, l.pageBytes()); + ^ + [SafeAllocator] (err): leaked [addr: 7d5109f0a8a0, len: 32 (0x20) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:380:36: 0x13e7d1b in make (cuda.zig) + const rings = try gpa.alloc(u64, 2 * (nd + 1)); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:186:30: 0x14012fb in test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f0a7d8, len: 64 (0x40) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:377:37: 0x13e734c in make (cuda.zig) + const caches = try gpa.alloc(dsa_block.Caches, nd); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:186:30: 0x14012fb in test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f0a748, len: 2 (0x2) align: 1] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:374:34: 0x13e714b in make (cuda.zig) + const cur = try gpa.alloc(u1, nk); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:186:30: 0x14012fb in test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f0a270, len: 488 (0x1e8) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:31:31: 0x13eaa9f in ownSequence (cuda.zig) + const q = try e.gpa.create(types.Seq); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:186:30: 0x14012fb in test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7d510ce2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x7d510ce2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f0a160, len: 136 (0x88) align: 8] allocated at: + /.local/share/zig/0.17.0/lib/std/array_list.zig:1375:56: 0x13f2c31 in ensureTotalCapacityPrecise (std.zig) + const new_memory = try gpa.alignedAlloc(T, alignment, new_capacity); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1349:51: 0x13f2859 in ensureTotalCapacity (std.zig) + return self.ensureTotalCapacityPrecise(gpa, growCapacity(new_capacity)); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1406:41: 0x13f671d in addOne (std.zig) + try self.ensureTotalCapacity(gpa, newlen); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1027:49: 0x13f64bc in append (std.zig) + const new_item_ptr = try self.addOne(gpa); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:29:49: 0x13ea9a5 in ownSequence (cuda.zig) + if (e.seqs.items.len == 0) try e.seqs.append(e.gpa, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:186:30: 0x14012fb in test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + (additional stack frames may have been skipped...) +error: 'cuda_pages_test.test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:58:49: 0x13eba62 in init (cuda.zig) + if (own == null or own.?.st.table == 0) return error.OwnNotPaged; // slot 0 holds a window, or none was made + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:103:9: 0x13f7af4 in pool__func_1708 (cuda.zig) + return pages_mod.Pages.init(f.e.gpa, &f.e, l, streams, pages, leader); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:199:15: 0x13fe38b in test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + [SafeAllocator] (err): leaked [addr: 7d5109f068a0, len: 32 (0x20) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:380:36: 0x13e7d1b in make (cuda.zig) + const rings = try gpa.alloc(u64, 2 * (nd + 1)); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:199:25: 0x13fe34b in test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f067d8, len: 64 (0x40) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:377:37: 0x13e734c in make (cuda.zig) + const caches = try gpa.alloc(dsa_block.Caches, nd); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:199:25: 0x13fe34b in test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f06748, len: 2 (0x2) align: 1] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:374:34: 0x13e714b in make (cuda.zig) + const cur = try gpa.alloc(u1, nk); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:199:25: 0x13fe34b in test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f06270, len: 488 (0x1e8) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:31:31: 0x13eaa9f in ownSequence (cuda.zig) + const q = try e.gpa.create(types.Seq); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:199:25: 0x13fe34b in test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7d510ce2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f06160, len: 136 (0x88) align: 8] allocated at: + /.local/share/zig/0.17.0/lib/std/array_list.zig:1375:56: 0x13f2c31 in ensureTotalCapacityPrecise (std.zig) + const new_memory = try gpa.alignedAlloc(T, alignment, new_capacity); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1349:51: 0x13f2859 in ensureTotalCapacity (std.zig) + return self.ensureTotalCapacityPrecise(gpa, growCapacity(new_capacity)); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1406:41: 0x13f671d in addOne (std.zig) + try self.ensureTotalCapacity(gpa, newlen); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1027:49: 0x13f64bc in append (std.zig) + const new_item_ptr = try self.addOne(gpa); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:29:49: 0x13ea9a5 in ownSequence (cuda.zig) + if (e.seqs.items.len == 0) try e.seqs.append(e.gpa, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:199:25: 0x13fe34b in test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + (additional stack frames may have been skipped...) +error: 'cuda_pages_test.test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:58:49: 0x13eba62 in init (cuda.zig) + if (own == null or own.?.st.table == 0) return error.OwnNotPaged; // slot 0 holds a window, or none was made + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:103:9: 0x13f7af4 in pool__func_1708 (cuda.zig) + return pages_mod.Pages.init(f.e.gpa, &f.e, l, streams, pages, leader); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:224:15: 0x13fcd8b in test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + [SafeAllocator] (err): leaked [addr: 7d5109f028a0, len: 32 (0x20) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:380:36: 0x13e7d1b in make (cuda.zig) + const rings = try gpa.alloc(u64, 2 * (nd + 1)); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:224:25: 0x13fcd4b in test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f027d8, len: 64 (0x40) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:377:37: 0x13e734c in make (cuda.zig) + const caches = try gpa.alloc(dsa_block.Caches, nd); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:224:25: 0x13fcd4b in test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f02748, len: 2 (0x2) align: 1] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:374:34: 0x13e714b in make (cuda.zig) + const cur = try gpa.alloc(u1, nk); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:224:25: 0x13fcd4b in test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f02270, len: 488 (0x1e8) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:31:31: 0x13eaa9f in ownSequence (cuda.zig) + const q = try e.gpa.create(types.Seq); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:224:25: 0x13fcd4b in test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7d510ce2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109f02160, len: 136 (0x88) align: 8] allocated at: + /.local/share/zig/0.17.0/lib/std/array_list.zig:1375:56: 0x13f2c31 in ensureTotalCapacityPrecise (std.zig) + const new_memory = try gpa.alignedAlloc(T, alignment, new_capacity); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1349:51: 0x13f2859 in ensureTotalCapacity (std.zig) + return self.ensureTotalCapacityPrecise(gpa, growCapacity(new_capacity)); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1406:41: 0x13f671d in addOne (std.zig) + try self.ensureTotalCapacity(gpa, newlen); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1027:49: 0x13f64bc in append (std.zig) + const new_item_ptr = try self.addOne(gpa); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:29:49: 0x13ea9a5 in ownSequence (cuda.zig) + if (e.seqs.items.len == 0) try e.seqs.append(e.gpa, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:224:25: 0x13fcd4b in test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not (cuda.zig) + const p = try f.pool(l, 2, 8, true); + ^ + (additional stack frames may have been skipped...) +error: 'cuda_pages_test.test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:58:49: 0x13eba62 in init (cuda.zig) + if (own == null or own.?.st.table == 0) return error.OwnNotPaged; // slot 0 holds a window, or none was made + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:103:9: 0x13f7af4 in pool__func_1708 (cuda.zig) + return pages_mod.Pages.init(f.e.gpa, &f.e, l, streams, pages, leader); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:243:15: 0x13f8e19 in test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written (cuda.zig) + const p = try f.pool(l, 1, 8, true); + ^ + [SafeAllocator] (err): leaked [addr: 7d5109efe8a0, len: 32 (0x20) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:380:36: 0x13e7d1b in make (cuda.zig) + const rings = try gpa.alloc(u64, 2 * (nd + 1)); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:243:25: 0x13f8dd9 in test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written (cuda.zig) + const p = try f.pool(l, 1, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109efe7d8, len: 64 (0x40) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:377:37: 0x13e734c in make (cuda.zig) + const caches = try gpa.alloc(dsa_block.Caches, nd); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:243:25: 0x13f8dd9 in test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written (cuda.zig) + const p = try f.pool(l, 1, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109efe748, len: 2 (0x2) align: 1] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:374:34: 0x13e714b in make (cuda.zig) + const cur = try gpa.alloc(u1, nk); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_state.zig:343:20: 0x13ea7ad in initFormat (cuda.zig) + return make(gpa, d, w, capacity, kv8, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:33:43: 0x13eab92 in ownSequence (cuda.zig) + q.* = .{ .st = try fw.State.initFormat(e.gpa, e.d, &e.w, e.capacity, false) }; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:243:25: 0x13f8dd9 in test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written (cuda.zig) + const p = try f.pool(l, 1, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109efe270, len: 488 (0x1e8) align: 8] allocated at: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:31:31: 0x13eaa9f in ownSequence (cuda.zig) + const q = try e.gpa.create(types.Seq); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:243:25: 0x13f8dd9 in test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written (cuda.zig) + const p = try f.pool(l, 1, 8, true); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7d510ce2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + (additional stack frames may have been skipped...) + + [SafeAllocator] (err): leaked [addr: 7d5109efe160, len: 136 (0x88) align: 8] allocated at: + /.local/share/zig/0.17.0/lib/std/array_list.zig:1375:56: 0x13f2c31 in ensureTotalCapacityPrecise (std.zig) + const new_memory = try gpa.alignedAlloc(T, alignment, new_capacity); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1349:51: 0x13f2859 in ensureTotalCapacity (std.zig) + return self.ensureTotalCapacityPrecise(gpa, growCapacity(new_capacity)); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1406:41: 0x13f671d in addOne (std.zig) + try self.ensureTotalCapacity(gpa, newlen); + ^ + /.local/share/zig/0.17.0/lib/std/array_list.zig:1027:49: 0x13f64bc in append (std.zig) + const new_item_ptr = try self.addOne(gpa); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:29:49: 0x13ea9a5 in ownSequence (cuda.zig) + if (e.seqs.items.len == 0) try e.seqs.append(e.gpa, null); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:102:34: 0x13f7a05 in pool__func_1708 (cuda.zig) + try pages_mod.ownSequence(&f.e); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:243:25: 0x13f8dd9 in test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written (cuda.zig) + const p = try f.pool(l, 1, 8, true); + ^ + (additional stack frames may have been skipped...) +error: 'cuda_pages_test.test.every host allocation failure while making the pages leaves no device buffer behind' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:58:49: 0x13eba62 in init (cuda.zig) + if (own == null or own.?.st.table == 0) return error.OwnNotPaged; // slot 0 holds a window, or none was made + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:261:15: 0x13eea82 in made (cuda.zig) + const p = try pages_mod.Pages.init(gpa, &f.e, l, 3, 8, true); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:274:17: 0x13ef031 in run (cuda.zig) + return err; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:1154:9: 0x13ef2d8 in checkAllAllocationFailures__func_1674 (std.zig) + try @call(.auto, test_fn, .{failing_allocator_inst.allocator()} ++ extra_args); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:270:5: 0x13ef8a9 in test.every host allocation failure while making the pages leaves no device buffer behind (cuda.zig) + try std.testing.checkAllAllocationFailures(std.testing.allocator, struct { + ^ +failed command: ./.zig-cache/o/6881e07932c1df0a66c4f0165ed1587e/test --cache-dir=./.zig-cache --seed=0xff383845 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/9f683251dcda953916db640db1debebb/test --cache-dir=./.zig-cache --seed=0xff383845 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0xff383845 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 252/260 tests passed (8 failed) +test-cuda-host transitive failure ++- run test 80 pass, 8 fail (88 total); 35 leaks diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-no-floor.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-no-floor.log new file mode 100644 index 0000000000..4f77492b16 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-no-floor.log @@ -0,0 +1,334 @@ +test-kv-pool ++- run test 10 pass, 1 fail (11 total) +error: 'kv_pool_test.test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest' failed: + expected 1073741824, found 14770634752 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1248193 in expectEqualInner__func_988 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x124b8cb in expectEqual (kv_pool.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:83:5: 0x124b924 in test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest (kv_pool.zig) + try expectEqual(g, tight.copied); + ^ +failed command: ./.zig-cache/o/e71c4189a65d218bc28b74ddc9a62074/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/5b7a8afca0f06a7eca0f93da44433ed7/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test 72 pass, 1 fail (73 total) +error: 'cuda_kv_pool.test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so' failed: + expected null, found .{ .lanes = 1, .snaps = 0, .copied = 113639423, .pages = 0 } + /.local/share/zig/0.17.0/lib/std/testing.zig:178:21: 0x18afc08 in expectEqualInner__func_3152 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x18b14dc in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:189:5: 0x18b156a in test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so (cuda.zig) + try std.testing.expectEqual(@as(?Plan, null), try plan(Owned, a, l, floor - 1, o, 8192, &problem, false, true)); + ^ +failed command: ./.zig-cache/o/0e03d9c00fced2773429e41a1d1e7331/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/dee1fc8eca0980534b7e7e07725db236/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/ea8853d7369fa3aa3f3fa65976c95e37/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/57b908803116a14975f48712119398c7/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/96bf0764ee30e6abf769bbe593e4ed2d/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/dbc8589efadb81cae588d97b87323144/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/054ad42bca867725326872cf410a9ff0/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test 0 pass, 1 fail (1 total) +(u8, &first, &changed)); + ^ +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql +failed command: ./.zig-cache/o/36fb31cc1ab209e48baec3e5969ac933/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/df24b1c14207ac90ba88e64e4d57cb3e/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/165312bbbacd7dceb45d092f32decb23/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test 351 pass, 1 fail (352 total) +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +error: 'kv_pool_test.test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest' failed: + expected 1073741824, found 14770634752 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x13a3cb3 in expectEqualInner__func_1458 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x13a73eb in expectEqual (root.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:83:5: 0x13a7444 in test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest (root.zig) + try expectEqual(g, tight.copied); + ^ +failed command: ./.zig-cache/o/6e1c502a3a7d75e0fd0b41453ed081eb/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/76203e23dc0c15c7d79780131fb6f1d7/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/4099ed0ec80aa66003b66abc96b3e6e0/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/80c8be8b2a2f6b5428219b19a8b99e39/test --cache-dir=./.zig-cache --seed=0x7a7bdf37 --listen=- + +Build Summary: 157/163 steps succeeded (4 failed); 1050/1058 tests passed (4 skipped, 4 failed) +test-cuda-host transitive failure ++- run test 0 pass, 1 fail (1 total) ++- run test 351 pass, 1 fail (352 total) ++- run test 72 pass, 1 fail (73 total) +test-kv-pool transitive failure ++- run test 10 pass, 1 fail (11 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-not-bounded.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-not-bounded.log new file mode 100644 index 0000000000..dce55754a7 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-not-bounded.log @@ -0,0 +1,321 @@ +test-kv-pool ++- run test 10 pass, 1 fail (11 total) +error: 'kv_pool_test.test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest' failed: + expected 2049, found 0 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1237839 in expectEqualInner__func_947 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x124bd5f in expectEqual (kv_pool.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:89:5: 0x124bdb8 in test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest (kv_pool.zig) + try expectEqual(@as(usize, 2049), alone.pages); + ^ +failed command: ./.zig-cache/o/ca7504c9c5400ff95334e620e0292312/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/ca3d1e2406cae671f264919b119a99cc/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/5d5d38a2e68adbb505fa0a253de1635f/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/b89911885aca51cb18c802b8345f6515/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/4e6265f314851f670e5c35c55c799335/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test 72 pass, 1 fail (73 total) +error: 'cuda_kv_pool.test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so' failed: + expected 68, found 51 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1350669 in expectEqualInner__func_981 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x18b1052 in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:187:5: 0x18b10e0 in test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so (cuda.zig) + try std.testing.expectEqual(@as(usize, 4 * 17), p.pages); // every stream at most 8,192 rows and a verify window: 17 pages + ^ +failed command: ./.zig-cache/o/5aab790d9404973e2792e9d21b1a4ff4/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/8fc1559f6f7374bd83c833e3f54d29fe/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/54d2480e0b098d3f6eebeba184bb65ea/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/a7bb2b9e1e2446497c94a75489df3a37/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test 351 pass, 1 fail (352 total) +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +error: 'kv_pool_test.test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest' failed: + expected 2049, found 0 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x12cd929 in expectEqualInner__func_947 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x13a787f in expectEqual (root.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:89:5: 0x13a78d8 in test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest (root.zig) + try expectEqual(@as(usize, 2049), alone.pages); + ^ +failed command: ./.zig-cache/o/8c31506d631bf9dce8b8f5666c512720/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/be22ff63c0c5150eec914b03614380d4/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/3f6ccc8157588de9e48675e1a377e863/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/d2d3a4096824fb8ac8c91d98e5735d5d/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/5857e5261774329899cadc1a2c1e249b/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +test-cuda-host ++- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/d6ae02fc1daae392149ed463231d820f/test --cache-dir=./.zig-cache --seed=0xfe64333 --listen=- + +Build Summary: 158/163 steps succeeded (3 failed); 1051/1058 tests passed (4 skipped, 3 failed) +test-cuda-host transitive failure ++- run test 351 pass, 1 fail (352 total) ++- run test 72 pass, 1 fail (73 total) +test-kv-pool transitive failure ++- run test 10 pass, 1 fail (11 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-not-placed.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-not-placed.log new file mode 100644 index 0000000000..c07d58069b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/own-not-placed.log @@ -0,0 +1,98 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xc04bafe2 --listen=- + +test-cuda-host ++- run test 87 pass, 1 fail (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected 1141899264, found 0 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x12f28a3 in expectEqualInner__func_1230 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x15531e1 in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:146:5: 0x155325c in test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others (cuda.zig) + try expectEqual(p.arena.?.ptr, own.st.caches[0].lc); // its DSA layer's part of page 0 starts the arena + ^ +failed command: ./.zig-cache/o/025a1094d7025ee19f17c20ed9a41ed5/test --cache-dir=./.zig-cache --seed=0xc04bafe2 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/1d1db3d74ce1cce6f47f35532e02bfad/test --cache-dir=./.zig-cache --seed=0xc04bafe2 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/d8e8498e99494877008f2478fe8e3de4/test --cache-dir=./.zig-cache --seed=0xc04bafe2 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0xc04bafe2 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 failed) +test-cuda-host transitive failure ++- run test 87 pass, 1 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/page-bytes-unchecked.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/page-bytes-unchecked.log new file mode 100644 index 0000000000..ab140306c8 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/page-bytes-unchecked.log @@ -0,0 +1,135 @@ +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x8e15b27f --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/012b8bc30e99bc550032e24cff54aa21/test --cache-dir=./.zig-cache --seed=0x8e15b27f --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0x8e15b27f --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/fe7b359feeffc594c5b79174156a2a90/test --cache-dir=./.zig-cache --seed=0x8e15b27f --listen=- + +test-cuda-host ++- run test 87 pass, 1 crash (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool' terminated with signal ABRT with stderr: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected error.RankMismatch, found .{ .gpa = .{ .ptr = anyopaque@1611c80, .vtable = .{ .alloc = fn (*anyopaque, usize, mem.Alignment, usize) ?[*]u8@12527d0, .resize = fn (*anyopaque, []u8, mem.Alignment, usize, usize) bool@125d520, .remap = fn (*anyopaque, []u8, mem.Alignment, usize, usize) ?[*]u8@12532c0, .free = fn (*anyopaque, []u8, mem.Alignment, usize) void@125ddf0 } }, .e = .{ .gpa = .{ .ptr = anyopaque@1611c80, .vtable = .{ ... } }, .io = .{ .userdata = anyopaque@aaaaaaaaaaaaaaaa, .vtable = General protection exception (no address available) + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1488:71: 0x151fefe in printValue__func_2038 (std.zig) + .@"enum", .@"union", .@"struct" => return w.printValue(fmt, options, value.*, max_depth), + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1481:33: 0x151f812 in printValue__func_1731 (std.zig) + try w.printValue(ANY, options, @field(value, f_name), max_depth - 1); + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1481:33: 0x1405923 in printValue__func_1730 (std.zig) + try w.printValue(ANY, options, @field(value, f_name), max_depth - 1); + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1488:71: 0x1404b52 in printValue__func_1724 (std.zig) + .@"enum", .@"union", .@"struct" => return w.printValue(fmt, options, value.*, max_depth), + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1481:33: 0x1402c19 in printValue__func_1723 (std.zig) + try w.printValue(ANY, options, @field(value, f_name), max_depth - 1); + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1488:71: 0x140234a in printValue__func_1722 (std.zig) + .@"enum", .@"union", .@"struct" => return w.printValue(fmt, options, value.*, max_depth), + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:805:25: 0x1401f13 in print__func_1721 (std.zig) + try w.printValue( + ^ + /.local/share/zig/0.17.0/lib/std/debug.zig:330:39: 0x1401b1a in print__func_1720 (std.zig) + stderr.file_writer.interface.print(fmt, args) catch return; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:47:24: 0x14019c3 in failPrint__func_1719 (std.zig) + std.debug.print(fmt, args); + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:633:18: 0x14010d1 in expectError__func_1718 (std.zig) + failPrint("expected error.{s}, found {any}\n", .{ @errorName(expected_error), actual_payload }); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:188:32: 0x140147f in test.a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool (cuda.zig) + try std.testing.expectError(error.RankMismatch, pages_mod.Pages.follow(std.testing.allocator, &f.e, 2, 8, more)); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x74156382a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x74156382a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x15a66f0 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/03204ff8732765ec51b65757a865a563/test --cache-dir=./.zig-cache --seed=0x8e15b27f --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 crashed) +test-cuda-host transitive failure ++- run test 87 pass, 1 crash (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/paged-first-at-cache.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/paged-first-at-cache.log new file mode 100644 index 0000000000..8314f191d2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/paged-first-at-cache.log @@ -0,0 +1,125 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/5d8e71be5b9e1e258f193eaa677cdc11/test --cache-dir=./.zig-cache --seed=0x4a58a149 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x4a58a149 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/6e55aabf9979dbef90f653ec32c8cbfd/test --cache-dir=./.zig-cache --seed=0x4a58a149 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0x4a58a149 --listen=- + +test-glm-snapshots-host ++- run test 13 pass, 3 fail (16 total) +error: 'glm_snapshot_host.test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + /zt/kvp-mut/zig/src/families/glm/cuda_snapshot_layout.zig:83:13: 0x124c74e in add (cuda.zig) + return error.SnapshotShape; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_snapshot_layout.zig:36:9: 0x124e12c in make (cuda.zig) + try p.add(st, st.arena.buf.ptr, @intCast(first - st.arena.buf.ptr)); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_snapshot_layout.zig:19:9: 0x125a27d in init (cuda.zig) + return make(st, c, at, head, false); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_snapshots.zig:64:18: 0x125c57f in restore__func_1009 (cuda.zig) + const plan = try layout.Plan.init(&e.cur.st, &e.c, s.pos, if (e.mtp) s.mtp_len else null); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:435:13: 0x1263afe in test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window (glm_snapshot_host.zig) + try f.e.restoreState(id); + ^ +error: 'glm_snapshot_host.test.GLM snapshots: while a paged stream is current, a prefix's size is the engine's own sequence's, past its pages' failed: + expected 6291456, found 18446744073709551615 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245393 in expectEqualInner__func_965 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x1256894 in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:488:5: 0x1256951 in test.GLM snapshots: while a paged stream is current, a prefix's size is the engine's own sequence's, past its pages (glm_snapshot_host.zig) + try t.expectEqual(own, f.e.stateBytes(4096)); + ^ +error: 'glm_snapshot_host.test.GLM snapshots: with the engine's own sequence on pages too, a prefix's size past a stream's pages is a window's' failed: + expected 6291456, found 18446744073709551615 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245393 in expectEqualInner__func_965 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x12464f6 in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:516:34: 0x12465c0 in test.GLM snapshots: with the engine's own sequence on pages too, a prefix's size past a stream's pages is a window's (glm_snapshot_host.zig) + for (ats, sizes) |at, s| try t.expectEqual(s, f.e.stateBytes(at)); + ^ +failed command: ./.zig-cache/o/07d5cca331266fc37a1dad3ce01ebd54/test --cache-dir=./.zig-cache --seed=0x4a58a149 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 257/260 tests passed (3 failed) +test-glm-snapshots-host transitive failure ++- run test 13 pass, 3 fail (16 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/paged-table-kept.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/paged-table-kept.log new file mode 100644 index 0000000000..f19857019c --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/paged-table-kept.log @@ -0,0 +1,94 @@ +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/f0a0408898b407674a5888beb5d37c3b/test --cache-dir=./.zig-cache --seed=0xfee8c2c0 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/59251a11aec4ecc970bc426dbda379f4/test --cache-dir=./.zig-cache --seed=0xfee8c2c0 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0xfee8c2c0 --listen=- + +test-glm-snapshots-host ++- run test 15 pass, 1 fail (16 total) +error: 'glm_snapshot_host.test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + /zt/kvp-mut/zig/src/families/glm/cuda_snapshots.zig:65:28: 0x125c690 in restore__func_1009 (cuda.zig) + if (plan.len != s.len) return error.SnapshotSize; + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:435:13: 0x1263b7e in test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window (glm_snapshot_host.zig) + try f.e.restoreState(id); + ^ +failed command: ./.zig-cache/o/d172b9c7d55192c5908bdd9be4e42efd/test --cache-dir=./.zig-cache --seed=0xfee8c2c0 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xfee8c2c0 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 failed) +test-glm-snapshots-host transitive failure ++- run test 15 pass, 1 fail (16 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/pages-not-agreed.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/pages-not-agreed.log new file mode 100644 index 0000000000..0ca3156a3d --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/pages-not-agreed.log @@ -0,0 +1,123 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/22edfe8778563327756b0726b022950c/test --cache-dir=./.zig-cache --seed=0xe29e212b --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0xe29e212b --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xe29e212b --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/189e7a1d56b778a7c0014cf365cd4ea6/test --cache-dir=./.zig-cache --seed=0xe29e212b --listen=- + +test-cuda-host ++- run test 86 pass, 2 crash (88 total) +error: 'cuda_ranks.test.fingerprints: agreeing ranks pass; the first differing field and rank are named' terminated with signal ABRT with stderr: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + thread 24110 panic: attempt to use null value + /zt/kvp-mut/zig/src/families/glm/cuda_ranks.zig:387:87: 0x13aaedb in test.fingerprints: agreeing ranks pass; the first differing field and rank are named (cuda.zig) + try std.testing.expectEqual(Mismatch{ .field = .pages, .rank = 1 }, mismatch(&all).?); // TF_KV_PAGES on one rank + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x77d8f1e2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x77d8f1e2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x15a6600 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +error: 'cuda_ranks.test.every scheduling and identity field that differs stops the ranks' terminated with signal ABRT with stderr: + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather + thread 24113 panic: attempt to use null value + /zt/kvp-mut/zig/src/families/glm/cuda_ranks.zig:514:84: 0x139b9b6 in test.every scheduling and identity field that differs stops the ranks (cuda.zig) + try std.testing.expectEqual(@field(Fingerprint.Field, name), mismatch(&all).?.field); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x78f07ac2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x78f07ac2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x15a6600 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/8d6ccf7bbc8527c8ef019cd2412c4c2c/test --cache-dir=./.zig-cache --seed=0xe29e212b --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 258/260 tests passed (2 crashed) +test-cuda-host transitive failure ++- run test 86 pass, 2 crash (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/pages-not-fingerprinted.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/pages-not-fingerprinted.log new file mode 100644 index 0000000000..ef35fcd0e2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/pages-not-fingerprinted.log @@ -0,0 +1,105 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x462bb931 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/549d84b90c948a699321f0900a697ed0/test --cache-dir=./.zig-cache --seed=0x462bb931 --listen=- + +test-cuda-host ++- run test 87 pass, 1 crash (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_ranks.test.fingerprints: agreeing ranks pass; the first differing field and rank are named' terminated with signal ABRT with stderr: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + thread 24348 panic: attempt to use null value + /zt/kvp-mut/zig/src/families/glm/cuda_ranks.zig:387:87: 0x13ab02b in test.fingerprints: agreeing ranks pass; the first differing field and rank are named (cuda.zig) + try std.testing.expectEqual(Mismatch{ .field = .pages, .rank = 1 }, mismatch(&all).?); // TF_KV_PAGES on one rank + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7420b9c2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x7420b9c2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x15ca020 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/3ca01de4814b8032df47d1625b879caa/test --cache-dir=./.zig-cache --seed=0x462bb931 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0x462bb931 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/69aa5048757b0b8ddd1bdd2cad245a95/test --cache-dir=./.zig-cache --seed=0x462bb931 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 crashed) +test-cuda-host transitive failure ++- run test 87 pass, 1 crash (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/piece-first-page.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/piece-first-page.log new file mode 100644 index 0000000000..1f3424b616 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/piece-first-page.log @@ -0,0 +1,123 @@ +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/950074aa9cfff862fb0a60b0786c7762/test --cache-dir=./.zig-cache --seed=0x369886e1 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/85052cdebc1c457fab1334282cfe3fb8/test --cache-dir=./.zig-cache --seed=0x369886e1 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0x369886e1 --listen=- + +test-glm-snapshots-host ++- run test 14 pass, 2 fail (16 total) +error: 'glm_snapshot_host.test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + slices differ. first difference occurs at index 0 (0x0) + + ============ expected this output: ============= len: 16 (0x10) + + EE 05 38 5F 72 A9 CC E3 06 3D 50 77 AA C1 E4 1B ..8_r....=Pw.... + + ============= instead found this: ============== len: 16 (0x10) + + E6 1D 30 57 8A A1 C4 FB 1E 35 68 8F A2 D9 FC 13 ..0W.....5h..... + + ================================================ + + /.local/share/zig/0.17.0/lib/std/testing.zig:346:5: 0x126054d in expectEqualSlices__func_1010 (std.zig) + return error.TestExpectedEqual; + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:383:5: 0x126072c in same (glm_snapshot_host.zig) + try t.expectEqualSlices(u8, rec.bytes(a, len), rec.bytes(b, len)); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:404:49: 0x1262064 in expectSamePrefix (glm_snapshot_host.zig) + for (0..@min(n / 4 + 2, x.pk_rows)) |p| try same(rowAt(a, x, x.pk, p, c.i_dim * 2, 128), rowAt(b, y, y.pk, p, c.i_dim * 2, 128), c.i_dim * 2); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:436:13: 0x1263ca1 in test.GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window (glm_snapshot_host.zig) + try expectSamePrefix(c, &window.st, &paged.st, at, head); + ^ +error: 'glm_snapshot_host.test.GLM snapshots: a paged part's piece from inside a page's span keeps its offset, on the page the order names' failed: + expected 1048606, found 1060894 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245393 in expectEqualInner__func_965 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x125702e in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:461:5: 0x1257087 in test.GLM snapshots: a paged part's piece from inside a page's span keeps its offset, on the page the order names (glm_snapshot_host.zig) + try t.expectEqual(@as(u64, (1 << 20) + 30), a.live); + ^ +failed command: ./.zig-cache/o/5df4ead3c54702efb172df1c1d79b6b6/test --cache-dir=./.zig-cache --seed=0x369886e1 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x369886e1 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 258/260 tests passed (2 failed) +test-glm-snapshots-host transitive failure ++- run test 14 pass, 2 fail (16 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/piece-no-offset.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/piece-no-offset.log new file mode 100644 index 0000000000..02c93f3799 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/piece-no-offset.log @@ -0,0 +1,98 @@ +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xdcaabcff --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/8a1030f5cb8110619351671764bf79b8/test --cache-dir=./.zig-cache --seed=0xdcaabcff --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/ff2ed87d5764d0a4a1b48021a5760a57/test --cache-dir=./.zig-cache --seed=0xdcaabcff --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/d82abf3611d30794f6cc32ca3431b6f9/test --cache-dir=./.zig-cache --seed=0xdcaabcff --listen=- + +test-glm-snapshots-host ++- run test 15 pass, 1 fail (16 total) +error: 'glm_snapshot_host.test.GLM snapshots: a paged part's piece from inside a page's span keeps its offset, on the page the order names' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + expected 1048606, found 1048576 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245393 in expectEqualInner__func_965 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x125702e in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:461:5: 0x1257087 in test.GLM snapshots: a paged part's piece from inside a page's span keeps its offset, on the page the order names (glm_snapshot_host.zig) + try t.expectEqual(@as(u64, (1 << 20) + 30), a.live); + ^ +failed command: ./.zig-cache/o/bce4dff7de5f8a6b50b35f983660d4a0/test --cache-dir=./.zig-cache --seed=0xdcaabcff --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 failed) +test-glm-snapshots-host transitive failure ++- run test 15 pass, 1 fail (16 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/plan-flag-dropped.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/plan-flag-dropped.log new file mode 100644 index 0000000000..f481a585d3 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/plan-flag-dropped.log @@ -0,0 +1,293 @@ +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/e4ba30d29a5143ca4e05e92e95d45ebe/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/46ab26c0f957a0f6c71df9e01db15533/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/8a659e07ef41ab76477b7d9e749b71ca/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test 72 pass, 1 fail (73 total) +ow: 17 pages + ^ +error: 'cuda_kv_pool.test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so' failed: + expected 68, found 51 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1350669 in expectEqualInner__func_981 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x18b10d2 in expectEqual (cuda.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:187:5: 0x18b1160 in test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so (cuda.zig) + try std.testing.expectEqual(@as(usize, 4 * 17), p.pages); // every stream at most 8,192 rows and a verify wind +failed command: ./.zig-cache/o/7f1a31e4a329fcd047a4f96ccb4d6ad0/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/f9419dba1a4fe6ebd6e84eb94f7dcebf/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/837932e445f6c077b161a75e76307665/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/f1b7dd59cf21e84f4db4b59615b85a48/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/f5874ec54c4a99c0402cf2abdb749534/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/826bec6bc03435dcc4d3f85f6a062ce2/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/da15d3c41ea5d2153781660b047e67e6/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/300e9d9bc5cc2177d3c6380f0e8a7c36/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +failed command: ./.zig-cache/o/92af39c21a90673092679820f4bb048b/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/24a9f3e50fc977a68a7fb2ada6585356/test --cache-dir=./.zig-cache --seed=0x76df0ce2 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 1053/1058 tests passed (4 skipped, 1 failed) +test-cuda-host transitive failure ++- run test 72 pass, 1 fail (73 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/pool-on-window.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/pool-on-window.log new file mode 100644 index 0000000000..e707d65eaa --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/pool-on-window.log @@ -0,0 +1,135 @@ +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/ada1faab43bcf3f9bfd818e6ba5a5db9/test --cache-dir=./.zig-cache --seed=0xb3c4e7f2 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xb3c4e7f2 --listen=- + +test-cuda-host ++- run test 87 pass, 1 crash (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.a pool is refused while the engine's own sequence holds a window or was never made, and nothing is made for it' terminated with signal ABRT with stderr: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + expected error.OwnNotPaged, found .{ .gpa = .{ .ptr = anyopaque@1611c80, .vtable = .{ .alloc = fn (*anyopaque, usize, mem.Alignment, usize) ?[*]u8@12527d0, .resize = fn (*anyopaque, []u8, mem.Alignment, usize, usize) bool@125d520, .remap = fn (*anyopaque, []u8, mem.Alignment, usize, usize) ?[*]u8@12532c0, .free = fn (*anyopaque, []u8, mem.Alignment, usize) void@125ddf0 } }, .e = .{ .gpa = .{ .ptr = anyopaque@1611c80, .vtable = .{ ... } }, .io = .{ .userdata = anyopaque@aaaaaaaaaaaaaaaa, .vtable = General protection exception (no address available) + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1488:71: 0x151ff1e in printValue__func_2038 (std.zig) + .@"enum", .@"union", .@"struct" => return w.printValue(fmt, options, value.*, max_depth), + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1481:33: 0x151f832 in printValue__func_1731 (std.zig) + try w.printValue(ANY, options, @field(value, f_name), max_depth - 1); + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1481:33: 0x1405943 in printValue__func_1730 (std.zig) + try w.printValue(ANY, options, @field(value, f_name), max_depth - 1); + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1488:71: 0x1404b72 in printValue__func_1724 (std.zig) + .@"enum", .@"union", .@"struct" => return w.printValue(fmt, options, value.*, max_depth), + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1481:33: 0x1402c39 in printValue__func_1723 (std.zig) + try w.printValue(ANY, options, @field(value, f_name), max_depth - 1); + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:1488:71: 0x140236a in printValue__func_1722 (std.zig) + .@"enum", .@"union", .@"struct" => return w.printValue(fmt, options, value.*, max_depth), + ^ + /.local/share/zig/0.17.0/lib/std/Io/Writer.zig:805:25: 0x1401f33 in print__func_1721 (std.zig) + try w.printValue( + ^ + /.local/share/zig/0.17.0/lib/std/debug.zig:330:39: 0x1401b3a in print__func_1720 (std.zig) + stderr.file_writer.interface.print(fmt, args) catch return; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:47:24: 0x14019e3 in failPrint__func_1719 (std.zig) + std.debug.print(fmt, args); + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:633:18: 0x15521c1 in expectError__func_2145 (std.zig) + failPrint("expected error.{s}, found {any}\n", .{ @errorName(expected_error), actual_payload }); + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:161:32: 0x1552630 in test.a pool is refused while the engine's own sequence holds a window or was never made, and nothing is made for it (cuda.zig) + try std.testing.expectError(error.OwnNotPaged, pages_mod.Pages.init(std.testing.allocator, &f.e, l, 2, 8, true)); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x122193a in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x122276d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x121f478 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x74fc33e2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x74fc33e2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x15a6710 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/c7aebe0ae22356fd2093fffe639bbb33/test --cache-dir=./.zig-cache --seed=0xb3c4e7f2 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0xb3c4e7f2 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/582675a7256f9468505a84d8da0545fa/test --cache-dir=./.zig-cache --seed=0xb3c4e7f2 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 259/260 tests passed (1 crashed) +test-cuda-host transitive failure ++- run test 87 pass, 1 crash (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/size-at-held.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/size-at-held.log new file mode 100644 index 0000000000..f18478a951 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/size-at-held.log @@ -0,0 +1,109 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/9e2b04ab82361c572294e1bdf2727314/test --cache-dir=./.zig-cache --seed=0xce915f2a --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0xce915f2a --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/3c8234e1c1e99935584410938bfa7f05/test --cache-dir=./.zig-cache --seed=0xce915f2a --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xce915f2a --listen=- + +test-glm-snapshots-host ++- run test 14 pass, 2 fail (16 total) +error: 'glm_snapshot_host.test.GLM snapshots: while a paged stream is current, a prefix's size is the engine's own sequence's, past its pages' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + expected 6291456, found 18446744073709551615 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245393 in expectEqualInner__func_965 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x1256974 in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:488:5: 0x1256a31 in test.GLM snapshots: while a paged stream is current, a prefix's size is the engine's own sequence's, past its pages (glm_snapshot_host.zig) + try t.expectEqual(own, f.e.stateBytes(4096)); + ^ +error: 'glm_snapshot_host.test.GLM snapshots: with the engine's own sequence on pages too, a prefix's size past a stream's pages is a window's' failed: + expected 6291456, found 18446744073709551615 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245393 in expectEqualInner__func_965 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x12464f6 in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:516:34: 0x12465c0 in test.GLM snapshots: with the engine's own sequence on pages too, a prefix's size past a stream's pages is a window's (glm_snapshot_host.zig) + for (ats, sizes) |at, s| try t.expectEqual(s, f.e.stateBytes(at)); + ^ +failed command: ./.zig-cache/o/3892422d2171543184fabc9bb6dcb96e/test --cache-dir=./.zig-cache --seed=0xce915f2a --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 258/260 tests passed (2 failed) +test-glm-snapshots-host transitive failure ++- run test 14 pass, 2 fail (16 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/size-mapped-only.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/size-mapped-only.log new file mode 100644 index 0000000000..b30e9775ad --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/size-mapped-only.log @@ -0,0 +1,109 @@ +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/deadb0aea7c95945e0e198ef8dce3caa/test --cache-dir=./.zig-cache --seed=0x12a51c00 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/6406764f821db2212e347971b30b7e18/test --cache-dir=./.zig-cache --seed=0x12a51c00 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x12a51c00 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0x12a51c00 --listen=- + +test-glm-snapshots-host ++- run test 14 pass, 2 fail (16 total) +error: 'glm_snapshot_host.test.GLM snapshots: while a paged stream is current, a prefix's size is the engine's own sequence's, past its pages' failed: + GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 + GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 + expected 6291456, found 18446744073709551615 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245393 in expectEqualInner__func_965 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x1256914 in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:488:5: 0x12569d1 in test.GLM snapshots: while a paged stream is current, a prefix's size is the engine's own sequence's, past its pages (glm_snapshot_host.zig) + try t.expectEqual(own, f.e.stateBytes(4096)); + ^ +error: 'glm_snapshot_host.test.GLM snapshots: with the engine's own sequence on pages too, a prefix's size past a stream's pages is a window's' failed: + expected 6291456, found 18446744073709551615 + /.local/share/zig/0.17.0/lib/std/testing.zig:93:17: 0x1245393 in expectEqualInner__func_965 (std.zig) + return error.TestExpectedEqual; + ^ + /.local/share/zig/0.17.0/lib/std/testing.zig:57:5: 0x12464f6 in expectEqual (glm_snapshot_host.zig) + return expectEqualInner(T, expected, actual); + ^ + /zt/kvp-mut/zig/glm_snapshot_host.zig:516:34: 0x12465c0 in test.GLM snapshots: with the engine's own sequence on pages too, a prefix's size past a stream's pages is a window's (glm_snapshot_host.zig) + for (ats, sizes) |at, s| try t.expectEqual(s, f.e.stateBytes(at)); + ^ +failed command: ./.zig-cache/o/51b8c7412c5810e652ebd87d22b9b3e2/test --cache-dir=./.zig-cache --seed=0x12a51c00 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 258/260 tests passed (2 failed) +test-glm-snapshots-host transitive failure ++- run test 14 pass, 2 fail (16 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/slot-zero-unpaged.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/slot-zero-unpaged.log new file mode 100644 index 0000000000..3eb56ba571 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/slot-zero-unpaged.log @@ -0,0 +1,128 @@ +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0x9664e595 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0x9664e595 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/4590eda8e18781893efc383efda4dcf2/test --cache-dir=./.zig-cache --seed=0x9664e595 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/607889dcb58796f2162b3fc63670876d/test --cache-dir=./.zig-cache --seed=0x9664e595 --listen=- + +test-cuda-host ++- run test 83 pass, 5 fail (88 total) +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +error: 'cuda_pages_test.test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back' failed: + GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 + GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 + GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 + GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x1270da9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:114:5: 0x155417f in test.a paged stream takes its request's pages, its table holds them in row order, and its end gives them back (cuda.zig) + try expect(p.paged(0) and p.paged(1) and p.paged(2) and !p.paged(3)); + ^ +error: 'cuda_pages_test.test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:135:45: 0x13f84a7 in open (cuda.zig) + if (!p.paged(slot) or p.used[slot]) return error.SlotTaken; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:147:56: 0x15533b1 in test.the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others (cuda.zig) + try std.testing.expectEqualSlices(u32, &.{ 0, 1 }, try p.open(0, 600, 100)); + ^ +error: 'cuda_pages_test.test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_lanes.zig:277:29: 0x13fb84d in pagesFor (cuda.zig) + if (!p.paged(slot)) return error.LaneMemory; // every paged sequence is taken + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_lanes.zig:268:13: 0x13fca47 in seat (cuda.zig) + return err; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:205:37: 0x13fe698 in test.TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window (cuda.zig) + try expectEqual(@as(usize, 2), (try ln.seat(&streams[0], 600, 100)).len); + ^ +error: 'cuda_pages_test.test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_lanes.zig:277:29: 0x13fb84d in pagesFor (cuda.zig) + if (!p.paged(slot)) return error.LaneMemory; // every paged sequence is taken + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_lanes.zig:268:13: 0x13fca47 in seat (cuda.zig) + return err; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:231:9: 0x13fd149 in test.TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not (cuda.zig) + _ = try ln.seat(&streams[0], 600, 100); // slot 0: 2 pages + ^ +error: 'cuda_pages_test.test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written' failed: + /zt/kvp-mut/zig/src/families/glm/cuda_pages.zig:135:45: 0x13f84a7 in open (cuda.zig) + if (!p.paged(slot) or p.used[slot]) return error.SlotTaken; + ^ + /zt/kvp-mut/zig/src/families/glm/cuda_pages_test.zig:246:9: 0x13f8d8d in test.TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written (cuda.zig) + _ = try p.open(0, 1200, 100); + ^ +failed command: ./.zig-cache/o/95c0ed63a0bfcfe40bb4f8ad3983669b/test --cache-dir=./.zig-cache --seed=0x9664e595 --listen=- + +Build Summary: 161/163 steps succeeded (1 failed); 255/260 tests passed (5 failed) +test-cuda-host transitive failure ++- run test 83 pass, 5 fail (88 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/mutations/split-no-slots.log b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/split-no-slots.log new file mode 100644 index 0000000000..a729774202 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/mutations/split-no-slots.log @@ -0,0 +1,441 @@ +test-kv-pool ++- run test 9 pass, 2 crash (11 total) +error: 'kv_pool_test.test.a family without snapshot slots (snap_bytes 0) keeps its whole cache as copied rows beside the pages' terminated with signal ABRT with stderr: + thread 868 panic: division by zero + /zt/kvp-mut/zig/src/core/kv_pool.zig:59:59: 0x124a32f in split (kv_pool.zig) + const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:59:23: 0x124f7e0 in test.a family without snapshot slots (snap_bytes 0) keeps its whole cache as copied rows beside the pages (kv_pool.zig) + const s = kv.split(16 * g, 0, null); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x11e78ea in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x11e871d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x11e5428 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x79ceba62a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x79ceba62a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x1276070 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +error: 'kv_pool_test.test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest' terminated with signal ABRT with stderr: + thread 881 panic: division by zero + /zt/kvp-mut/zig/src/core/kv_pool.zig:59:59: 0x124a32f in split (kv_pool.zig) + const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); + ^ + /zt/kvp-mut/zig/src/core/kv_pool.zig:73:20: 0x1249377 in plan (kv_pool.zig) + const c = split(@min(b.cache, left - fit * l.pageBytes()), l.snap_bytes, b.snaps); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:76:26: 0x124b218 in test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest (kv_pool.zig) + const p = try kv.plan(glm, .{ .room = 40 * g, .engine = 0, .lanes = 4, .cache = 16 * g, .window = window, .engine_paged = true }); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x11e78ea in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x11e871d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x11e5428 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7dd77102a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x7dd77102a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x1276070 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/d14c02b3c6b32f09f2b2bca324c2dd10/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test 0 pass, 1 fail (1 total) +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-mut/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql(u8, &first, &changed)); + ^ +failed command: ./.zig-cache/o/02332ec8b02017daef0b6f3ac2477a5f/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/4a85b2029e4b8e63dee66758ecdc52ac/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/18cd56a78b6d5537960d229a5a0168fa/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-glm-snapshots-host ++- run test w +GLM snapshot sizing rank=0 world=2 at=2048 bytes=203423744 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=34816 bytes=643825664 arena=7034250752 +GLM snapshot sizing rank=0 world=2 at=266240 bytes=3665821696 arena=7034250752 +failed command: ./.zig-cache/o/31b814cfad326797687c1b5a0862bd2a/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/cd95b0e257824408061bc5ebb06b5d89/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/c9051709c1cdb691ea97a5a035ff2e67/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test 71 pass, 2 crash (73 total) +error: 'cuda_kv_pool.test.kv pages: a family without snapshot slots keeps the paged plan's cache, not what window-sized streams leave' terminated with signal ABRT with stderr: + thread 1132 panic: division by zero + /zt/kvp-mut/zig/src/core/kv_pool.zig:59:59: 0x189a5cf in split (root.zig) + const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); + ^ + /zt/kvp-mut/zig/src/core/kv_pool.zig:73:20: 0x189cd57 in plan (root.zig) + const c = split(@min(b.cache, left - fit * l.pageBytes()), l.snap_bytes, b.snaps); + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:166:31: 0x18bca2c in test.kv pages: a family without snapshot slots keeps the paged plan's cache, not what window-sized streams leave (cuda.zig) + const p = try kv_pool.plan(l, .{ .room = 40 << 30, .engine = 0, .lanes = 4, .cache = 16 << 30 }); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x12faada in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x12fb90d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x12f8618 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x73355422a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x73355422a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x1b7dbb0 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +error: 'cuda_kv_pool.test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so' terminated with signal ABRT with stderr: + thread 1181 panic: division by zero + /zt/kvp-mut/zig/src/core/kv_pool.zig:59:59: 0x189a5cf in split (root.zig) + const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); + ^ + /zt/kvp-mut/zig/src/core/kv_pool.zig:73:20: 0x189cd57 in plan (root.zig) + const c = split(@min(b.cache, left - fit * l.pageBytes()), l.snap_bytes, b.snaps); + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:54:24: 0x18aeb1f in plan__func_3151 (cuda.zig) + return kv_pool.plan(l, .{ .room = room, .engine = 0, .lanes = o.lanes, .fixed = o.lanes_fixed, .cache = want, .snaps = named(o), .window = window + rows, .engine_paged = engine_paged }) catch |e| { + ^ + /zt/kvp-mut/zig/src/native/cuda_kv_pool.zig:185:24: 0x18b0e7b in test.kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so (cuda.zig) + const p = (try plan(Owned, a, l, 40 << 30, o, 8192, &problem, false, true)).?; + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x12faada in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x12fb90d in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x12f8618 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7edabd02a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x7edabd02a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x1b7dbb0 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/bd9f6e75b10111f93fa6d00de0c05e71/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/be431451037c06928c7e320997c63f24/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/8ea554a32cfa6810e83b0922a435f8e0/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/51dd570d00c8ab3edeeff6f6e13850e3/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test 350 pass, 2 crash (352 total) +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +error: 'kv_pool_test.test.a family without snapshot slots (snap_bytes 0) keeps its whole cache as copied rows beside the pages' terminated with signal ABRT with stderr: + thread 1200 panic: division by zero + /zt/kvp-mut/zig/src/core/kv_pool.zig:59:59: 0x13a5e4f in split (root.zig) + const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:59:23: 0x13aaed0 in test.a family without snapshot slots (snap_bytes 0) keeps its whole cache as copied rows beside the pages (root.zig) + const s = kv.split(16 * g, 0, null); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x127d9ba in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x127e7ed in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x127b4f8 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x7c6f39c2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x7c6f39c2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x18eb530 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +error: 'kv_pool_test.test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest' terminated with signal ABRT with stderr: + thread 3247 panic: division by zero + /zt/kvp-mut/zig/src/core/kv_pool.zig:59:59: 0x13a5e4f in split (root.zig) + const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); + ^ + /zt/kvp-mut/zig/src/core/kv_pool.zig:73:20: 0x13a4e97 in plan (root.zig) + const c = split(@min(b.cache, left - fit * l.pageBytes()), l.snap_bytes, b.snaps); + ^ + /zt/kvp-mut/zig/src/core/kv_pool_test.zig:76:26: 0x13a6d38 in test.the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest (root.zig) + const p = try kv.plan(glm, .{ .room = 40 * g, .engine = 0, .lanes = 4, .cache = 16 * g, .window = window, .engine_paged = true }); + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:143:68: 0x127d9ba in mainServer (test_runner.zig) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + /.local/share/zig/0.17.0/lib/compiler/test_runner.zig:72:26: 0x127e7ed in main (test_runner.zig) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + /.local/share/zig/0.17.0/lib/std/start.zig:789:88: 0x127b4f8 in callMain (std.zig) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x73ae8fc2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6) + ???:?:?: 0x73ae8fc2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6) + /.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S:115: 0x18eb530 in ??? (/.local/share/zig/0.17.0/lib/libc/glibc/sysdeps/x86_64/start.S) + call *__libc_start_main@GOTPCREL(%rip) +failed command: ./.zig-cache/o/84a6c88909a543a0d7bd081594ee7a13/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/806a785d488ee0821436b4df7319da20/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/b411c529933dc8818d1705e142a012a3/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/5941e8e99e8203fdaeea5581886456b4/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/c5408344010c513e4748c01b8985f672/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/8ab887e368b86eed12820e32292d38cf/test --cache-dir=./.zig-cache --seed=0xfa8b2c3a --listen=- + +Build Summary: 157/163 steps succeeded (4 failed); 1047/1058 tests passed (4 skipped, 1 failed, 6 crashed) +test-cuda-host transitive failure ++- run test 0 pass, 1 fail (1 total) ++- run test 350 pass, 2 crash (352 total) ++- run test 71 pass, 2 crash (73 total) +test-kv-pool transitive failure ++- run test 9 pass, 2 crash (11 total) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/security.log b/docs/receipts/glm-kv-pages-8e705835/ci/security.log new file mode 100644 index 0000000000..d5f8bde78f --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/security.log @@ -0,0 +1,38 @@ +PASS key required for /systemone +PASS valid key admits /systemone +PASS key required for /alternative/systemone +PASS valid key admits /alternative/systemone +PASS key required for /systemone/?x=1 +PASS valid key admits /systemone/?x=1 +PASS key required for /v1/systemone +PASS valid key admits /v1/systemone +PASS GET body cannot become another request +PASS GET chunked body stays framed +PASS DELETE body cannot become another request +PASS DELETE chunked body stays framed +PASS malformed header 0 closes before routing +PASS malformed header 1 closes before routing +PASS malformed header 2 closes before routing +PASS malformed header 3 closes before routing +PASS malformed header 4 closes before routing +PASS malformed header 5 closes before routing +PASS malformed header 6 closes before routing +PASS malformed header 7 closes before routing +PASS truncated request head cannot dispatch +PASS GET conflicting framing is rejected +PASS GET body preserves a genuine following request +PASS ordinary pipelining remains supported +SUMMARY server-security: 24 passed, 0 failed, 0 skipped +Build Summary: 12/12 steps succeeded; 20/20 tests passed +test-pull-security success ++- run test 1 pass (1 total) 31ms MaxRSS:6M +| +- compile test debug native success 1s MaxRSS:147M ++- run test 4 pass (4 total) 3s MaxRSS:9M +| +- compile test debug native success 3s MaxRSS:204M ++- run test 1 pass (1 total) 36ms MaxRSS:6M + +- compile test debug native success 1s MaxRSS:153M +test-server-security success ++- run python3 success 340ms +| +- compile exe fake_serve debug native cached 58ms MaxRSS:39M ++- run test 14 pass (14 total) 21ms MaxRSS:6M + +- compile test debug native success 1s MaxRSS:171M diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/test-golden.log b/docs/receipts/glm-kv-pages-8e705835/ci/test-golden.log new file mode 100644 index 0000000000..bfe9443ea2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/test-golden.log @@ -0,0 +1,82 @@ + keys 6 equal 0 differ 0 known + origin 20 equal 0 differ 0 known +26/26 equal to server goldens, 0 unexpected differences +KNOWN-DIFFER tools/two_tools-stream + reply 0 body differs at 313: golden [content": "\n"}, "finish_reason": null}]} + +data: {"id": "", "type": "function"] +KNOWN-DIFFER tools/two_tools-single-stream + reply 0 body differs at 313: golden [content": "\n"}, "finish_reason": null}]} + +data: {"id": "", "type": "function"] +KNOWN-DIFFER tools/choice-"required" + reply 0 status: golden HTTP/1.1 200 OK, zig HTTP/1.1 400 Bad Request + reply 0 body differs at 2: golden [id": "", "object": "chat.completion", "created": , ] zig [error": {"message": "backend 'lanes' cannot enforce tool_cho] + reply 0 headers differ +KNOWN-DIFFER tools/choice-{"type": "function", "function": {"name": "lookup"}} + reply 0 status: golden HTTP/1.1 200 OK, zig HTTP/1.1 400 Bad Request + reply 0 body differs at 2: golden [id": "", "object": "chat.completion", "created": , ] zig [error": {"message": "backend 'lanes' cannot enforce tool_cho] + reply 0 headers differ +KNOWN-DIFFER errors/format-json + reply 0 body differs at 41: golden [needs the checkpoint's config.json vocab_size", "type": "inv] zig [(response_format and the guided_* fields) is not supported b] + reply 0 headers differ +KNOWN-DIFFER errors/stream-structured + connection after: golden closed, zig open + reply 0 status: golden HTTP/1.1 200 OK, zig HTTP/1.1 400 Bad Request + reply 0 body differs at 0: golden [data: {"id": "", "object": "chat.completion.chunk", "cr] zig [{"error": {"message": "structured output (response_format an] +KNOWN-DIFFER anthropic/tool-use-stream + reply 0 body differs at 704: golden [\n"}} + +event: content_block_stop +data: {"type": "content_blo] zig ["}} + +event: content_block_stop +data: {"type": "content_block] +KNOWN-DIFFER anthropic/tool-result + reply 0 status: golden HTTP/1.1 200 OK, zig HTTP/1.1 400 Bad Request + reply 0 body differs at 2: golden [id": "", "type": "message", "role": "assistant", "model] zig [type": "error", "error": {"type": "invalid_request_error", "] + reply 0 headers differ +KNOWN-DIFFER anthropic/refuse-format + reply 0 body differs at 91: golden [needs the checkpoint's config.json vocab_size"}}] zig [(response_format and the guided_* fields) is not supported b] + reply 0 headers differ +KNOWN-DIFFER cancel/metrics-after + reply 0 body differs at 307: golden [prompt_tokens_total Prompt tokens of finished requests. +# TY] zig [generation_tokens_running Generated tokens held by live stre] + reply 0 headers differ +KNOWN-DIFFER metrics-open/metrics + reply 0 body differs at 307: golden [prompt_tokens_total Prompt tokens of finished requests. +# TY] zig [generation_tokens_running Generated tokens held by live stre] + reply 0 headers differ +KNOWN-DIFFER metrics-open/v1-metrics + reply 0 body differs at 487: golden [prompt_tokens_total Prompt tokens of finished requests. +# TY] zig [generation_tokens_running Generated tokens held by live stre] + reply 0 headers differ +KNOWN-DIFFER keys/metrics-counted + reply 0 body differs at 887: golden [prompt_tokens_total Prompt tokens of finished requests. +# TY] zig [generation_tokens_running Generated tokens held by live stre] + reply 0 headers differ + chat 16 equal 0 differ 0 known + thinking 16 equal 0 differ 0 known + stops 13 equal 0 differ 0 known + usage 10 equal 0 differ 0 known + tools 48 equal 0 differ 4 known + errors 37 equal 0 differ 2 known + completions 15 equal 0 differ 0 known + responses 29 equal 0 differ 0 known + anthropic 26 equal 0 differ 3 known + tokens 19 equal 0 differ 0 known + status 11 equal 0 differ 0 known + framing 42 equal 0 differ 0 known + cancel 1 equal 0 differ 1 known + lifecycle 2 equal 0 differ 0 known +metrics-open 2 equal 0 differ 2 known + keys 19 equal 0 differ 1 known +306/319 equal to server goldens, 0 unexpected differences +Build Summary: 5/5 steps succeeded +test-golden success ++- run exe golden_check success 1m +| +- compile exe golden_check debug native success 850ms MaxRSS:150M +| +- compile exe fake_serve debug native success 4s MaxRSS:319M ++- run exe golden_check success 4s + +- compile exe golden_check debug native (reused) + +- compile exe fake_serve debug native (reused) diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/test-kernel-contract.log b/docs/receipts/glm-kv-pages-8e705835/ci/test-kernel-contract.log new file mode 100644 index 0000000000..7111286962 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/test-kernel-contract.log @@ -0,0 +1,12 @@ +Build Summary: 11/11 steps succeeded; 72/72 tests passed +test-kernel-contract success ++- run test 53 pass (53 total) 997ms MaxRSS:10M +| +- compile test debug native success 5s MaxRSS:167M ++- test-gemma4-contract-host success +| +- run test 9 pass (9 total) 38s MaxRSS:14M +| +- compile test debug native success 1s MaxRSS:227M +| +- WriteFile gemma4-host-empty.metallib success ++- run test 3 pass (3 total) 21ms MaxRSS:6M +| +- compile test debug native success 933ms MaxRSS:149M ++- run test 7 pass (7 total) 1s MaxRSS:55M + +- compile test debug native success 1s MaxRSS:151M diff --git a/docs/receipts/glm-kv-pages-8e705835/ci/test.log b/docs/receipts/glm-kv-pages-8e705835/ci/test.log new file mode 100644 index 0000000000..44e18a0c50 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/ci/test.log @@ -0,0 +1,901 @@ +test ++- test-cuda-host + +- run test w +[default] (warn): slide pocket: a deleted fact's 1 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a deleted fact's 1 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a delete failed (UploadFailed), and its put-back did not reach the device (PutBackNotOnDevice) +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a delete failed (UploadFailed), and its put-back did not reach the device (PutBackNotOnDevice) +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +failed command: ./.zig-cache/o/e3d115154491929e494aec103397c87f/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/f1b7dd59cf21e84f4db4b59615b85a48/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/ba389278848dd62bdeff7f7a7b458416/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +DEPTH_REPLAY n=2 intercept=50 slope=3 head=1 probe=false deepest=6 crossed={ 62, 62 } last={ 6, 6 } trimmed=0 +DEPTH_REPLAY n=2 intercept=50 slope=3 head=1 probe=true deepest=8 crossed={ 59, 57 } last={ 7, 8 } trimmed=11 +DEPTH_REPLAY n=4 intercept=50 slope=3 head=1 probe=false deepest=4 crossed={ 83, 83, 83, 83 } last={ 4, 4, 4, 4 } trimmed=0 +DEPTH_REPLAY n=4 intercept=50 slope=3 head=1 probe=true deepest=5 crossed={ 75, 83, 83, 76 } last={ 5, 5, 5, 5 } trimmed=9 +DEPTH_REPLAY n=2 intercept=55 slope=2.8 head=1 probe=false deepest=6 crossed={ 61, 61 } last={ 6, 6 } trimmed=0 +DEPTH_REPLAY n=2 intercept=55 slope=2.8 head=1 probe=true deepest=8 crossed={ 54, 56 } last={ 8, 8 } trimmed=2 +DEPTH_REPLAY n=4 intercept=55 slope=2.8 head=1 probe=false deepest=4 crossed={ 82, 82, 82, 82 } last={ 4, 4, 4, 4 } trimmed=0 +DEPTH_REPLAY n=4 intercept=55 slope=2.8 head=1 probe=true deepest=6 crossed={ 75, 71, 72, 74 } last={ 5, 6, 5, 5 } trimmed=9 +DEPTH_REPLAY n=8 intercept=100 slope=0 head=0 probe=false deepest=3 crossed={ 100, 100, 100, 100, 100, 100, 100, 100 } last={ 3, 3, 3, 3, 3, 3, 3, 3 } trimmed=0 +DEPTH_REPLAY n=8 intercept=100 slope=0 head=0 probe=true deepest=3 crossed={ 100, 100, 100, 100, 100, 100, 100, 100 } last={ 3, 3, 3, 3, 3, 3, 3, 3 } trimmed=0 +failed command: ./.zig-cache/o/e1835a97d62c9729c6e0d24440f0ea35/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/46ab26c0f957a0f6c71df9e01db15533/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/b04b5cfc81a4382b056d0bf0dc2e8cba/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/da15d3c41ea5d2153781660b047e67e6/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +[default] (err): rank 0: another rank's process ended (its line closed); this rank stops instead of waiting on it +[default] (err): rank 0: rank 1 stopped (peer OOM); this rank stops too +failed command: ./.zig-cache/o/8000cd26070d78c69d965bddff4a4b1e/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/f212d84e453b6f0cd7cad1de23a8ec5e/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +failed command: ./.zig-cache/o/92af39c21a90673092679820f4bb048b/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): libcublasLt.so.13 is required for CUDA vision towers (--vision) and CUDA cuBLASLt projections; install the NVIDIA CUDA 13 cuBLAS runtime; on Linux, add its directory to LD_LIBRARY_PATH; a driver-only host does not include it +[default] (warn): libnccl.so.2 is required for CUDA ranks (including the GLM pair); install the NVIDIA NCCL 2 runtime compatible with CUDA 13; on Linux, add its directory to LD_LIBRARY_PATH +[default] (warn): libibverbs.so.1 is required for RoCE/RDMA gathers; install rdma-core's libibverbs runtime and the provider for the network adapter +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +failed command: ./.zig-cache/o/f26a02eefb66879c6e2c3a9dc0402315/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/826bec6bc03435dcc4d3f85f6a062ce2/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test ++- test-cuda-host + +- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/3983434796fd5597d682d8fcf0216c28/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test 0 pass, 1 fail (1 total) + the same path invalidates it (cuda_cost_identity.zig) + try std.testing.expect(!std.mem.eql(u8, &first, &changed)); + ^ +error: 'cuda_cost_identity.test.cost identity: stable reads reuse a drafter, while a changed payload at the same path invalidates it' failed: + /.local/share/zig/0.17.0/lib/std/testing.zig:584:14: 0x123efa9 in expect (std.zig) + if (!ok) return error.TestUnexpectedResult; + ^ + /zt/kvp-r9c/zig/src/families/qwen3_5/cuda_cost_identity.zig:67:5: 0x123fa53 in test.cost identity: stable reads reuse a drafter, while a changed payload at +failed command: ./.zig-cache/o/50d121a9ab9d688f31bd98df2c908ad4/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): lane host: stopping at a round boundary: 2 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): 4 prompts together: SamplingRefused; each goes alone +[default] (warn): lane host: a round failed (the planner refused): every admitted stream fails +[default] (warn): lane host: a round failed (PlannerFull): every admitted stream fails +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (drain deadline) +[default] (warn): lane host: stopping at a round boundary: 0 running and 1 queued request(s) end (NoContext) +[default] (warn): lane host: a round failed (OutOfMemory): every admitted stream fails +[default] (warn): lane host: a round failed (AdmittedFailed): every admitted stream fails +[default] (warn): lane host: a round failed (FinalsFailed): every admitted stream fails +failed command: ./.zig-cache/o/9d16dac97cd5dc37476baef383e62487/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/f9419dba1a4fe6ebd6e84eb94f7dcebf/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +failed command: ./.zig-cache/o/39948fb54177a7eb0a6235aad72245ce/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- run test w +[default] (warn): glm: restoring 1500 tokens failed (PoolMoveNotDrained); pool relocation aborted +failed command: ./.zig-cache/o/86c563b75b7beeb0cea5736fc8837245/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- run test w +[default] (warn): XPU: a command list did not drain at close; its memory and context stay until the process ends +failed command: ./.zig-cache/o/c4054c8ba5e1acde7faea349e8073ec3/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +failed command: ./.zig-cache/o/003b37803413a97189099bed5210dfa2/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/793d6734cfc114163bf1a61b2669885d/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/300e9d9bc5cc2177d3c6380f0e8a7c36/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- run test w +[default] (warn): a round could not be recorded: OutOfDeviceMemory +failed command: ./.zig-cache/o/85f697be90ab2a912b67eb074b4ef0f2/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/f5874ec54c4a99c0402cf2abdb749534/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- test-cuda-host + +- run test w +PROVENANCE_FIELDS fixtures=3 fields=14 plus decision labels +PROVENANCE_TIMING mode=raw one=0 bytes=409600 added=1000 rounds=12 ns_per=28778465 +PROVENANCE_TIMING mode=masked one=0 bytes=409600 added=1000 rounds=12 ns_per=29583226 +PROVENANCE_MASK_WORK one=0 spans=3 matcher=3 output=409603 +PROVENANCE_TIMING mode=raw one=1 bytes=409600 added=1000 rounds=12 ns_per=25882607 +PROVENANCE_TIMING mode=masked one=1 bytes=409600 added=1000 rounds=12 ns_per=43887156 +PROVENANCE_MASK_WORK one=1 spans=3 matcher=3 output=409603 +[default] (warn): TENSORFOLD_REQUEST_LOG .zig-cache/tmp/Y9eBuDbzDMDbFtiV/taken.jsonl gets no requests: it is not a regular file only its owner can read +VISION_MARKERS all 8 fields refuse plain vocabulary; unflagged additions pass +failed command: ./.zig-cache/o/8c26e0cde058da70f2dbee57f29a6773/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- run test w +[default] (warn): flash next: rotary cache table unavailable: OutOfMemory +failed command: ./.zig-cache/o/1a473237d1660ef55b1b64cf4464d721/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +test ++- run test w +[default] (warn): lane host: stopping at a round boundary: 2 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): 4 prompts together: SamplingRefused; each goes alone +[default] (warn): lane host: a round failed (the planner refused): every admitted stream fails +[default] (warn): lane host: a round failed (PlannerFull): every admitted stream fails +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (drain deadline) +[default] (warn): lane host: stopping at a round boundary: 0 running and 1 queued request(s) end (NoContext) +[default] (warn): lane host: a round failed (OutOfMemory): every admitted stream fails +[default] (warn): lane host: a round failed (AdmittedFailed): every admitted stream fails +[default] (warn): lane host: a round failed (FinalsFailed): every admitted stream fails +failed command: ./.zig-cache/o/51eeee2cc21ac2488f1fc5933657e2be/test --cache-dir=./.zig-cache --seed=0xaeadb87e --listen=- + +Build Summary: 273/276 steps succeeded (1 failed); 3470/3486 tests passed (15 skipped, 1 failed) +test transitive failure ++- test-kernel-symbols-host success +| +- run test 3 pass (3 total) 15s MaxRSS:77M +| +- compile test debug native success 3s MaxRSS:146M ++- test-checkpoint-security success +| +- run test 32 pass (32 total) 321ms MaxRSS:14M +| | +- compile test safe native success 50s MaxRSS:548M +| +- run test 5 pass (5 total) 10ms MaxRSS:14M +| | +- compile test safe native success 41s MaxRSS:461M +| +- run test 6 pass (6 total) 8ms MaxRSS:15M +| +- compile test safe native success 36s MaxRSS:416M ++- run test 37 pass (37 total) 147ms MaxRSS:12M +| +- compile test debug native success 2s MaxRSS:155M ++- run test 4 pass (4 total) 74ms MaxRSS:15M +| +- compile test fast x86_64-linux-gnu success 25s MaxRSS:392M ++- run test 24 pass (24 total) 5s MaxRSS:59M +| +- compile test debug native success 4s MaxRSS:157M ++- run test 13 pass (13 total) 830ms MaxRSS:12M +| +- compile test debug native success 1s MaxRSS:148M +| +- options success +| +- compile exe digest-writer debug native success 1s MaxRSS:143M ++- run test 6 pass (6 total) 27ms MaxRSS:14M +| +- compile test debug native success 1s MaxRSS:145M ++- test-flashnext-scheduling-host success +| +- run test 30 pass, 1 skip (31 total) 31s MaxRSS:20M +| +- compile test debug native success 5s MaxRSS:192M +| +- WriteFile flashnext-test-empty.zig success ++- run test 15 pass (15 total) 94ms MaxRSS:15M +| +- compile test debug native success 3s MaxRSS:161M +| +- WriteFile flashnext-test-empty.zig (reused) ++- test-flashnext-batch-host success +| +- run test 14 pass (14 total) 5s MaxRSS:34M +| +- compile test debug native success 3s MaxRSS:184M +| +- WriteFile flashnext-test-empty.zig (reused) ++- run test 2 pass (2 total) 618ms MaxRSS:15M +| +- compile test debug native success 3s MaxRSS:145M ++- run test 4 pass (4 total) 30ms MaxRSS:16M +| +- compile test debug native success 1s MaxRSS:143M ++- run test 4 pass (4 total) 87ms MaxRSS:15M +| +- compile test debug native success 3s MaxRSS:147M ++- run test 3 pass (3 total) 27ms MaxRSS:12M +| +- compile test debug native success 1s MaxRSS:143M ++- run test 23 pass (23 total) 53ms MaxRSS:13M +| +- compile test debug native success 2s MaxRSS:148M ++- run test 29 pass (29 total) 8s MaxRSS:24M +| +- compile test debug native success 1s MaxRSS:162M ++- run test 2 pass (2 total) 7ms MaxRSS:16M +| +- compile test fast native success 47s MaxRSS:419M ++- run test 59 pass, 1 skip (60 total) 56s MaxRSS:51M +| +- compile test debug native success 2s MaxRSS:166M +| | +- options cached +| +- compile obj tp_host_compile debug x86_64-linux-gnu success 931ms MaxRSS:109M +| +- options cached ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native success 1m MaxRSS:658M +| +- WriteFile tp_kernel.zig success +| +- options cached ++- run exe tf-tp-test success 3ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 5ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 3ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 5ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 7ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 10ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 6ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- test-cuda-host transitive failure +| +- run test 1 pass (1 total) 28ms MaxRSS:7M +| | +- compile test debug native success 3s MaxRSS:158M +| +- run test 1 pass (1 total) 29ms MaxRSS:10M +| | +- compile test debug native success 1s MaxRSS:143M +| +- run test 10 pass (10 total) 53ms MaxRSS:11M +| | +- compile test debug native success 2s MaxRSS:155M +| +- run test 2 pass (2 total) 102ms MaxRSS:11M +| | +- compile test debug native success 2s MaxRSS:148M +| +- run exe runtime-discovery-exit success 44ms +| | +- compile exe runtime-discovery-exit debug native success 1s MaxRSS:144M +| | +- options success +| | +- compile lib runtime-discovery-mock debug native success 2s MaxRSS:70M +| +- run exe rpc-warning-check success 4ms +| | +- compile exe rpc-warning-check debug native success 2s MaxRSS:140M +| | +- options success +| | +- compile lib rpc-mock-complete debug native success 369ms MaxRSS:61M +| | +- compile lib rpc-mock-abi debug native success 1s MaxRSS:62M +| | +- compile lib rpc-mock-wait debug native success 391ms MaxRSS:62M +| | +- compile lib rpc-mock-store debug native success 1s MaxRSS:62M +| | +- compile lib rpc-mock-wrong debug native success 346ms MaxRSS:62M +| +- run test 11 pass (11 total) 38ms MaxRSS:10M +| | +- compile test debug native success 2s MaxRSS:150M +| | +- options (+5 more reused dependencies) +| +- run test 1 pass (1 total) 40ms MaxRSS:9M +| | +- compile test debug native success 2s MaxRSS:143M +| +- run test 30 pass (30 total) 129ms MaxRSS:8M +| | +- compile test debug native success 4s MaxRSS:228M +| | +- WriteFile h3-compile-only.fatbin success +| | +- options success +| +- run test 3 pass (3 total) 1s MaxRSS:74M +| | +- compile test debug native success 3s MaxRSS:162M +| +- run test 3 pass (3 total) 23ms MaxRSS:11M +| | +- compile test debug native success 10s MaxRSS:170M +| | +- WriteFile attn_train.fatbin success +| | +- WriteFile dsv41_exl3_experts.fatbin success +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- options success +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- options success +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| +- run test 3 pass (3 total) 310ms MaxRSS:8M +| | +- compile test debug native success 2s MaxRSS:177M +| | +- options success +| | | +- compile exe qwen-round-timing-probe debug native success 2s MaxRSS:179M +| | | +- WriteFile draft_ids.txt success +| | | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 10 pass (10 total) 34ms MaxRSS:6M +| | +- compile test debug native success 2s MaxRSS:170M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 7 pass (7 total) 44ms MaxRSS:9M +| | +- compile test debug native success 2s MaxRSS:178M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test 6 pass (6 total) 45ms MaxRSS:9M +| | +- compile test debug native success 1s MaxRSS:175M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test 10 pass (10 total) 1s MaxRSS:8M +| | +- compile test debug native success 2s MaxRSS:196M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 9 pass (9 total) 227ms MaxRSS:8M +| | +- compile test debug native success 2s MaxRSS:171M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- compile obj exl3_host_compile debug native success 2s MaxRSS:127M +| | +- options (reused) +| +- run test 6 pass (6 total) 27ms MaxRSS:9M +| | +- compile test debug native success 1s MaxRSS:167M +| | +- options (reused) +| +- run test 6 pass (6 total) 346ms MaxRSS:9M +| | +- compile test debug native success 2s MaxRSS:162M +| | +- options (reused) +| +- run test 5 pass (5 total) 77ms MaxRSS:6M +| | +- compile test debug native success 2s MaxRSS:160M +| | +- options (reused) +| +- run test 3 pass (3 total) 53ms MaxRSS:6M +| | +- compile test debug native success 2s MaxRSS:158M +| | +- options (reused) +| +- run test 3 pass (3 total) 17ms MaxRSS:5M +| | +- compile test debug native success 1s MaxRSS:156M +| | +- options (reused) +| +- run test 3 pass (3 total) 26ms MaxRSS:8M +| | +- compile test debug native success 1s MaxRSS:156M +| | +- options (reused) +| +- run test 8 pass (8 total) 68ms MaxRSS:11M +| | +- compile test debug native success 2s MaxRSS:155M +| +- run test 3 pass (3 total) 468ms MaxRSS:8M +| | +- compile test debug native success 2s MaxRSS:153M +| +- run test 8 pass (8 total) 106ms MaxRSS:6M +| | +- compile test debug native success 2s MaxRSS:156M +| +- run test 4 pass (4 total) 31ms MaxRSS:13M +| | +- compile test debug native success 1s MaxRSS:160M +| | +- options (reused) +| +- run test 1 pass (1 total) 105ms MaxRSS:8M +| | +- compile test debug native success 2s MaxRSS:161M +| | +- options (reused) +| +- run test 4 pass (4 total) 4ms MaxRSS:13M +| | +- compile test fast native success 53s MaxRSS:431M +| | +- options (reused) +| +- run test 1 pass (1 total) 41ms MaxRSS:13M +| | +- compile test fast native success 56s MaxRSS:439M +| | +- options (reused) +| +- compile obj bolt_host_compile debug x86_64-linux-gnu success 2s MaxRSS:144M +| | +- options cached +| | +- WriteFile draft_ids.txt (reused) +| +- run test 26 pass (26 total) 8s MaxRSS:9M +| | +- compile test debug native success 2s MaxRSS:177M +| | +- options (reused) +| +- run test 4 pass (4 total) 26ms MaxRSS:6M +| | +- compile test debug native success 2s MaxRSS:170M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 3 pass (3 total) 44s MaxRSS:41M +| | +- compile test debug native success 3s MaxRSS:218M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run exe qwen-cost-key-process success 11s +| | +- compile exe qwen-cost-key-process debug native success 1s MaxRSS:174M +| | | +- WriteFile draft_ids.txt (reused) +| | | +- options (reused) +| | +- WriteFile target/config.json success +| +- run test 0 pass, 1 fail (1 total) +| | +- compile test debug native success 1s MaxRSS:162M +| +- compile obj drama_cuda_native_host debug native success 2s MaxRSS:130M +| | +- options (reused) +| +- run test 5 pass (5 total) 40ms MaxRSS:10M +| | +- compile test debug native success 2s MaxRSS:146M +| +- run test 5 pass (5 total) 317ms MaxRSS:7M +| | +- compile test debug native success 2s MaxRSS:163M +| +- run test 4 pass (4 total) 24ms MaxRSS:9M +| | +- compile test debug native success 2s MaxRSS:166M +| +- run test 604 pass (604 total) 1m MaxRSS:57M +| | +- compile test debug native success 16s MaxRSS:458M +| +- run test 5 pass (5 total) 27ms MaxRSS:11M +| | +- compile test debug native success 3s MaxRSS:148M +| +- run test 218 pass (218 total) 740ms MaxRSS:10M +| | +- compile test debug native success 3s MaxRSS:183M +| | +- options (reused) +| +- run test 352 pass (352 total) 24s MaxRSS:136M +| | +- compile test debug native success 11s MaxRSS:232M +| +- run test 212 pass (212 total) 18s MaxRSS:46M +| | +- compile test debug native success 4s MaxRSS:189M +| +- run test 74 pass, 1 skip (75 total) 369ms MaxRSS:14M +| | +- compile test debug native success 2s MaxRSS:220M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 58 pass, 2 skip (60 total) 19s MaxRSS:28M +| | +- compile test debug native success 2s MaxRSS:184M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 12 pass (12 total) 51ms MaxRSS:7M +| | +- compile test debug native success 2s MaxRSS:175M +| | +- options (reused) +| +- run test 97 pass (97 total) 952ms MaxRSS:79M +| | +- compile test debug native success 3s MaxRSS:221M +| | +- options (reused) +| +- run test 15 pass (15 total) 13s MaxRSS:13M +| | +- compile test debug native success 2s MaxRSS:184M +| | +- options (reused) +| +- run test 29 pass (29 total) 12s MaxRSS:1G +| | +- compile test debug native success 3s MaxRSS:176M +| | +- options (reused) +| +- run test 5 pass (5 total) 71ms MaxRSS:6M +| | +- compile test debug native success 2s MaxRSS:147M +| +- run test 36 pass (36 total) 639ms MaxRSS:8M +| | +- compile test debug native success 3s MaxRSS:166M +| +- run test 238 pass (238 total) 18s MaxRSS:45M +| | +- compile test debug native success 3s MaxRSS:210M +| +- run test 73 pass (73 total) 462ms MaxRSS:33M +| | +- compile test debug native success 7s MaxRSS:285M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test 3 pass (3 total) 62ms MaxRSS:11M +| | +- compile test debug native success 1s MaxRSS:144M +| +- run test 1 pass (1 total) 35ms MaxRSS:11M +| | +- compile test debug native success 2s MaxRSS:153M +| +- run test 12 pass (12 total) 64ms MaxRSS:7M +| | +- compile test debug native success 2s MaxRSS:169M +| | +- options (reused) +| +- run test 9 pass (9 total) 39ms MaxRSS:6M +| | +- compile test debug native success 2s MaxRSS:173M +| | +- options (reused) +| +- run test 2 pass (2 total) 66ms MaxRSS:9M +| | +- compile test debug native success 3s MaxRSS:208M +| | +- options (reused) +| +- run test 88 pass (88 total) 230ms MaxRSS:18M +| | +- compile test debug native success 2s MaxRSS:206M +| | +- options (reused) +| +- run test 19 pass, 1 skip (20 total) 99ms MaxRSS:8M +| | +- compile test debug native success 3s MaxRSS:178M +| | +- options (reused) +| +- run test 5 pass (5 total) 24ms MaxRSS:11M +| | +- compile test debug native success 2s MaxRSS:146M +| +- run test 34 pass (34 total) 26s MaxRSS:12M +| | +- compile test debug native success 3s MaxRSS:178M +| +- run test 55 pass (55 total) 3s MaxRSS:14M +| | +- compile test debug native success 3s MaxRSS:177M +| +- run test 50 pass, 1 skip (51 total) 4s MaxRSS:11M +| | +- compile test debug native success 3s MaxRSS:172M +| +- run test 1 pass (1 total) 21ms MaxRSS:6M +| +- compile test debug native success 2s MaxRSS:186M +| +- options success +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile exlr8_alias.zig success ++- run test 4 pass (4 total) 27ms MaxRSS:12M +| +- compile test debug native success 1s MaxRSS:145M +| +- options cached ++- run test 13 pass, 1 skip (14 total) 95ms MaxRSS:12M +| +- compile test debug native success 2s MaxRSS:165M +| +- options (reused) ++- run test 3 pass (3 total) 26ms MaxRSS:12M +| +- compile test debug native success 1s MaxRSS:157M +| +- options (reused) ++- run test 1 pass (1 total) 20ms MaxRSS:12M +| +- compile test debug native success 1s MaxRSS:145M ++- run test 238 pass (238 total) 3s MaxRSS:16M +| +- compile test fast native success 1m MaxRSS:1G ++- test-kv-pool success +| +- run test 11 pass (11 total) 27ms MaxRSS:6M +| | +- compile test debug native success 3s MaxRSS:146M +| +- run test 6 pass (6 total) 1s MaxRSS:9M +| +- compile test debug native success 3s MaxRSS:179M ++- run test 51 pass (51 total) 84ms MaxRSS:14M +| +- compile test debug native success 1s MaxRSS:167M +| +- options success +| | +- compile lib hip-mock-success debug native success 362ms MaxRSS:62M +| | +- compile lib hip-mock-failed debug native success 326ms MaxRSS:62M +| | +- compile lib hip-mock-missing debug native success 351ms MaxRSS:62M +| +- WriteFile kernels.zig cached ++- run test 23 pass, 1 skip (24 total) 151ms MaxRSS:16M +| +- compile test debug native success 2s MaxRSS:168M ++- run test 49 pass (49 total) 482ms MaxRSS:13M +| +- compile test debug native success 2s MaxRSS:170M +| +- WriteFile kernels.zig success ++- run test 35 pass, 1 skip (36 total) 172ms MaxRSS:13M +| +- compile test debug native success 8s MaxRSS:214M +| +- WriteFile kernels.zig (reused) ++- run test 19 pass (19 total) 1s MaxRSS:14M +| +- compile test debug native success 9s MaxRSS:218M +| +- WriteFile kernels.zig (reused) ++- run test 4 pass (4 total) 42ms MaxRSS:12M +| +- compile test debug native success 1s MaxRSS:161M +| +- WriteFile kernels.zig (reused) ++- run test 4 pass (4 total) 55ms MaxRSS:13M +| +- compile test debug native success 2s MaxRSS:169M +| +- WriteFile kernels.zig (reused) ++- compile exe tf-qwen35-test debug native success 8s MaxRSS:167M +| +- WriteFile kernels.zig (reused) ++- compile exe tensorfold-native safe native success 19s MaxRSS:252M +| +- options (reused) +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile kernels.zig (reused) ++- run test 31 pass, 5 skip (36 total) 35ms MaxRSS:16M +| +- compile test safe native success 1m MaxRSS:610M ++- run test 4 pass (4 total) 34ms MaxRSS:6M +| +- compile test debug native success 3s MaxRSS:144M ++- run test 121 pass (121 total) 12s MaxRSS:42M +| +- compile test debug native success 4s MaxRSS:170M ++- run test 22 pass (22 total) 321ms MaxRSS:14M +| +- compile test debug native success 1s MaxRSS:169M ++- run exe learned-warning-test success 65ms + +- compile exe learned-warning-test debug native success 1s MaxRSS:153M diff --git a/docs/receipts/glm-kv-pages-8e705835/commit.txt b/docs/receipts/glm-kv-pages-8e705835/commit.txt new file mode 100644 index 0000000000..cddb41c1df --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/commit.txt @@ -0,0 +1,13 @@ +tested implementation: 8e70583591644011bbb680fd3f2ae6e17a16b7f3 (commit 20: #1163's 14 commits and this pull request's first six) +implementation tree: ae42919c8f46bf12e4ccc2f399eba299b94dd3d2 +base: #1163's head d7ff6494dcd3bd9164d223d710a4dcc4659590da, on main after v1.1.2 f8d254c5e0f9ab91eb325a2e7c28d2e5cb07c94d +the no-GPU CI and the mutations: on commit 20 +after it: the receipt's commit adds this folder +TP=3's ranks: 10698886af0b9db9acfda5687d5a94b9b9059a95, commit 20 with the two commits in tp3/ring.patch + +62e2aeb3905d7f067f1bc4a5550a023bba929584 kv pool: the engine's own sequence can take pages too, a window of them planned before the other streams +7d4f8dc854f9304bd18ccb9d82dcf36035666878 native cuda: an engine whose own rows are pages plans a window of them first and says so at startup +8737d30dfaa0b03e81fa09bb35074b474b69d0cb glm cuda: the plan's window holds a request's last verify window, so its pages are a stream's whole table +b216a74b808018e68fc1d9d8b044c124beae7545 glm cuda: a kept prefix's size is counted to the capacity, so it needs no whole window to measure +27feda8b637dfe4b3c4da0ae40c9e847de53dff6 glm cuda: the engine's own sequence takes pages too, so its window's pages serve the other streams while it is short +8e70583591644011bbb680fd3f2ae6e17a16b7f3 glm cuda tests: the paged check's stream is the engine's own sequence, on pages from its init diff --git a/docs/receipts/glm-kv-pages-8e705835/records-sha256.txt b/docs/receipts/glm-kv-pages-8e705835/records-sha256.txt new file mode 100644 index 0000000000..1a51783e56 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/records-sha256.txt @@ -0,0 +1,3 @@ +30754de9fadf6035f2c9e1998a4efdc88fb03b029690fd732e79ad00077ad8f0 3121337 tp2/serve/r2all/rows-without-bodies.jsonl +b76dce4de6ace42730b10a0755cb2b414d302a478b2e3dec1d00350e1643a01e 3121339 tp2/serve/r2on/rows-without-bodies.jsonl +075037e948a4f8770281ed7229ea53467e434c7072e3701ecde86a841415507d 3054316 tp3/serve/r6auto/rows-without-bodies.jsonl diff --git a/docs/receipts/glm-kv-pages-8e705835/redaction-report.json b/docs/receipts/glm-kv-pages-8e705835/redaction-report.json new file mode 100644 index 0000000000..ed1870bc4e --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/redaction-report.json @@ -0,0 +1,170 @@ +{ + "policy": "CRLF line ends made LF and trailing blanks dropped; machine names, internal addresses, home and local paths, and the work's session, plan, records, container and image names replaced with placeholders (host-a, host-b and host-c run TP=3's ranks 0, 1 and 2, host-d and host-e TP=2's ranks 0 and 1; and name the direct links' and the ring's addresses); result values, token ids and token hashes kept.", + "replacements_by_file": { + "README.md": 0, + "binaries.txt": 0, + "ci/flashnext-served.log": 0, + "ci/fuzz-cortheon.log": 0, + "ci/glm-host.log": 2, + "ci/lean.txt": 0, + "ci/mutations/auto-at-two.log": 2, + "ci/mutations/bind-before-pages.log": 3, + "ci/mutations/close-keeps-pages.log": 7, + "ci/mutations/fits-flag-dropped.log": 2, + "ci/mutations/fits-own-free.log": 2, + "ci/mutations/fp8-kernel-table.log": 4, + "ci/mutations/fp8-on-pages.log": 2, + "ci/mutations/head.log": 0, + "ci/mutations/holds-no-floor.log": 6, + "ci/mutations/kept-window-cache.log": 5, + "ci/mutations/map-any-id.log": 2, + "ci/mutations/no-poison.log": 3, + "ci/mutations/no-verify-rows.log": 3, + "ci/mutations/own-as-window.log": 261, + "ci/mutations/own-no-floor.log": 11, + "ci/mutations/own-not-bounded.log": 9, + "ci/mutations/own-not-placed.log": 3, + "ci/mutations/page-bytes-unchecked.log": 16, + "ci/mutations/paged-first-at-cache.log": 11, + "ci/mutations/paged-table-kept.log": 2, + "ci/mutations/pages-not-agreed.log": 12, + "ci/mutations/pages-not-fingerprinted.log": 6, + "ci/mutations/piece-first-page.log": 7, + "ci/mutations/piece-no-offset.log": 3, + "ci/mutations/plan-flag-dropped.log": 3, + "ci/mutations/pool-on-window.log": 16, + "ci/mutations/size-at-held.log": 6, + "ci/mutations/size-mapped-only.log": 6, + "ci/mutations/slot-zero-unpaged.log": 12, + "ci/mutations/split-no-slots.log": 49, + "ci/security.log": 0, + "ci/test-golden.log": 0, + "ci/test-kernel-contract.log": 0, + "ci/test.log": 2, + "commit.txt": 0, + "records-sha256.txt": 0, + "runs/tp2-chain.txt": 40, + "single/glm-paged-check.log": 0, + "single/reg/glm-lanes-concurrent.log": 0, + "single/reg/glm-lanes-cover.log": 0, + "single/reg/glm-shared-check.log": 0, + "single/reg/paged.log": 0, + "single/reg/ref.log": 0, + "single/reg/summary.log": 0, + "single/reg/synth.log": 0, + "tools/kvp_admit.py": 0, + "tools/kvp_sum.py": 0, + "tools/media_http.py": 0, + "tools/media_resume_http.py": 0, + "tp2/build.log": 0, + "tp2/build/build.log": 0, + "tp2/build/fmt.log": 0, + "tp2/build/host.log": 0, + "tp2/build/kvpool.log": 0, + "tp2/build/lean.log": 0, + "tp2/build/native.log": 0, + "tp2/build/test.log": 0, + "tp2/check/glm-paged-check-rank0.log": 0, + "tp2/check/glm-paged-check-rank1.log": 0, + "tp2/compare/auto-against-all.log": 0, + "tp2/compare/auto-against-on.log": 0, + "tp2/compare/main-against-all.log": 0, + "tp2/compare/main-against-on.log": 0, + "tp2/serve/r2all/client.log": 0, + "tp2/serve/r2all/conc.json": 0, + "tp2/serve/r2all/rows-without-bodies.jsonl": 0, + "tp2/serve/r2all/server-rank0.log": 121, + "tp2/serve/r2all/server-rank1.log": 114, + "tp2/serve/r2auto/client.log": 0, + "tp2/serve/r2auto/fresh.jsonl": 0, + "tp2/serve/r2auto/server-rank0.log": 121, + "tp2/serve/r2auto/server-rank1.log": 114, + "tp2/serve/r2main/client.log": 0, + "tp2/serve/r2main/fresh.jsonl": 0, + "tp2/serve/r2main/server-rank0.log": 121, + "tp2/serve/r2main/server-rank1.log": 114, + "tp2/serve/r2on/client.log": 0, + "tp2/serve/r2on/conc.json": 0, + "tp2/serve/r2on/rows-without-bodies.jsonl": 0, + "tp2/serve/r2on/server-rank0.log": 121, + "tp2/serve/r2on/server-rank1.log": 114, + "tp2/serve/r2vis/client.log": 0, + "tp2/serve/r2vis/media-resume.json": 0, + "tp2/serve/r2vis/media.json": 0, + "tp2/serve/r2vis/server-rank0.log": 121, + "tp2/serve/r2vis/server-rank1.log": 114, + "tp2/speed/r3mainr9/cache.json": 0, + "tp2/speed/r3mainr9/client.log": 0, + "tp2/speed/r3mainr9/cold.json": 0, + "tp2/speed/r3mainr9/decode.json": 0, + "tp2/speed/r3mainr9/server-rank0.log": 121, + "tp2/speed/r3mainr9/server-rank1.log": 114, + "tp2/speed/r3mainr92/cache.json": 0, + "tp2/speed/r3mainr92/client.log": 0, + "tp2/speed/r3mainr92/cold.json": 0, + "tp2/speed/r3mainr92/decode.json": 0, + "tp2/speed/r3mainr92/server-rank0.log": 121, + "tp2/speed/r3mainr92/server-rank1.log": 114, + "tp2/speed/r3r9a/cache.json": 0, + "tp2/speed/r3r9a/client.log": 0, + "tp2/speed/r3r9a/cold.json": 0, + "tp2/speed/r3r9a/decode.json": 0, + "tp2/speed/r3r9a/server-rank0.log": 121, + "tp2/speed/r3r9a/server-rank1.log": 114, + "tp2/speed/r3r9all/cache.json": 0, + "tp2/speed/r3r9all/client.log": 0, + "tp2/speed/r3r9all/cold.json": 0, + "tp2/speed/r3r9all/decode.json": 0, + "tp2/speed/r3r9all/server-rank0.log": 121, + "tp2/speed/r3r9all/server-rank1.log": 114, + "tp2/speed/r3r9b/cache.json": 0, + "tp2/speed/r3r9b/client.log": 0, + "tp2/speed/r3r9b/cold.json": 0, + "tp2/speed/r3r9b/decode.json": 0, + "tp2/speed/r3r9b/server-rank0.log": 121, + "tp2/speed/r3r9b/server-rank1.log": 114, + "tp3/build.log": 0, + "tp3/chain.log": 10, + "tp3/check/glm-paged-check-rank0.log": 9791, + "tp3/check/glm-paged-check-rank1.log": 6299, + "tp3/check/glm-paged-check-rank2.log": 6519, + "tp3/ring.patch": 2, + "tp3/serve/r6admit/admit.json": 0, + "tp3/serve/r6admit/client.log": 0, + "tp3/serve/r6admit/hostwatch-rank0.log": 0, + "tp3/serve/r6admit/hostwatch-rank1.log": 0, + "tp3/serve/r6admit/hostwatch-rank2.log": 0, + "tp3/serve/r6admit/server-rank0.log": 12051, + "tp3/serve/r6admit/server-rank1.log": 3183, + "tp3/serve/r6admit/server-rank2.log": 6714, + "tp3/serve/r6all/client.log": 0, + "tp3/serve/r6all/conc.json": 0, + "tp3/serve/r6all/hostwatch-rank0.log": 0, + "tp3/serve/r6all/hostwatch-rank1.log": 0, + "tp3/serve/r6all/hostwatch-rank2.log": 0, + "tp3/serve/r6all/server-rank0.log": 2961, + "tp3/serve/r6all/server-rank1.log": 8083, + "tp3/serve/r6all/server-rank2.log": 4088, + "tp3/serve/r6auto/client.log": 0, + "tp3/serve/r6auto/cold.json": 0, + "tp3/serve/r6auto/conc.json": 0, + "tp3/serve/r6auto/hostwatch-rank0.log": 0, + "tp3/serve/r6auto/hostwatch-rank1.log": 0, + "tp3/serve/r6auto/hostwatch-rank2.log": 0, + "tp3/serve/r6auto/long-beside-short.json": 0, + "tp3/serve/r6auto/rows-without-bodies.jsonl": 0, + "tp3/serve/r6auto/server-rank0.log": 11357, + "tp3/serve/r6auto/server-rank1.log": 5574, + "tp3/serve/r6auto/server-rank2.log": 1655, + "tp3/serve/r6off/client.log": 0, + "tp3/serve/r6off/compare-auto.log": 0, + "tp3/serve/r6off/fresh.jsonl": 0, + "tp3/serve/r6off/hostwatch-rank0.log": 0, + "tp3/serve/r6off/hostwatch-rank1.log": 0, + "tp3/serve/r6off/hostwatch-rank2.log": 0, + "tp3/serve/r6off/server-rank0.log": 16538, + "tp3/serve/r6off/server-rank1.log": 5055, + "tp3/serve/r6off/server-rank2.log": 3140 + }, + "undecodable_bytes_replaced_by_file": {} +} diff --git a/docs/receipts/glm-kv-pages-8e705835/runs/tp2-chain.txt b/docs/receipts/glm-kv-pages-8e705835/runs/tp2-chain.txt new file mode 100644 index 0000000000..2d1933d8a3 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/runs/tp2-chain.txt @@ -0,0 +1,90 @@ +kvp-r9t2 start 10:01:18 +04: 67 logs aside +05: 11 logs aside +2026-10-23 10:01:24 fmt rc=0 +2026-10-23 10:06:48 build rc=0 324s: +2026-10-23 10:09:21 native rc=0 153s: +2026-10-23 10:11:43 host rc=0 142s: failed command: ./.zig-cache/o/4326653e6bd8ad6fdc5508bee4ba3a38/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- failed command: ./.zig-cache/o/da1ace8218 +2026-10-23 10:11:48 kvpool rc=0 5s: Build Summary: 5/5 steps succeeded; 17/17 tests passed +2026-10-23 10:12:30 test rc=0 42s: failed command: ./.zig-cache/o/33d3a117abb9ad8e9ed4ec4bcd11c3a0/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- Build Summary: 276/276 steps succeeded; 20 +2026-10-23 10:12:30 lean rc=1 0s: +0c7727776fda6e28 d82043de5d8b1f04 05: 0c7727776fda6e28 +kvp-k2 start 10:16:00 +=== kvp-check pg2r9 rc 0 0 +started rank 1 (r2all) on host-e +started rank 0 (r2all) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2all rc 0 +started rank 1 (r2on) on host-e +started rank 0 (r2on) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2on rc 0 +started rank 1 (r2auto) on host-e +started rank 0 (r2auto) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2auto rc 0 +started rank 1 (r2main) on host-e +started rank 0 (r2main) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2main rc 0 +compare auto-on rc=0 +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh +compare auto-all rc=0 +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh +compare main-on rc=0 +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh +compare main-all rc=0 +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh +=== kvp-r2m done +2026-10-23 11:47:35 === kvp-reg r9 done 0 failed +=== kvp-check pg1r9 rc 0 +2026-10-23 11:52:47 === kvp-base tp3 done +2026-10-23 11:49:45 build rc=0 289s: +2026-10-23 11:52:20 native rc=0 155s: +2026-10-23 11:52:38 host rc=0 18s: failed command: ./.zig-cache/o/a8dc8f3d17c2d0d217c7f824aa1d3537/test --cache-dir=./.zig-cache --seed=0x2c5a266f --listen=- failed command: ./.zig-cache/o/46e19a6c1fc +2026-10-23 11:52:38 kvpool rc=0 0s: Build Summary: 5/5 steps succeeded +2026-10-23 11:52:47 test rc=0 9s: failed command: ./.zig-cache/o/8a4d02bb9d3c4e71d1e5e01e68a5f951/test --cache-dir=./.zig-cache --seed=0xc2d912a1 --listen=- Build Summary: 276/276 steps succeeded; 224 +2026-10-23 11:52:47 lean rc=1 0s: +2026-10-23 11:52:47 === kvp-base tp3 done +358a8335696f005a +=== kvp-post3 done +started rank 1 (r2vis) on host-e +started rank 0 (r2vis) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r2vis rc 0 +=== kvp-k2 done 12:02:30 +started rank 1 (r3r9a) on host-e +started rank 0 (r3r9a) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3r9a rc 0 +started rank 1 (r3mainr9) on host-e +started rank 0 (r3mainr9) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3mainr9 rc 0 +started rank 1 (r3r9b) on host-e +started rank 0 (r3r9b) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3r9b rc 0 +started rank 1 (r3mainr92) on host-e +started rank 0 (r3mainr92) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3mainr92 rc 0 +started rank 1 (r3r9all) on host-e +started rank 0 (r3r9all) on host-d +stopped on host-d +stopped on host-e +=== kvp-phase r3r9all rc 0 +=== kvp-r9t2 done 13:24:36 diff --git a/docs/receipts/glm-kv-pages-8e705835/single/glm-paged-check.log b/docs/receipts/glm-kv-pages-8e705835/single/glm-paged-check.log new file mode 100644 index 0000000000..0ecb5d0702 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/single/glm-paged-check.log @@ -0,0 +1,9 @@ +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +PASS pages at TP=1: on the engine's own sequence, 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after diff --git a/docs/receipts/glm-kv-pages-8e705835/single/reg/glm-lanes-concurrent.log b/docs/receipts/glm-kv-pages-8e705835/single/reg/glm-lanes-concurrent.log new file mode 100644 index 0000000000..cd8eb49304 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/single/reg/glm-lanes-concurrent.log @@ -0,0 +1,39 @@ +info: checkpoint digest: 0.0 GiB read in 0.0 s, 1/1 cached files +PASS all at once: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS all at once: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS all at once: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS all at once: 3 streams at most in one round, 127 rounds with two or more +PASS staggered by 5 rounds: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS staggered by 5 rounds: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS staggered by 5 rounds: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS staggered by 5 rounds: 3 streams at most in one round, 127 rounds with two or more +PASS in reverse: stream 0 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS in reverse: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS in reverse: stream 2 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS in reverse: 3 streams at most in one round, 127 rounds with two or more +PASS the same prompt twice: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS the same prompt twice: stream 1 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS the same prompt twice: 2 streams at most in one round, 127 rounds with two or more +PASS sampled: stream 0 (prompt 0, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled: stream 1 (prompt 1, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled: stream 2 (prompt 2, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled: 3 streams at most in one round, 127 rounds with two or more +PASS the second discarded after 6 rounds: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS the second discarded after 6 rounds: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS the second discarded after 6 rounds: 3 streams at most in one round, 127 rounds with two or more +PASS prompts a chunk at a time while the others decode: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS prompts a chunk at a time while the others decode: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS prompts a chunk at a time while the others decode: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS prompts a chunk at a time while the others decode: 3 streams at most in one round, 127 rounds with two or more +PASS prompts a chunk at a time while the others decode: 2 rounds of the others between a prompt's chunks +PASS every prompt a chunk at a time: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS every prompt a chunk at a time: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS every prompt a chunk at a time: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS every prompt a chunk at a time: 3 streams at most in one round, 127 rounds with two or more +PASS every prompt a chunk at a time: 2 rounds of the others between a prompt's chunks +PASS sampled, prompts a chunk at a time: stream 0 (prompt 0, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled, prompts a chunk at a time: stream 1 (prompt 1, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled, prompts a chunk at a time: stream 2 (prompt 2, sampled) against its solo run: 128 ids equal, finished by length (127 rounds) +PASS sampled, prompts a chunk at a time: 3 streams at most in one round, 127 rounds with two or more +PASS sampled, prompts a chunk at a time: 2 rounds of the others between a prompt's chunks +PASS 1143 forwards held several streams' windows (Engine.verifyShared) diff --git a/docs/receipts/glm-kv-pages-8e705835/single/reg/glm-lanes-cover.log b/docs/receipts/glm-kv-pages-8e705835/single/reg/glm-lanes-cover.log new file mode 100644 index 0000000000..2b133b692a --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/single/reg/glm-lanes-cover.log @@ -0,0 +1,47 @@ +info: checkpoint digest: 2.0 GiB read in 2.0 s, 0/1 cached files +PASS copy drafts, greedy: 256 ids equal, finished by length (120 rounds) +PASS copy drafts: the proposer judged 139 copied rows (135 kept) +PASS copy drafts, sampled: 256 ids equal, finished by length (254 rounds) +PASS copy drafts, sampled, thin model: the proposer judged 0 copied rows (0 kept) +PASS a cancel at ask 0 of the prompt pass ended it with error.Cancelled +PASS the cancel was heard at ask 0 (1 asks) +PASS the request after a cancel at prompt ask 0: 64 ids equal, finished by length (63 rounds) +PASS a cancel at ask 1 of the prompt pass ended it with error.Cancelled +PASS the cancel was heard at ask 1 (2 asks) +PASS the request after a cancel at prompt ask 1: 64 ids equal, finished by length (63 rounds) +PASS a cancel at ask 3 of the prompt pass ended it with error.Cancelled +PASS the cancel was heard at ask 3 (4 asks) +PASS the request after a cancel at prompt ask 3: 64 ids equal, finished by length (63 rounds) +PASS a stream was discarded mid-decode (after 1 rounds) +PASS the request after a discard after 1 rounds: 64 ids equal, finished by length (63 rounds) +PASS a stream was discarded mid-decode (after 8 rounds) +PASS the request after a discard after 8 rounds: 64 ids equal, finished by length (63 rounds) +PASS a stream was discarded mid-decode (after a rejected draft) +PASS the request after a discard after a rejected draft: 64 ids equal, finished by length (63 rounds) +PASS eos at token 0 with drafts: 1 ids equal, finished by stop (0 rounds) +PASS eos at token 0: 1 ids equal, finished by stop (0 rounds) +PASS eos at token 9 with drafts: 10 ids equal, finished by stop (9 rounds) +PASS eos at token 9: 10 ids equal, finished by stop (9 rounds) +PASS eos at token 33 with drafts: 34 ids equal, finished by stop (33 rounds) +PASS eos at token 33: 34 ids equal, finished by stop (33 rounds) +PASS eos at token 100 with drafts: 101 ids equal, finished by stop (100 rounds) +PASS eos at token 100: 101 ids equal, finished by stop (100 rounds) +PASS max_new 1 with drafts: 1 ids equal, finished by length (0 rounds) +PASS max_new 2 with drafts: 2 ids equal, finished by length (1 rounds) +PASS max_new 3 with drafts: 3 ids equal, finished by length (2 rounds) +PASS max_new 4 with drafts: 4 ids equal, finished by length (3 rounds) +PASS max_new 5 with drafts: 5 ids equal, finished by length (4 rounds) +PASS the pass kept 2 states at its marks (2 asked) +PASS the pass resumed at 2048 (2048 restored) +PASS resumed at 2048, against the fresh pass: 64 ids equal, finished by length (63 rounds) +PASS cached drafted against cached plain: 64 ids equal, finished by length (63 rounds) +PASS cached concurrent equals fresh solo at 2048 +PASS cached concurrent equals fresh solo at 2048 +PASS the pass resumed at 4096 (4096 restored) +PASS resumed at 4096, against the fresh pass: 64 ids equal, finished by length (63 rounds) +PASS cached drafted against cached plain: 64 ids equal, finished by length (63 rounds) +PASS cached concurrent equals fresh solo at 4096 +PASS cached concurrent equals fresh solo at 4096 +PASS another tail resumed at 4096 (4096 restored) +PASS another tail after the kept 4096, against its fresh pass: 64 ids equal, finished by length (63 rounds) +PASS the last request, all of W with drafts: 512 ids equal, finished by length (511 rounds) diff --git a/docs/receipts/glm-kv-pages-8e705835/single/reg/glm-shared-check.log b/docs/receipts/glm-kv-pages-8e705835/single/reg/glm-shared-check.log new file mode 100644 index 0000000000..0aaef6b747 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/single/reg/glm-shared-check.log @@ -0,0 +1,2 @@ +info: checkpoint digest: 0.0 GiB read in 0.0 s, 1/1 cached files +PASS shared windows at TP=1: 108 rows of 5 streams over 12 rounds (2 to 5 streams a forward, 1 to 4 rows each, greedy and sampled) equal their lone twins' logits, tokens and drafts byte for byte, and every state its twin's diff --git a/docs/receipts/glm-kv-pages-8e705835/single/reg/paged.log b/docs/receipts/glm-kv-pages-8e705835/single/reg/paged.log new file mode 100644 index 0000000000..6618ea8927 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/single/reg/paged.log @@ -0,0 +1,9 @@ +info: checkpoint digest: 0.0 GiB read in 0.0 s, 1/1 cached files +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +PASS pages at TP=1: on the engine's own sequence, 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after diff --git a/docs/receipts/glm-kv-pages-8e705835/single/reg/ref.log b/docs/receipts/glm-kv-pages-8e705835/single/reg/ref.log new file mode 100644 index 0000000000..6b607222b1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/single/reg/ref.log @@ -0,0 +1,30 @@ +RESULT b16 matmul 300 rows x [512, 4096] (K in 8 slices), bf16 out: worst column 93 at 5.502e-1 of its tolerance +RESULT b16 matmul 300 rows x [512, 4096] (K in 8 slices), fp32 out: worst column 165 at 2.086e-4 of its tolerance +PASS b16 matmul [512, 4096] (8 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal +RESULT b16 matmul 300 rows x [1280, 1536] (K in 4 slices), bf16 out: worst column 257 at 7.984e-1 of its tolerance +RESULT b16 matmul 300 rows x [1280, 1536] (K in 4 slices), fp32 out: worst column 1227 at 7.001e-4 of its tolerance +PASS b16 matmul [1280, 1536] (4 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal +RESULT b16 matmul 300 rows x [1024, 128] (K in 1 slices), bf16 out: worst column 182 at 9.820e-1 of its tolerance +RESULT b16 matmul 300 rows x [1024, 128] (K in 1 slices), fp32 out: worst column 839 at 1.471e-2 of its tolerance +PASS b16 matmul [1024, 128] (1 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal +RESULT router 300 rows x [256, 4096] (K in 8 slices): worst column 57 at 1.681e-4 of its tolerance +PASS router [256, 4096]: within float64 tolerance, router_tile and router_part bits equal at every row count +RESULT select 300 rows, top 8 of 256: weights at 1.807e-2 of 2^-16 relative, 0 near-tie swaps +PASS select top-8 of 256: picks follow the float64 ranking (near ties aside), weights within float64 tolerance +RESULT latent attention 70 rows at 1430, 32 heads, 32-head blocks: worst column 306 at 1.809e-1 of its tolerance +PASS latent attention over 1500 keys: within float64 tolerance, both head blocks and one-row windows equal +RESULT latent sparse attention 70 rows, up to 1100 of 6000 keys: worst column 349 at 4.737e-1 of its tolerance +PASS latent sparse attention over up to 1100 tokens: within float64 tolerance, one-row windows equal, count 0 left alone +RESULT GLM FP8 reconstructed attention vs FP64: worst column 5 at 5.919e-1 of its tolerance +RESULT GLM FP8 reconstructed attention vs FP64: worst column 5 at 5.919e-1 of its tolerance +RESULT GLM FP8 reconstructed attention vs FP64: worst column 171 at 2.159e-1 of its tolerance +RESULT GLM FP8 reconstructed attention vs FP64: worst column 171 at 2.159e-1 of its tolerance +PASS GLM FP8 synthetic format, masking, row/chunk identity and reconstructed BF16/FP64 attention +RESULT hc mixing 70 rows x 16 K blocks: 24 dots and the squares (column 24): worst column 24 at 2.226e-3 of its tolerance +PASS hc mixing: dots and squares within float64 tolerance, decode windows' partials and outputs equal the prompt rows' +RESULT KDA 70 rows x 32 heads, fused chain against float64 (column = head x 128 + t): worst column 1041 at 4.563e-1 of its tolerance +RESULT KDA 3 rows x 32 heads, fused chain against float64 (column = head x 128 + t): worst column 476 at 3.576e-1 of its tolerance +PASS KDA 32 heads: within float64 tolerance, fused and three-kernel chains equal at 70 and 3 rows, replays rebuild the state +RESULT index scores 96 rows at 6000, 32 heads x 128, 1536 pools: worst column 862 at 1.974e-3 of its tolerance +PASS DSA indexer 96 rows at 6000: scores within float64 tolerance, top-512 pools equal the host's rule exactly +PASS glm-ref: every projection within 9.820e-1 of its float64 tolerance diff --git a/docs/receipts/glm-kv-pages-8e705835/single/reg/summary.log b/docs/receipts/glm-kv-pages-8e705835/single/reg/summary.log new file mode 100644 index 0000000000..5923bc9b46 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/single/reg/summary.log @@ -0,0 +1,8 @@ +-- run 2026-10-23 11:44:56 8e705835 glm cuda tests: the paged check's stream is the engine's own sequence, +2026-10-23 11:45:46 ref rc=0 50s: PASS b16 matmul [512, 4096] (8 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal PASS b16 matmul [1280, 1536] (4 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal PASS b16 matmul [1024, 128] (1 slices): bf16 and fp32 out within float64 tolerance, every entry and row count's bits equal +2026-10-23 11:45:51 synth rc=0 5s: PASS glm-synth: 330 tensors, 2095 MiB, config 2 layers 16 experts vocabulary 8192, dc.json in /work/r9/synth +2026-10-23 11:46:29 glm-lanes-cover rc=0 38s: PASS copy drafts, greedy: 256 ids equal, finished by length (120 rounds) PASS copy drafts: the proposer judged 139 copied rows (135 kept) PASS copy drafts, sampled: 256 ids equal, finished by length (254 rounds) +2026-10-23 11:47:16 glm-lanes-concurrent rc=0 47s: PASS all at once: stream 0 (prompt 0) against its solo run: 128 ids equal, finished by length (127 rounds) PASS all at once: stream 1 (prompt 1) against its solo run: 128 ids equal, finished by length (127 rounds) PASS all at once: stream 2 (prompt 2) against its solo run: 128 ids equal, finished by length (127 rounds) +2026-10-23 11:47:25 glm-shared-check rc=0 9s: PASS shared windows at TP=1: 108 rows of 5 streams over 12 rounds (2 to 5 streams a forward, 1 to 4 rows each, greedy and sampled) equal their lone twins' logits, tokens and drafts byte for byte, and every state its twin's +2026-10-23 11:47:34 paged rc=0 9s: PASS pages at TP=1: on the engine's own sequence, 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after +2026-10-23 11:47:35 === kvp-reg r9 done 0 failed diff --git a/docs/receipts/glm-kv-pages-8e705835/single/reg/synth.log b/docs/receipts/glm-kv-pages-8e705835/single/reg/synth.log new file mode 100644 index 0000000000..4998997f48 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/single/reg/synth.log @@ -0,0 +1 @@ +PASS glm-synth: 330 tensors, 2095 MiB, config 2 layers 16 experts vocabulary 8192, dc.json in /work/r9/synth diff --git a/docs/receipts/glm-kv-pages-8e705835/tools/kvp_admit.py b/docs/receipts/glm-kv-pages-8e705835/tools/kvp_admit.py new file mode 100644 index 0000000000..1ad3e03675 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tools/kvp_admit.py @@ -0,0 +1,105 @@ +"""Admission on pages: each prompt alone, then all at once (an optional long one first), from a cold-prompt file. + +Usage: kvp_admit.py BASE MODEL PROMPTS.json OUT.json (--length L | --fit LOG) --n N [--long LONG] [--tokens T] [--tools DIR] +PROMPTS.json is reuse_served.py cold's file. The N prompts of length L run alone, one after another, then all at once; +with --fit, L is the longest length of which one request fits the pool rank 0's LOG reports and two do not; +with --long, the first prompt of length LONG starts first and the N follow once its stream is admitted. Each reply is +T tokens (ignore_eos), seeded and sampled. The report: every request's wait (sent to first token) and end, which +requests started only after another ended (the pool held too few pages for them), and each together reply's token SHA +against its alone run. Exit 0 only when every reply equals its alone run. +""" +import argparse +import json +import re +import sys +import threading +import time + + +def fit(log: str, lengths: list, tokens: int) -> int: + """The longest of `lengths` whose request takes the pool's pages once but not twice (0 when none).""" + m = re.search(r"taking (\d+) pages of 512 rows", open(log, errors="replace").read()) + if not m: + print(f"fit: no page count in {log}", flush=True) + return 0 + pages = int(m.group(1)) + need = lambda n: -(-(n + 64 + tokens + 16) // 512) # noqa: E731 - the prompt with its template, the reply, the slack + ok = [n for n in lengths if need(n) <= pages < 2 * need(n)] + print(f"fit: {pages} pages; " + (f"{ok[-1]}-token prompts take {need(ok[-1])} each" if ok else "no length fits once and not twice"), flush=True) + return ok[-1] if ok else 0 + + +def main() -> int: + ap = argparse.ArgumentParser() + ap.add_argument("base") + ap.add_argument("model") + ap.add_argument("prompts") + ap.add_argument("out") + ap.add_argument("--length", type=int, default=0) + ap.add_argument("--fit", default="") + ap.add_argument("--n", type=int, required=True) + ap.add_argument("--long", type=int, default=0) + ap.add_argument("--tokens", type=int, default=128) + ap.add_argument("--tools", default="tools") + o = ap.parse_args() + sys.path.insert(0, o.tools) + import bench_concurrent as bench # noqa: E402 - the tree's own client + + items = json.load(open(o.prompts)) + if o.fit: + o.length = fit(o.fit, sorted({it["length"] for it in items}), o.tokens) + if not o.length: + return 2 + pick = lambda length: [it for it in items if it["length"] == length] # noqa: E731 + short = [{"name": f"p{it['length']}-{it['rep']}", "kind": "chat", "prompt": it["messages"][0]["content"]} + for it in pick(o.length)] + if len(short) < o.n: + short = [dict(s, name=f"{s['name']}-{k}") for k in range(o.n) for s in short][: o.n] + short = short[: o.n] + long = None + if o.long: + it = pick(o.long)[0] + long = {"name": f"long{o.long}", "kind": "chat", "prompt": it["messages"][0]["content"]} + alone = [] + for i, s in enumerate(short): + r = bench.stream(o.base, o.model, s, o.tokens, 1.0, 1000 + i) + alone.append(r) + print(f"alone {s['name']}: prompt {r.get('prompt_tokens')} ttft {r.get('ttft_s')} sha {r.get('token_sha256')}", flush=True) + runs: list = [None] * (len(short) + (1 if long else 0)) + threads = [] + + def go(k, item, seed): + runs[k] = bench.stream(o.base, o.model, item, o.tokens, 1.0, seed) + + t0 = time.perf_counter() + if long: + threads.append(threading.Thread(target=go, args=(len(short), long, 999))) + threads[-1].start() + time.sleep(2.0) # its stream is admitted before the others arrive + for i, s in enumerate(short): + threads.append(threading.Thread(target=go, args=(i, s, 1000 + i))) + threads[-1].start() + for t in threads: + t.join() + rows, equal = [], 0 + for k, r in enumerate(runs): + name = long["name"] if long and k == len(short) else short[k]["name"] + same = None if k == len(short) else (r.get("token_sha256") is not None and r.get("token_sha256") == alone[k].get("token_sha256")) + equal += 1 if same else 0 + rows.append({"name": name, "prompt": r.get("prompt_tokens"), "sent": r["sent"] - t0 if r.get("sent") else None, + "first": r["first"] - t0 if r.get("first") else None, "last": r["last"] - t0 if r.get("last") else None, + "error": r.get("error"), "equal_alone": same}) + ends = sorted((x["last"], x["name"]) for x in rows if x["last"] is not None) + for x in rows: + before = [n for e, n in ends if x["first"] is not None and e < x["first"] and n != x["name"]] + x["started_after"] = before + print(f"{x['name']:>12}: prompt {x['prompt']} first {x['first']:.2f} s last {x['last']:.2f} s" + f" equal_alone {x['equal_alone']} started after {before or '-'}" if x["first"] is not None else f"{x['name']}: {x['error']}", flush=True) + waited = sum(1 for x in rows if x["started_after"]) + print(f"together == alone: {equal}/{len(short)}; {waited} request(s) started only after another ended", flush=True) + json.dump({"alone": alone, "together": runs, "rows": rows}, open(o.out, "w")) + return 0 if equal == len(short) else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/docs/receipts/glm-kv-pages-8e705835/tools/kvp_sum.py b/docs/receipts/glm-kv-pages-8e705835/tools/kvp_sum.py new file mode 100644 index 0000000000..aa56708108 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tools/kvp_sum.py @@ -0,0 +1,87 @@ +"""KV_PAGES receipts in one line each: bench_concurrent's alone/serial counts, a record's planned count, two records' bodies, +and R3's speed (each serve's decode, 32k cold and 2k-64k cold, with the regression rule).""" +import json +import statistics +import sys + + +def conc(path): + """Cells, and the alone and serial (draft: false) equal / compared counts over every cell; false when a request + failed or a reply differed (bench_concurrent.py ends 0 even then, so a chain reads this instead).""" + rows = json.load(open(path))["cells"] + a = [sum(r.get("alone", {}).get(k, 0) for r in rows) for k in ("equal", "unequal", "failed")] + s = [sum(r.get("serial", {}).get(k, 0) for r in rows) for k in ("equal", "unequal", "failed")] + print(f"{path}: {len(rows)} cells; alone {a[0]}/{sum(a)} equal ({a[1]} unequal, {a[2]} failed); " + f"serial {s[0]}/{sum(s)} equal ({s[1]} unequal, {s[2]} failed)") + return a[1] + a[2] + s[1] + s[2] + sum(r.get("failed", 0) for r in rows) == 0 + + +def bodies(a_path, b_path): + """Whether two records sent the same bodies in the same order (so one replay serves both compares).""" + a = [json.loads(line) for line in open(a_path)] + b = [json.loads(line) for line in open(b_path)] + same = sum(1 for x, y in zip(a, b) if x["i"] == y["i"] and x["body"] == y["body"]) + shas = sum(1 for x, y in zip(a, b) if x["got"].get("sha") == y["got"].get("sha")) + cached = sum(1 for x, y in zip(a, b) if x["got"].get("cached") == y["got"].get("cached")) + print(f"{a_path} vs {b_path}: {len(a)} and {len(b)} rows; bodies equal {same}; shas equal {shas}; cached equal {cached}") + + +def measures(d, label): + """A serve's times (lower is better): greedy decode s/token per prompt, the 32k cold TTFT (median of its three + requests) and the cold TTFT per length (median of the reps); sampled decode apart, as each run decodes its own text.""" + t, own, shas = {}, {}, {} + for r in json.load(open(f"{d}/{label}-decode.json")): + key = f"decode {r['prompt']}" + if r["temperature"] == 0: + t[key] = 1 / r["decode_tps_median"] + shas[key] = {e["output_sha256"] for e in r["evidence"]} + else: + own[key] = r["decode_tps_median"] + rows = json.load(open(f"{d}/{label}-cache.json"))["rows"] + t["cold 32k chat"] = statistics.median(r["ttft_s"] for r in rows) + shas["cold 32k chat"] = {r["token_sha256"] for r in rows} + cold = {} + for r in json.load(open(f"{d}/{label}-cold.json")): + cold.setdefault(r["length"], []).append(r["ttft"]) + for n, v in sorted(cold.items()): + t[f"cold {n // 1024}k"] = statistics.median(v) + return t, own, shas + + +def speed(d, *groups): + """groups: main=L1,L2 head=L3,L4 [all=L5]. Slower is a time over 1.02 times the other's: a regression when both + head serves are slower than both main serves on one measure, or the all serve slower than both main serves. Greedy + tokens that differ between serves are a mismatch, a stop before any speed.""" + g = {k: v.split(",") for k, v in (x.split("=") for x in groups)} + m = {label: measures(d, label) for v in g.values() for label in v} + labels = [label for v in g.values() for label in v] + print("measure (s, lower is better)".ljust(30) + "".join(label.rjust(10) for label in labels)) + for key in m[labels[0]][0]: + print(key.ljust(30) + "".join(f"{m[label][0][key]:10.4f}" for label in labels)) + for key in m[labels[0]][1]: + print(f"{key} sampled tok/s (own text)".ljust(30) + "".join(f"{m[label][1][key]:10.2f}" for label in labels)) + differ = [] + for key in m[labels[0]][2]: + same = len(set().union(*(m[label][2][key] for label in labels))) == 1 + print(f"{key}: the same tokens in every serve: {same}") + if not same: + differ.append(key) + slow = [] + for key in m[labels[0]][0]: + worst_main = max(m[label][0][key] for label in g["main"]) + for k in [k for k in g if k != "main"]: + best = min(m[label][0][key] for label in g[k]) + ratio = best / worst_main + print(f"{key}: {k} best / main worst = {ratio:.4f}") + if ratio > 1.02: + slow.append(f"{key} ({k} {ratio:.4f})") + print("REGRESSION: " + "; ".join(slow) if slow else "no regression: no measure over 1.02") + if differ: + print("MISMATCH (stop: the serves' tokens differ): " + "; ".join(differ)) + + +if __name__ == "__main__": + mode, *paths = sys.argv[1:] + if mode == "conc": + sys.exit(0 if all([conc(p) for p in paths]) else 1) # every file printed + {"bodies": lambda: bodies(*paths), "speed": lambda: speed(*paths)}[mode]() diff --git a/docs/receipts/glm-kv-pages-8e705835/tools/media_http.py b/docs/receipts/glm-kv-pages-8e705835/tools/media_http.py new file mode 100644 index 0000000000..965824f8ce --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tools/media_http.py @@ -0,0 +1,56 @@ +"""Against `tensorfold serve --vision` of GLM-5.3-Flash on CUDA: one image request (IMAGE as a data URL, a question +about it, thinking off, greedy, ignore_eos, 128 tokens, seed 1234) through upstream's bench_concurrent stream: + - first, with drafts (a cold prompt) + - repeated with drafts: the server's cached_tokens says whether it resumed; its ids equal the first's (resumed = fresh) + - with "draft": false: its ids equal the first's (drafted = plain) + - three at once with drafts: each one's ids equal the first's (concurrent = solo) +Every row's counters and hashes to OUT; one line a request, then "exit 0|1" (1 when any ids differ or a row is +unverified). Run beside upstream's tools/bench_concurrent.py and bench_evidence.py. + + python3 media_http.py BASE MODEL IMAGE OUT +""" +import base64 +import json +import sys + +import bench_concurrent as bench +from bench_evidence import token_equal + +FIELDS = ("prompt_tokens", "cached_tokens", "tokens", "cache_state", "complete", "token_sha256", "error") + + +def main(base, model, image, out): + with open(image, "rb") as f: + url = "data:image/png;base64," + base64.b64encode(f.read()).decode() + item = {"name": "image", "kind": "chat", "prompt": [ + {"type": "image_url", "image_url": {"url": url}}, + {"type": "text", "text": "Describe this image in two sentences."}]} + rows, ok = [], True + + def keep(phase, run, ref=None): + nonlocal ok + row = {"phase": phase, **{k: run.get(k) for k in FIELDS}} + if ref is not None: + row["equal_first"] = token_equal(ref, run) + ok &= row["equal_first"] is True + rows.append(row) + print(f"{phase}: prompt {row['prompt_tokens']} cached {row['cached_tokens']} ({row['cache_state']}) tokens " + f"{row['tokens']} sha256 {str(row['token_sha256'])[:12]}" + + ("" if ref is None else f" equal_first {row['equal_first']}"), flush=True) + + first = bench.stream(base, model, item, 128, 0, 1234) + ok &= first.get("complete") is True and first.get("token_sha256") is not None + keep("first", first) + keep("repeated", bench.stream(base, model, item, 128, 0, 1234), first) + keep("draft_false", bench.stream(base, model, item, 128, 0, 1234, draft=False), first) + for run in bench.together(base, model, [(item, 1234)] * 3, 128, 0): + keep("concurrent", run, first) + with open(out, "w", encoding="utf-8") as f: + json.dump({"image_bytes": len(url), "requested_reply_tokens": 128, "temperature": 0, "seed": 1234, + "rows": rows}, f, indent=1) + print(f"exit {0 if ok else 1}") + return 0 if ok else 1 + + +if __name__ == "__main__": + sys.exit(main(*sys.argv[1:5])) diff --git a/docs/receipts/glm-kv-pages-8e705835/tools/media_resume_http.py b/docs/receipts/glm-kv-pages-8e705835/tools/media_resume_http.py new file mode 100644 index 0000000000..042644a14b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tools/media_resume_http.py @@ -0,0 +1,92 @@ +"""Against `tensorfold serve --vision` of GLM-5.3-Flash on CUDA: a prompt resumed from a state after an image equals a +fresh one. media_http.py's request is one message, so nothing before it is kept and its repeat is cold; here the image +sits in the first of three messages (resume_http.py's form: the image as a data URL first, as media_http.py sends it, +then a nonce and about 5,000 tokens of notes, a fixed assistant reply, a follow-up question), thinking off, greedy, +ignore_eos, 64 tokens: + - first: cold (cached_tokens 0) + - repeated: resumed from the state the first kept (cached_tokens above 0, past the image), the same ids + - "draft": false: the same ids + - two at once: each the same ids +Every reply's counters and hashes to OUT; one line a request, then "exit 0|1". Standard library only. + + python3 media_resume_http.py PORT IMAGE OUT +""" +import base64 +import json +import sys +import threading +import time +import urllib.request + +LINE = "Line {i}: the river carries {n} stones past the mill, and the miller counts them twice before the bell. " + + +def body(nonce, url, draft=True): + notes = "".join(LINE.format(i=i, n=(i * 37) % 101) for i in range(220)) + b = {"model": "local-model", "max_tokens": 64, "temperature": 0, "ignore_eos": True, + "chat_template_kwargs": {"enable_thinking": False}, + "messages": [{"role": "user", "content": [ + {"type": "image_url", "image_url": {"url": url}}, + {"type": "text", "text": f"Reference notes {nonce}, after the picture.\n" + notes + + "\nWhat does the picture show, in one sentence?"}]}, + {"role": "assistant", "content": "The picture shows a logo on a plain background."}, + {"role": "user", "content": "Which line mentions the most stones? Answer in one sentence."}]} + if not draft: + b["draft"] = False + return b + + +def post(port, b): + req = urllib.request.Request(f"http://127.0.0.1:{port}/v1/chat/completions", data=json.dumps(b).encode(), + headers={"Content-Type": "application/json"}) + t0 = time.perf_counter() + with urllib.request.urlopen(req, timeout=900) as r: + d = json.loads(r.read()) + usage = d.get("usage") or {} + return {"seconds": round(time.perf_counter() - t0, 3), "prompt_tokens": usage.get("prompt_tokens"), + "cached_tokens": (usage.get("prompt_tokens_details") or {}).get("cached_tokens"), + "tokens": usage.get("completion_tokens"), + "token_sha256": (d.get("tensorfold") or {}).get("token_sha256"), + "content": d["choices"][0]["message"].get("content")} + + +def main(port, image, out): + with open(image, "rb") as f: + url = "data:image/png;base64," + base64.b64encode(f.read()).decode() + nonce = f"{time.time_ns():x}" + rows, ok = [], True + + def keep(phase, row, first=None): + nonlocal ok + row = {"phase": phase, **row} + if first is not None: + row["same_as_first"] = (row["token_sha256"] or row["content"]) == (first["token_sha256"] or first["content"]) + ok &= row["same_as_first"] + rows.append(row) + print(f"{phase}: prompt {row['prompt_tokens']} cached {row['cached_tokens']} tokens {row['tokens']} " + f"{row['seconds']} s sha256 {str(row['token_sha256'])[:12]}" + + ("" if first is None else f" same_as_first {row['same_as_first']}"), flush=True) + return row + + first = keep("first", post(port, body(nonce, url))) + ok &= first["cached_tokens"] == 0 and first["tokens"] == 64 + again = keep("repeated", post(port, body(nonce, url)), first) + ok &= (again["cached_tokens"] or 0) > 0 + keep("draft_false", post(port, body(nonce, url, draft=False)), first) + both = [None, None] + threads = [threading.Thread(target=lambda i=i: both.__setitem__(i, post(port, body(nonce, url)))) for i in range(2)] + for t in threads: + t.start() + for t in threads: + t.join() + for row in both: + keep("concurrent", row, first) + with open(out, "w", encoding="utf-8") as f: + json.dump({"nonce": nonce, "image_bytes": len(url), "requested_reply_tokens": 64, "temperature": 0, + "rows": rows}, f, indent=1) + print(f"exit {0 if ok else 1}") + return 0 if ok else 1 + + +if __name__ == "__main__": + sys.exit(main(int(sys.argv[1]), sys.argv[2], sys.argv[3])) diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/build.log b/docs/receipts/glm-kv-pages-8e705835/tp2/build.log new file mode 100644 index 0000000000..3af6a67717 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/build.log @@ -0,0 +1,11 @@ +-- run 2026-10-23 10:01:23 +2026-10-23 10:01:24 tree 8e705835 glm cuda tests: the paged check's stream is the engine's own sequence, on pages f; changed: zig/glm_snapshot_host.zig zig/src/core/contract/glm_cuda_records.zig zig/src/core/contract/registry.zig zig/src/core/kv_pool.zig zig/src/core/kv_pool_test.zig zig/src/families/glm/cuda.zig zig/src/families/glm/cuda_dsa_block.zig zig/src/families/glm/cuda_engine.zig zig/src/families/glm/cuda_follow.zig zig/src/families/glm/cuda_kv8_test.zig zig/src/families/glm/cuda_lanes.zig zig/src/families/glm/cuda_latent.zig zig/src/families/glm/cuda_native.zig zig/src/families/glm/cuda_pages.zig zig/src/families/glm/cuda_pages_test.zig zig/src/families/glm/cuda_ranks.zig zig/src/families/glm/cuda_snapshot_layout.zig zig/src/families/glm/cuda_snapshot_pages.zig zig/src/families/glm/cuda_snapshots.zig zig/src/families/glm/cuda_sparse.zig zig/src/families/glm/cuda_state.zig zig/src/native/cuda.zig zig/src/native/cuda_kv_pool.zig zig/src/native/cuda_kv_safety_test.zig zig/tests/cuda/contract_glm_attn.zig zig/tests/cuda/contract_glm_index.zig zig/tests/cuda/contract_glm_pages.zig zig/tests/cuda/contract_tests.zig zig/tests/cuda/glm_paged_tests.zig zig/tests/cuda/glm_ref_attn.zig zig/tests/cuda/glm_ref_fp8.zig zig/tests/cuda/glm_ref_index.zig zig/tests/cuda/main.zig +2026-10-23 10:01:24 fmt rc=0 +2026-10-23 10:01:24 -- r9: 8e705835 glm cuda tests: the paged check's stream is the engine's own sequence, on pages f +2026-10-23 10:06:48 build rc=0 324s: +2026-10-23 10:09:21 native rc=0 153s: +2026-10-23 10:11:43 host rc=0 142s: failed command: ./.zig-cache/o/4326653e6bd8ad6fdc5508bee4ba3a38/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- failed command: ./.zig-cache/o/da1ace8218d407d58b4a1b7a9ef6c22b/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- Build Summary: 155/155 steps succeeded; 2516/2522 test +2026-10-23 10:11:48 kvpool rc=0 5s: Build Summary: 5/5 steps succeeded; 17/17 tests passed +2026-10-23 10:12:30 test rc=0 42s: failed command: ./.zig-cache/o/33d3a117abb9ad8e9ed4ec4bcd11c3a0/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- Build Summary: 276/276 steps succeeded; 2023/2037 tests passed (14 skipped) | | +- compile lib hip-mock-failed debug native success 65ms MaxRSS:62M +2026-10-23 10:12:30 lean rc=1 0s: +2026-10-23 10:12:31 === kvp-base r9 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/build/build.log b/docs/receipts/glm-kv-pages-8e705835/tp2/build/build.log new file mode 100644 index 0000000000..d899c6d407 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/build/build.log @@ -0,0 +1,126 @@ +install ++- install tensorfold + +- compile exe tensorfold fast native + +- run /usr/local/cuda/bin/nvcc (prefill_attention.fatbin) w +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +install ++- install tensorfold + +- compile exe tensorfold fast native + +- run /usr/local/cuda/bin/nvcc (glm_mm.fatbin) w +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + + +install ++- install tensorfold + +- compile exe tensorfold fast native + +- run /usr/local/cuda/bin/nvcc (fp8_experts.fatbin) w +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +install ++- install tensorfold + +- compile exe tensorfold fast native + +- run /usr/local/cuda/bin/nvcc (flashnext_fact.fatbin) w +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " + + +install ++- install tensorfold + +- compile exe tensorfold fast native + +- run /usr/local/cuda/bin/nvcc (drama3_fp8.fatbin) w +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +install ++- install tensorfold + +- compile exe tensorfold fast native + +- run /usr/local/cuda/bin/nvcc (flashnext_attn.fatbin) w +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/build/fmt.log b/docs/receipts/glm-kv-pages-8e705835/tp2/build/fmt.log new file mode 100644 index 0000000000..e69de29bb2 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/build/host.log b/docs/receipts/glm-kv-pages-8e705835/tp2/build/host.log new file mode 100644 index 0000000000..a71f8cb33f --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/build/host.log @@ -0,0 +1,677 @@ +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/a28b555d9dcc808b85766f1b640d123b/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/d8e707fd3e041e02c6aef984e8beea66/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/e05e415a6fa69a267b4b17ff06630cc4/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/78bea61ec4f6eee37e5ad757dc454d5e/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +[default] (err): rank 0: another rank's process ended (its line closed); this rank stops instead of waiting on it +[default] (err): rank 0: rank 1 stopped (peer OOM); this rank stops too +failed command: ./.zig-cache/o/55830d4bb5be391358eedc8f2e55aead/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/80498000c83d5242022cdbc134bfde66/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +[default] (warn): libcublasLt.so.13 is required for CUDA vision towers (--vision) and CUDA cuBLASLt projections; install the NVIDIA CUDA 13 cuBLAS runtime; on Linux, add its directory to LD_LIBRARY_PATH; a driver-only host does not include it +[default] (warn): libnccl.so.2 is required for CUDA ranks (including the GLM pair); install the NVIDIA NCCL 2 runtime compatible with CUDA 13; on Linux, add its directory to LD_LIBRARY_PATH +[default] (warn): libibverbs.so.1 is required for RoCE/RDMA gathers; install rdma-core's libibverbs runtime and the provider for the network adapter +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +failed command: ./.zig-cache/o/9f68af4ccd7bd72c6d05bb87559e21af/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +DEPTH_REPLAY n=2 intercept=50 slope=3 head=1 probe=false deepest=6 crossed={ 62, 62 } last={ 6, 6 } trimmed=0 +DEPTH_REPLAY n=2 intercept=50 slope=3 head=1 probe=true deepest=8 crossed={ 59, 57 } last={ 7, 8 } trimmed=11 +DEPTH_REPLAY n=4 intercept=50 slope=3 head=1 probe=false deepest=4 crossed={ 83, 83, 83, 83 } last={ 4, 4, 4, 4 } trimmed=0 +DEPTH_REPLAY n=4 intercept=50 slope=3 head=1 probe=true deepest=5 crossed={ 75, 83, 83, 76 } last={ 5, 5, 5, 5 } trimmed=9 +DEPTH_REPLAY n=2 intercept=55 slope=2.8 head=1 probe=false deepest=6 crossed={ 61, 61 } last={ 6, 6 } trimmed=0 +DEPTH_REPLAY n=2 intercept=55 slope=2.8 head=1 probe=true deepest=8 crossed={ 54, 56 } last={ 8, 8 } trimmed=2 +DEPTH_REPLAY n=4 intercept=55 slope=2.8 head=1 probe=false deepest=4 crossed={ 82, 82, 82, 82 } last={ 4, 4, 4, 4 } trimmed=0 +DEPTH_REPLAY n=4 intercept=55 slope=2.8 head=1 probe=true deepest=6 crossed={ 75, 71, 72, 74 } last={ 5, 6, 5, 5 } trimmed=9 +DEPTH_REPLAY n=8 intercept=100 slope=0 head=0 probe=false deepest=3 crossed={ 100, 100, 100, 100, 100, 100, 100, 100 } last={ 3, 3, 3, 3, 3, 3, 3, 3 } trimmed=0 +DEPTH_REPLAY n=8 intercept=100 slope=0 head=0 probe=true deepest=3 crossed={ 100, 100, 100, 100, 100, 100, 100, 100 } last={ 3, 3, 3, 3, 3, 3, 3, 3 } trimmed=0 +failed command: ./.zig-cache/o/62ed3ca445191230b0f77fda2a4229a8/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +[default] (warn): slide pocket: a deleted fact's 1 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a deleted fact's 1 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a delete failed (UploadFailed), and its put-back did not reach the device (PutBackNotOnDevice) +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a delete failed (UploadFailed), and its put-back did not reach the device (PutBackNotOnDevice) +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +failed command: ./.zig-cache/o/320bae808114722277b84ec2a9a69002/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/b464ac90f976fdf4c624332a585536ce/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zig-cache/o/06016b4095ec7ae48e3f03f7f7ca710d/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/c3f4ee9f92155a228a3554058e110091/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/556123d28496d1cefe7cf53ba86632e5/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/a8dc8f3d17c2d0d217c7f824aa1d3537/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +failed command: ./.zig-cache/o/46e19a6c1fca2865b833323c7095c543/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/8a4d02bb9d3c4e71d1e5e01e68a5f951/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +failed command: ./.zig-cache/o/10c56d7015a0164d0244c527eaf3551e/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/bd9a9f03edfdd1dc3dc02cd0c4950c00/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +[default] (warn): lane host: stopping at a round boundary: 2 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): 4 prompts together: SamplingRefused; each goes alone +[default] (warn): lane host: a round failed (the planner refused): every admitted stream fails +[default] (warn): lane host: a round failed (PlannerFull): every admitted stream fails +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (drain deadline) +[default] (warn): lane host: stopping at a round boundary: 0 running and 1 queued request(s) end (NoContext) +[default] (warn): lane host: a round failed (OutOfMemory): every admitted stream fails +[default] (warn): lane host: a round failed (AdmittedFailed): every admitted stream fails +[default] (warn): lane host: a round failed (FinalsFailed): every admitted stream fails +failed command: ./.zig-cache/o/4326653e6bd8ad6fdc5508bee4ba3a38/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +test-cuda-host ++- run test w +PROVENANCE_FIELDS fixtures=3 fields=14 plus decision labels +PROVENANCE_TIMING mode=raw one=0 bytes=409600 added=1000 rounds=12 ns_per=6048145 +PROVENANCE_TIMING mode=masked one=0 bytes=409600 added=1000 rounds=12 ns_per=6421471 +PROVENANCE_MASK_WORK one=0 spans=3 matcher=3 output=409603 +PROVENANCE_TIMING mode=raw one=1 bytes=409600 added=1000 rounds=12 ns_per=6231768 +PROVENANCE_TIMING mode=masked one=1 bytes=409600 added=1000 rounds=12 ns_per=6429743 +PROVENANCE_MASK_WORK one=1 spans=3 matcher=3 output=409603 +[default] (warn): TENSORFOLD_REQUEST_LOG .zig-cache/tmp/_2Tp58NifJI1Alta/taken.jsonl gets no requests: it is not a regular file only its owner can read +VISION_MARKERS all 8 fields refuse plain vocabulary; unflagged additions pass +failed command: ./.zig-cache/o/da1ace8218d407d58b4a1b7a9ef6c22b/test --cache-dir=./.zig-cache --seed=0xf03f5e41 --listen=- + +Build Summary: 155/155 steps succeeded; 2516/2522 tests passed (6 skipped) +test-cuda-host success ++- run test 1 pass (1 total) 15ms MaxRSS:13M +| +- compile test debug native success 3s MaxRSS:401M ++- run test 1 pass (1 total) 11ms MaxRSS:8M +| +- compile test debug native success 4s MaxRSS:325M ++- run test 10 pass (10 total) 14ms MaxRSS:9M +| +- compile test debug native success 5s MaxRSS:367M ++- run test 2 pass (2 total) 9ms MaxRSS:18M +| +- compile test debug native success 2s MaxRSS:328M ++- run exe runtime-discovery-exit success 17ms +| +- compile exe runtime-discovery-exit debug native success 1s MaxRSS:329M +| +- options success +| +- compile lib runtime-discovery-mock debug native success 7s MaxRSS:463M ++- run exe rpc-warning-check success 6ms +| +- compile exe rpc-warning-check debug native success 2s MaxRSS:309M +| +- options success +| +- compile lib rpc-mock-complete debug native success 7s MaxRSS:266M +| +- compile lib rpc-mock-abi debug native success 80ms MaxRSS:62M +| +- compile lib rpc-mock-wait debug native success 72ms MaxRSS:62M +| +- compile lib rpc-mock-store debug native success 7s MaxRSS:475M +| +- compile lib rpc-mock-wrong debug native success 8s MaxRSS:514M ++- run test 11 pass (11 total) 15ms MaxRSS:13M +| +- compile test debug native success 2s MaxRSS:360M +| +- options (+5 more reused dependencies) ++- run test 1 pass (1 total) 7ms MaxRSS:8M +| +- compile test debug native success 4s MaxRSS:330M ++- run test 30 pass (30 total) 50ms MaxRSS:18M +| +- compile test debug native success 3s MaxRSS:464M +| +- WriteFile h3-compile-only.fatbin success +| +- options cached ++- run test 3 pass (3 total) 460ms MaxRSS:72M +| +- compile test debug native success 2s MaxRSS:351M ++- run test 3 pass (3 total) 13ms MaxRSS:9M +| +- compile test debug native success 6s MaxRSS:355M +| +- WriteFile attn_train.fatbin success +| +- WriteFile dsv41_exl3_experts.fatbin success +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- WriteFile dsv41_exl3_experts.fatbin (reused) +| +- options cached +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- options cached +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) +| +- WriteFile attn_train.fatbin (reused) ++- run test 3 pass (3 total) 122ms MaxRSS:17M +| +- compile test debug native success 2s MaxRSS:406M +| +- options success +| | +- compile exe qwen-round-timing-probe debug native success 3s MaxRSS:425M +| | +- WriteFile draft_ids.txt cached +| | +- options (reused) +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test 10 pass (10 total) 12ms MaxRSS:16M +| +- compile test debug native success 2s MaxRSS:351M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test 7 pass (7 total) 14ms MaxRSS:10M +| +- compile test debug native success 2s MaxRSS:397M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run test 6 pass (6 total) 34ms MaxRSS:15M +| +- compile test debug native success 3s MaxRSS:389M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run test 10 pass (10 total) 390ms MaxRSS:14M +| +- compile test debug native success 4s MaxRSS:533M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test 9 pass (9 total) 55ms MaxRSS:10M +| +- compile test debug native success 2s MaxRSS:378M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- compile obj exl3_host_compile debug native success 211ms MaxRSS:119M +| +- options (reused) ++- run test 6 pass (6 total) 15ms MaxRSS:16M +| +- compile test debug native success 2s MaxRSS:386M +| +- options (reused) ++- run test 6 pass (6 total) 88ms MaxRSS:10M +| +- compile test debug native success 2s MaxRSS:371M +| +- options (reused) ++- run test 5 pass (5 total) 18ms MaxRSS:15M +| +- compile test debug native success 3s MaxRSS:354M +| +- options (reused) ++- run test 3 pass (3 total) 8ms MaxRSS:17M +| +- compile test debug native success 2s MaxRSS:352M +| +- options (reused) ++- run test 3 pass (3 total) 12ms MaxRSS:13M +| +- compile test debug native success 2s MaxRSS:340M +| +- options (reused) ++- run test 3 pass (3 total) 15ms MaxRSS:14M +| +- compile test debug native success 2s MaxRSS:340M +| +- options (reused) ++- run test 8 pass (8 total) 17ms MaxRSS:8M +| +- compile test debug native success 4s MaxRSS:346M ++- run test 3 pass (3 total) 96ms MaxRSS:18M +| +- compile test debug native success 2s MaxRSS:341M ++- run test 8 pass (8 total) 32ms MaxRSS:14M +| +- compile test debug native success 2s MaxRSS:353M ++- run test 4 pass (4 total) 12ms MaxRSS:8M +| +- compile test debug native success 4s MaxRSS:357M +| +- options (reused) ++- run test 1 pass (1 total) 82ms MaxRSS:8M +| +- compile test debug native success 4s MaxRSS:353M +| +- options (reused) ++- run test 4 pass (4 total) 2ms MaxRSS:12M +| +- compile test fast native success 9s MaxRSS:543M +| +- options (reused) ++- run test 1 pass (1 total) 8ms MaxRSS:18M +| +- compile test fast native success 10s MaxRSS:563M +| +- options (reused) ++- compile obj bolt_host_compile debug aarch64-linux-gnu success 443ms MaxRSS:143M +| +- options cached +| +- WriteFile draft_ids.txt (reused) ++- run test 26 pass (26 total) 4s MaxRSS:17M +| +- compile test debug native success 3s MaxRSS:469M +| +- options (reused) ++- run test 4 pass (4 total) 12ms MaxRSS:8M +| +- compile test debug native success 4s MaxRSS:354M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test 3 pass (3 total) 2s MaxRSS:15M +| +- compile test debug native success 5s MaxRSS:587M +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run exe qwen-cost-key-process success 1s +| +- compile exe qwen-cost-key-process debug native success 2s MaxRSS:390M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- WriteFile target/config.json success ++- run test 1 pass (1 total) 14ms MaxRSS:18M +| +- compile test debug native success 2s MaxRSS:381M ++- compile obj drama_cuda_native_host debug native success 387ms MaxRSS:130M +| +- options (reused) ++- run test 5 pass (5 total) 15ms MaxRSS:14M +| +- compile test debug native success 2s MaxRSS:338M ++- run test 5 pass (5 total) 173ms MaxRSS:14M +| +- compile test debug native success 3s MaxRSS:398M ++- run test 4 pass (4 total) 18ms MaxRSS:13M +| +- compile test debug native success 2s MaxRSS:380M ++- run test 604 pass (604 total) 47s MaxRSS:54M +| +- compile test debug native success 21s MaxRSS:2G ++- run test 5 pass (5 total) 9ms MaxRSS:12M +| +- compile test debug native success 2s MaxRSS:351M ++- run test 218 pass (218 total) 416ms MaxRSS:12M +| +- compile test debug native success 4s MaxRSS:557M +| +- options (reused) ++- run test 352 pass (352 total) 6s MaxRSS:133M +| +- compile test debug native success 10s MaxRSS:832M ++- run test 212 pass (212 total) 6s MaxRSS:45M +| +- compile test debug native success 7s MaxRSS:616M ++- run test 74 pass, 1 skip (75 total) 111ms MaxRSS:18M +| +- compile test debug native success 5s MaxRSS:638M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test 58 pass, 2 skip (60 total) 7s MaxRSS:25M +| +- compile test debug native success 3s MaxRSS:465M +| +- WriteFile draft_ids.txt (reused) +| +- options (reused) ++- run test 12 pass (12 total) 21ms MaxRSS:11M +| +- compile test debug native success 3s MaxRSS:411M +| +- options (reused) ++- run test 97 pass (97 total) 307ms MaxRSS:77M +| +- compile test debug native success 5s MaxRSS:669M +| +- options (reused) ++- run test 15 pass (15 total) 4s MaxRSS:16M +| +- compile test debug native success 4s MaxRSS:478M +| +- options (reused) ++- run test 29 pass (29 total) 3s MaxRSS:1G +| +- compile test debug native success 3s MaxRSS:418M +| +- options (reused) ++- run test 5 pass (5 total) 13ms MaxRSS:12M +| +- compile test debug native success 2s MaxRSS:349M ++- run test 36 pass (36 total) 116ms MaxRSS:9M +| +- compile test debug native success 5s MaxRSS:433M ++- run test 238 pass (238 total) 9s MaxRSS:42M +| +- compile test debug native success 6s MaxRSS:700M ++- run test 73 pass (73 total) 104ms MaxRSS:30M +| +- compile test debug native success 9s MaxRSS:1G +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) ++- run test 3 pass (3 total) 7ms MaxRSS:17M +| +- compile test debug native success 1s MaxRSS:325M ++- run test 1 pass (1 total) 12ms MaxRSS:8M +| +- compile test debug native success 4s MaxRSS:336M ++- run test 12 pass (12 total) 53ms MaxRSS:9M +| +- compile test debug native success 4s MaxRSS:369M +| +- options (reused) ++- run test 9 pass (9 total) 17ms MaxRSS:14M +| +- compile test debug native success 2s MaxRSS:398M +| +- options (reused) ++- run test 2 pass (2 total) 25ms MaxRSS:12M +| +- compile test debug native success 4s MaxRSS:607M +| +- options (reused) ++- run test 88 pass (88 total) 100ms MaxRSS:16M +| +- compile test debug native success 5s MaxRSS:615M +| +- options (reused) ++- run test 19 pass, 1 skip (20 total) 42ms MaxRSS:17M +| +- compile test debug native success 3s MaxRSS:444M +| +- options (reused) ++- run test 5 pass (5 total) 13ms MaxRSS:18M +| +- compile test debug native success 2s MaxRSS:327M ++- run test 34 pass (34 total) 9s MaxRSS:12M +| +- compile test debug native success 5s MaxRSS:414M ++- run test 55 pass (55 total) 1s MaxRSS:13M +| +- compile test debug native success 4s MaxRSS:537M ++- run test 49 pass, 2 skip (51 total) 1s MaxRSS:18M +| +- compile test debug native success 3s MaxRSS:489M ++- run test 1 pass (1 total) 11ms MaxRSS:10M + +- compile test debug native success 1s MaxRSS:366M + +- options cached + +- options (reused) + +- WriteFile draft_ids.txt (reused) + +- WriteFile exlr8_alias.zig cached diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/build/kvpool.log b/docs/receipts/glm-kv-pages-8e705835/tp2/build/kvpool.log new file mode 100644 index 0000000000..db95bdccf0 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/build/kvpool.log @@ -0,0 +1,6 @@ +Build Summary: 5/5 steps succeeded; 17/17 tests passed +test-kv-pool success ++- run test 11 pass (11 total) 8ms MaxRSS:5M +| +- compile test debug native success 2s MaxRSS:337M ++- run test 6 pass (6 total) 547ms MaxRSS:9M + +- compile test debug native success 4s MaxRSS:499M diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/build/lean.log b/docs/receipts/glm-kv-pages-8e705835/tp2/build/lean.log new file mode 100644 index 0000000000..3666d6027b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/build/lean.log @@ -0,0 +1,439 @@ +zig/build/hip.zig:1: comment past 120 columns +zig/kernels/cuda/experts.cu:1: comment past 120 columns +zig/kernels/cuda/experts_pack.cu:1: comment past 120 columns +zig/kernels/cuda/experts_prefill.cu:1: comment past 120 columns +zig/kernels/cuda/gdn.cu:1: comment past 120 columns +zig/kernels/cuda/qmm_frag.cuh:1: comment past 120 columns +zig/kernels/cuda/qmm_group.cu:1: comment past 120 columns +zig/kernels/cuda/qmm_prefill.cu:1: comment past 120 columns +zig/kernels/hip/affine.hip:1: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:1: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:368: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:473: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:503: comment past 120 columns +zig/kernels/metal/core/affine_mm.metal:521: block comment +zig/kernels/metal/core/hc_boundary.metal:5: comment past 120 columns +zig/kernels/metal/core/hc_boundary.metal:13: comment past 120 columns +zig/kernels/metal/core/hc_boundary.metal:82: comment past 120 columns +zig/kernels/metal/core/hc_boundary.metal:109: comment past 120 columns +zig/kernels/metal/core/moe_route.metal:1: comment past 120 columns +zig/kernels/metal/core/moe_route.metal:11: comment past 120 columns +zig/kernels/metal/core/moe_route.metal:68: comment past 120 columns +zig/kernels/metal/core/row_projection.metal:8: comment past 120 columns +zig/kernels/metal/core/row_projection.metal:72: comment past 120 columns +zig/kernels/metal/core/row_projection.metal:161: comment past 120 columns +zig/kernels/metal/decode/fn_gdn.metal:1: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:2: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:3: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:4: multi-line comment +zig/kernels/metal/decode/fn_gdn.metal:5: multi-line comment +zig/kernels/metal/glm_absorb_nax.metal:1: comment past 120 columns +zig/kernels/metal/glm_absorb_nax.metal:13: comment past 120 columns +zig/kernels/metal/glm_attn.metal:1: comment past 120 columns +zig/kernels/metal/glm_attn.metal:12: comment past 120 columns +zig/kernels/metal/glm_attn.metal:54: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:1: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:27: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:107: comment past 120 columns +zig/kernels/metal/glm_index_nax.metal:116: comment past 120 columns +zig/kernels/metal/glm_kda_prompt.metal:2: comment past 120 columns +zig/kernels/metal/glm_kda_prompt.metal:4: comment past 120 columns +zig/kernels/metal/glm_kda_prompt.metal:65: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:1: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:41: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:170: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:201: comment past 120 columns +zig/kernels/metal/glm_sparse_nax.metal:235: comment past 120 columns +zig/kernels/metal/kimi/dense.metal:10: comment past 120 columns +zig/kernels/metal/kimi/experts.metal:86: comment past 120 columns +zig/kernels/metal/nemotron_experts.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_experts.metal:97: comment past 120 columns +zig/kernels/metal/nemotron_glue.metal:172: comment past 120 columns +zig/kernels/metal/nemotron_head.metal:20: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:7: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:189: comment past 120 columns +zig/kernels/metal/nemotron_round.metal:298: comment past 120 columns +zig/kernels/metal/nemotron_sample.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_sample.metal:145: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:1: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:8: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:134: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:149: comment past 120 columns +zig/kernels/metal/nemotron_tree.metal:313: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:1: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:28: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:64: comment past 120 columns +zig/kernels/metal/prefill/fn_attn.metal:198: comment past 120 columns +zig/kernels/metal/prefill/fn_prompt.metal:1: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:2: comment past 120 columns +zig/kernels/metal/prefill/fn_prompt.metal:128: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:137: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:200: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:201: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:253: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:254: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:255: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:318: multi-line comment +zig/kernels/metal/prefill/fn_prompt.metal:319: multi-line comment +zig/kernels/metal/prefill/fn_select.metal:1: comment past 120 columns +zig/kernels/metal/prefill/fn_select.metal:14: comment past 120 columns +zig/kernels/metal/prefill/fn_select.metal:39: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:1: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:44: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:117: comment past 120 columns +zig/kernels/metal/prefill/glue.metal:159: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:1: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:47: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:111: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:271: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:311: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax.metal:334: comment past 120 columns +zig/kernels/metal/prefill/qmm6_nax_b.metal:102: comment past 120 columns +zig/kernels/metal/prefill/sort.metal:52: comment past 120 columns +zig/kernels/metal/sources.zig:57: multi-line comment +zig/src/cli/cuda_segments.zig:1: comment past 120 columns +zig/src/cli/cuda_segments.zig:22: comment past 120 columns +zig/src/cli/info.zig:13: multi-line comment +zig/src/cli/pull.zig:232: multi-line comment +zig/src/cluster/admission.zig:1: comment past 120 columns +zig/src/cluster/admission_test.zig:1: comment past 120 columns +zig/src/cluster/backend.zig:1: comment past 120 columns +zig/src/cluster/backend.zig:11: comment past 120 columns +zig/src/cluster/backend.zig:101: comment past 120 columns +zig/src/cluster/backend_test.zig:1: comment past 120 columns +zig/src/cluster/budget.zig:1: comment past 120 columns +zig/src/cluster/canon.zig:1: comment past 120 columns +zig/src/cluster/canon.zig:14: comment past 120 columns +zig/src/cluster/canon.zig:75: comment past 120 columns +zig/src/cluster/cli_model.zig:98: comment past 120 columns +zig/src/cluster/cluster.zig:1: comment past 120 columns +zig/src/cluster/config.zig:1: comment past 120 columns +zig/src/cluster/cost.zig:1: comment past 120 columns +zig/src/cluster/estimate.zig:1: comment past 120 columns +zig/src/cluster/exchange.zig:1: comment past 120 columns +zig/src/cluster/exchange.zig:179: comment past 120 columns +zig/src/cluster/fabric_link.zig:1: comment past 120 columns +zig/src/cluster/fabric_link_test.zig:1: comment past 120 columns +zig/src/cluster/launch.zig:1: comment past 120 columns +zig/src/cluster/loader.zig:1: comment past 120 columns +zig/src/cluster/loader.zig:55: comment past 120 columns +zig/src/cluster/loader_test.zig:1: comment past 120 columns +zig/src/cluster/membership.zig:1: comment past 120 columns +zig/src/cluster/membership_test.zig:1: comment past 120 columns +zig/src/cluster/node.zig:83: comment past 120 columns +zig/src/cluster/plan.zig:1: comment past 120 columns +zig/src/cluster/plan.zig:12: comment past 120 columns +zig/src/cluster/plan.zig:15: comment past 120 columns +zig/src/cluster/plan.zig:116: comment past 120 columns +zig/src/cluster/plan.zig:176: comment past 120 columns +zig/src/cluster/probe_fixtures.zig:1: comment past 120 columns +zig/src/cluster/pull.zig:1: comment past 120 columns +zig/src/cluster/pull_test.zig:1: comment past 120 columns +zig/src/cluster/round.zig:1: comment past 120 columns +zig/src/cluster/status.zig:1: comment past 120 columns +zig/src/cluster/toy.zig:1: comment past 120 columns +zig/src/cluster/toy.zig:93: comment past 120 columns +zig/src/cluster/toy.zig:193: comment past 120 columns +zig/src/cluster/traffic.zig:1: comment past 120 columns +zig/src/cluster/traffic.zig:18: comment past 120 columns +zig/src/cluster/traffic.zig:38: comment past 120 columns +zig/src/cluster/traffic.zig:49: comment past 120 columns +zig/src/cluster/wire.zig:1: comment past 120 columns +zig/src/core/affine_mm.zig:1: comment past 120 columns +zig/src/core/affine_mm.zig:25: comment past 120 columns +zig/src/core/affine_mm.zig:33: comment past 120 columns +zig/src/core/affine_mm.zig:54: comment past 120 columns +zig/src/core/affine_mm.zig:60: comment past 120 columns +zig/src/core/affine_mm.zig:75: comment past 120 columns +zig/src/core/affine_mm.zig:99: comment past 120 columns +zig/src/core/checkpoint.zig:1: comment past 120 columns +zig/src/core/checkpoint.zig:11: comment past 120 columns +zig/src/core/checkpoint_metal.zig:65: comment past 120 columns +zig/src/core/checkpoint_metal.zig:178: comment past 120 columns +zig/src/core/copy_index.zig:1: comment past 120 columns +zig/src/core/hc.zig:1: comment past 120 columns +zig/src/core/hc.zig:30: comment past 120 columns +zig/src/core/lanes/fill.zig:26: comment past 120 columns +zig/src/core/lanes/fill.zig:54: comment past 120 columns +zig/src/core/lanes/gpu_full.zig:40: comment past 120 columns +zig/src/core/lanes/gpu_full.zig:114: comment past 120 columns +zig/src/core/lanes/gpu_full_test.zig:1: comment past 120 columns +zig/src/core/moe_route.zig:1: comment past 120 columns +zig/src/core/moe_route.zig:33: comment past 120 columns +zig/src/core/paced_read.zig:5: comment past 120 columns +zig/src/core/prompt_cache.zig:26: comment past 120 columns +zig/src/core/prompt_cache.zig:176: comment past 120 columns +zig/src/core/prompt_cache.zig:300: comment past 120 columns +zig/src/core/prompt_cache.zig:349: comment past 120 columns +zig/src/core/prompt_cache_test.zig:1: comment past 120 columns +zig/src/core/row_projection.zig:21: comment past 120 columns +zig/src/core/row_projection.zig:24: comment past 120 columns +zig/src/core/row_projection.zig:96: comment past 120 columns +zig/src/core/safetensors.zig:64: comment past 120 columns +zig/src/core/segments.zig:1: comment past 120 columns +zig/src/core/segments.zig:26: comment past 120 columns +zig/src/core/segments.zig:61: comment past 120 columns +zig/src/core/stagger.zig:7: comment past 120 columns +zig/src/core/stagger.zig:22: comment past 120 columns +zig/src/core/stagger.zig:144: comment past 120 columns +zig/src/core/template/template.zig:103: comment past 120 columns +zig/src/core/tokenizer/regex.zig:1: comment past 120 columns +zig/src/core/tokenizer/regex.zig:272: comment past 120 columns +zig/src/core/tokenizer/tokenizer.zig:1: comment past 120 columns +zig/src/core/tokenizer/unicode.zig:224: comment past 120 columns +zig/src/cuda/cublaslt.zig:1: comment past 120 columns +zig/src/cuda/segments.zig:142: comment past 120 columns +zig/src/cuda/triton.zig:72: comment past 120 columns +zig/src/fabric/collective.zig:1: comment past 120 columns +zig/src/fabric/collective.zig:130: comment past 120 columns +zig/src/fabric/collective.zig:183: comment past 120 columns +zig/src/fabric/cost.zig:34: comment past 120 columns +zig/src/fabric/cost.zig:43: comment past 120 columns +zig/src/fabric/link_fake.zig:8: comment past 120 columns +zig/src/fabric/mailbox.zig:59: comment past 120 columns +zig/src/fabric/manifest.zig:1: comment past 120 columns +zig/src/fabric/mcdma.zig:160: comment past 120 columns +zig/src/fabric/packet.zig:1: comment past 120 columns +zig/src/fabric/prefill_source.zig:18: comment past 120 columns +zig/src/fabric/verbs_abi.zig:1: comment past 120 columns +zig/src/fabric/verbs_link.zig:1: comment past 120 columns +zig/src/fabric/words.zig:142: comment past 120 columns +zig/src/families/flashnext/call_log.zig:1: comment past 120 columns +zig/src/families/flashnext/follow.zig:51: multi-line comment +zig/src/families/flashnext/follow.zig:51: comment past 120 columns +zig/src/families/flashnext/follow.zig:136: comment past 120 columns +zig/src/families/flashnext/follow.zig:147: comment past 120 columns +zig/src/families/flashnext/follow.zig:150: comment past 120 columns +zig/src/families/flashnext/gdn.zig:1: multi-line comment +zig/src/families/flashnext/gdn.zig:13: multi-line comment +zig/src/families/flashnext/gdn.zig:32: multi-line comment +zig/src/families/flashnext/marks.zig:1: comment past 120 columns +zig/src/families/flashnext/marks.zig:13: comment past 120 columns +zig/src/families/flashnext/marks.zig:40: comment past 120 columns +zig/src/families/flashnext/marks.zig:62: comment past 120 columns +zig/src/families/flashnext/marks.zig:72: comment past 120 columns +zig/src/families/flashnext/marks.zig:103: comment past 120 columns +zig/src/families/flashnext/marks.zig:121: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:17: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:20: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:25: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:49: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:108: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:155: comment past 120 columns +zig/src/families/flashnext/snapshot.zig:205: comment past 120 columns +zig/src/families/flashnext/split.zig:1: comment past 120 columns +zig/src/families/flashnext/split.zig:8: comment past 120 columns +zig/src/families/flashnext/split.zig:62: comment past 120 columns +zig/src/families/flashnext/tp.zig:19: comment past 120 columns +zig/src/families/flashnext/tp.zig:216: multi-line comment +zig/src/families/flashnext/tp.zig:216: comment past 120 columns +zig/src/families/flashnext/tp.zig:247: multi-line comment +zig/src/families/flashnext/tp.zig:263: multi-line comment +zig/src/families/flashnext/tp.zig:283: comment past 120 columns +zig/src/families/flashnext/tp.zig:319: multi-line comment +zig/src/families/flashnext/tp.zig:320: multi-line comment +zig/src/families/flashnext/tp.zig:341: comment past 120 columns +zig/src/families/flashnext/tp.zig:371: comment past 120 columns +zig/src/families/flashnext/tp.zig:392: comment past 120 columns +zig/src/families/flashnext/tp.zig:407: comment past 120 columns +zig/src/families/flashnext/tp.zig:449: multi-line comment +zig/src/families/flashnext/tp.zig:450: multi-line comment +zig/src/families/flashnext/tp_settings.zig:8: comment past 120 columns +zig/src/families/flashnext/tp_settings.zig:39: comment past 120 columns +zig/src/families/glm/attn_check.zig:1: comment past 120 columns +zig/src/families/glm/attn_check.zig:127: comment past 120 columns +zig/src/families/glm/backend.zig:178: comment past 120 columns +zig/src/families/glm/backend.zig:326: comment past 120 columns +zig/src/families/glm/checks.zig:37: comment past 120 columns +zig/src/families/glm/checks.zig:72: comment past 120 columns +zig/src/families/glm/checks.zig:105: comment past 120 columns +zig/src/families/glm/checks.zig:153: comment past 120 columns +zig/src/families/glm/checks.zig:190: comment past 120 columns +zig/src/families/glm/config.zig:222: comment past 120 columns +zig/src/families/glm/engine.zig:1: comment past 120 columns +zig/src/families/glm/engine.zig:42: comment past 120 columns +zig/src/families/glm/engine.zig:105: comment past 120 columns +zig/src/families/glm/engine.zig:238: comment past 120 columns +zig/src/families/glm/engine.zig:455: comment past 120 columns +zig/src/families/glm/ep.zig:1: comment past 120 columns +zig/src/families/glm/ep.zig:25: comment past 120 columns +zig/src/families/glm/ep.zig:50: comment past 120 columns +zig/src/families/glm/ep.zig:204: comment past 120 columns +zig/src/families/glm/ep.zig:218: comment past 120 columns +zig/src/families/glm/ep.zig:309: comment past 120 columns +zig/src/families/glm/ep_control.zig:1: comment past 120 columns +zig/src/families/glm/ep_control.zig:120: comment past 120 columns +zig/src/families/glm/forward.zig:1: comment past 120 columns +zig/src/families/glm/forward.zig:197: comment past 120 columns +zig/src/families/glm/forward.zig:218: comment past 120 columns +zig/src/families/glm/forward.zig:286: comment past 120 columns +zig/src/families/glm/forward.zig:330: comment past 120 columns +zig/src/families/glm/forward.zig:393: comment past 120 columns +zig/src/families/glm/forward.zig:406: comment past 120 columns +zig/src/families/glm/forward.zig:536: comment past 120 columns +zig/src/families/glm/kept.zig:29: comment past 120 columns +zig/src/families/glm/kept.zig:81: comment past 120 columns +zig/src/families/glm/kept.zig:105: comment past 120 columns +zig/src/families/glm/kept.zig:218: comment past 120 columns +zig/src/families/glm/kernels.zig:1: comment past 120 columns +zig/src/families/glm/load_plan.zig:1: comment past 120 columns +zig/src/families/glm/load_plan.zig:99: comment past 120 columns +zig/src/families/glm/moe.zig:18: comment past 120 columns +zig/src/families/glm/moe.zig:150: comment past 120 columns +zig/src/families/glm/mtp.zig:1: comment past 120 columns +zig/src/families/glm/mtp.zig:17: comment past 120 columns +zig/src/families/glm/plain_rounds.zig:14: comment past 120 columns +zig/src/families/glm/plain_rounds.zig:53: comment past 120 columns +zig/src/families/glm/prompt.zig:1: comment past 120 columns +zig/src/families/glm/prompt.zig:21: comment past 120 columns +zig/src/families/glm/prompt.zig:134: comment past 120 columns +zig/src/families/glm/prompt.zig:156: comment past 120 columns +zig/src/families/glm/prompt.zig:218: comment past 120 columns +zig/src/families/glm/prompt.zig:246: comment past 120 columns +zig/src/families/glm/prompt.zig:342: comment past 120 columns +zig/src/families/glm/prompt.zig:393: comment past 120 columns +zig/src/families/glm/prompt.zig:417: comment past 120 columns +zig/src/families/glm/prompt.zig:432: comment past 120 columns +zig/src/families/glm/prompt.zig:553: comment past 120 columns +zig/src/families/glm/snapshot.zig:1: comment past 120 columns +zig/src/families/glm/snapshot.zig:59: comment past 120 columns +zig/src/families/glm/snapshot.zig:91: comment past 120 columns +zig/src/families/glm/weights.zig:1: comment past 120 columns +zig/src/families/kimi_k3/kernels.zig:283: comment past 120 columns +zig/src/families/kimi_k3/prepare.zig:1: comment past 120 columns +zig/src/families/nemotron/backend.zig:301: comment past 120 columns +zig/src/families/nemotron/backend.zig:421: comment past 120 columns +zig/src/families/nemotron/copy_lanes.zig:1: comment past 120 columns +zig/src/families/nemotron/copy_lanes.zig:66: comment past 120 columns +zig/src/families/nemotron/cuda_mtp.zig:372: comment past 120 columns +zig/src/families/nemotron/cuda_segments.zig:1: comment past 120 columns +zig/src/families/nemotron/cuda_segments.zig:64: comment past 120 columns +zig/src/families/nemotron/cuda_triton.zig:109: comment past 120 columns +zig/src/families/nemotron/cuda_triton.zig:116: comment past 120 columns +zig/src/families/nemotron/forward.zig:35: comment past 120 columns +zig/src/families/nemotron/head_block.zig:21: comment past 120 columns +zig/src/families/nemotron/layers.zig:61: comment past 120 columns +zig/src/families/nemotron/mtp.zig:193: comment past 120 columns +zig/src/families/nemotron/prefill_launch.zig:59: comment past 120 columns +zig/src/families/nemotron/prefill_launch.zig:137: comment past 120 columns +zig/src/families/nemotron/prefill_mixers.zig:1: comment past 120 columns +zig/src/families/nemotron/round_depth.zig:1: comment past 120 columns +zig/src/families/nemotron/simd_attention.zig:1: comment past 120 columns +zig/src/families/nemotron/simd_attention.zig:5: comment past 120 columns +zig/src/families/nemotron/simd_attention.zig:88: comment past 120 columns +zig/src/families/qwen3_5/config.zig:1: comment past 120 columns +zig/src/families/qwen3_5/qualify.zig:1: comment past 120 columns +zig/src/families/qwen3_5/weights.zig:1: comment past 120 columns +zig/src/hip/device_arch.c:1: block comment +zig/src/hip/driver.zig:1: comment past 120 columns +zig/src/hip/memory.zig:1: comment past 120 columns +zig/src/main.zig:226: comment past 120 columns +zig/src/native/flashnext_host.zig:445: comment past 120 columns +zig/src/native/metal.zig:318: comment past 120 columns +zig/src/server/chunk_plan.zig:1: comment past 120 columns +zig/src/server/hf_text.zig:23: comment past 120 columns +zig/src/server/log.zig:106: comment past 120 columns +zig/src/server/messages.zig:25: comment past 120 columns +zig/src/server/messages.zig:151: comment past 120 columns +zig/src/server/metrics.zig:325: multi-line comment +zig/src/server/model_text.zig:103: comment past 120 columns +zig/src/server/tool_params.zig:120: comment past 120 columns +zig/src/server/tool_parse.zig:1: comment past 120 columns +zig/src/server/tool_stream.zig:417: comment past 120 columns +zig/src/server/tool_streaming.zig:1: comment past 120 columns +zig/tests/cuda/bench.zig:17: comment past 120 columns +zig/tests/cuda/box/run.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/box/build.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/box/capture.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/box/zrun.sh:2: comment past 120 columns +zig/tests/cuda/nemotron/capture.py:2: docstring past 120 columns +zig/tests/cuda/nemotron/test_aot_manifest.py:2: multi-line docstring +zig/tests/cuda/oracle/oracle.py:1: docstring past 120 columns +zig/tests/cuda/oracle/oracle.py:82: docstring past 120 columns +zig/tests/cuda/oracle/sass_compare.py:1: docstring past 120 columns +zig/tests/cuda/runtime_tests.zig:276: comment past 120 columns +zig/tests/dispatch_bench.zig:1: comment past 120 columns +zig/tests/fabric_handoff.zig:1: comment past 120 columns +zig/tests/fabric_library.zig:1: comment past 120 columns +zig/tests/fabric_vectors.zig:1: comment past 120 columns +zig/tests/flashnext_bench.zig:69: comment past 120 columns +zig/tests/flashnext_bench.zig:119: multi-line comment +zig/tests/flashnext_bench.zig:169: multi-line comment +zig/tests/flashnext_reuse.zig:93: comment past 120 columns +zig/tests/flashnext_reuse.zig:183: comment past 120 columns +zig/tests/flashnext_run.zig: 1690 lines (split past 600) +zig/tests/flashnext_run.zig:1: multi-line comment +zig/tests/flashnext_run.zig:2: multi-line comment +zig/tests/flashnext_run.zig:762: multi-line comment +zig/tests/flashnext_run.zig:828: multi-line comment +zig/tests/flashnext_run.zig:1308: multi-line comment +zig/tests/flashnext_run.zig:1393: multi-line comment +zig/tests/flashnext_run.zig:1441: multi-line comment +zig/tests/flashnext_run.zig:1442: multi-line comment +zig/tests/glm_run.zig:1: comment past 120 columns +zig/tests/grid_sync_bench.zig:1: comment past 120 columns +zig/tests/hip_mock.c:1: block comment +zig/tests/linktest_suite.zig:1: comment past 120 columns +zig/tests/mm_bench.zig:1: comment past 120 columns +zig/tests/moe_bench.zig:1: comment past 120 columns +zig/tests/nemotron_dense.zig:1: comment past 120 columns +zig/tests/nemotron_dense.zig:11: comment past 120 columns +zig/tests/nemotron_dense.zig:100: comment past 120 columns +zig/tests/nemotron_dense.zig:292: comment past 120 columns +zig/tests/nemotron_experts.zig:1: comment past 120 columns +zig/tests/nemotron_experts.zig:133: comment past 120 columns +zig/tests/nemotron_prefill_check.zig:1: comment past 120 columns +zig/tests/nemotron_sample_check.zig:1: comment past 120 columns +zig/tests/nemotron_wide.zig:1: comment past 120 columns +zig/tests/nemotron_wide.zig:244: comment past 120 columns +zig/tests/ordering.zig:1: comment past 120 columns +zig/tests/ordering.zig:29: comment past 120 columns +zig/tests/row_projection_check.zig:1: comment past 120 columns +zig/tests/row_projection_check.zig:19: comment past 120 columns +zig/tests/row_projection_check.zig:163: comment past 120 columns +zig/tests/server/build.sh:2: comment past 120 columns +zig/tests/server/freeze_cases.py:1: docstring past 120 columns +zig/tests/server/freeze_cases.py:150: comment past 120 columns +zig/tests/server/golden_check.zig:79: comment past 120 columns +zig/tests/server/golden_check.zig:173: comment past 120 columns +zig/tests/server/wire.py:52: docstring past 120 columns +zig/tests/server/zig_test.sh:2: comment past 120 columns +zig/tests/tp2_bench.zig:1: comment past 120 columns +zig/tests/wide_heads.zig:46: comment past 120 columns +zig/tests/wide_probe.zig:1: comment past 120 columns +zig/tests/wide_probe.zig:263: comment past 120 columns +zig/tests/wide_trees.zig:1: comment past 120 columns +zig/tests/wide_trees.zig:126: comment past 120 columns +zig/tests/wide_trees.zig:156: comment past 120 columns +tools/zig/check_prefill_layers.py:1: docstring past 120 columns +tools/zig/check_prefill_model.py:1: docstring past 120 columns +tools/zig/check_prefill_ops.py:1: docstring past 120 columns +tools/zig/flashnext_dump.py:1: multi-line docstring +tools/zig/flashnext_hc_coalesce.py:1: multi-line docstring +tools/zig/gen_glm_kernels.py:43: comment past 120 columns +tools/zig/glm_compare.py:1: docstring past 120 columns +tools/zig/glm_ref.py:1: docstring past 120 columns +tools/zig/glm_ref.py:96: docstring past 120 columns +tools/zig/glm_ref.py:122: docstring past 120 columns +tools/zig/oracle_nemotron_rows.py:38: docstring past 120 columns +tools/zig/prefill_dump.py:1: docstring past 120 columns +tools/zig/prefill_glue.py:1: docstring past 120 columns +tools/zig/prefill_kernels.py:34: docstring past 120 columns +tools/zig/prefill_launch.py:1: docstring past 120 columns +tools/zig/prefill_launch.py:173: docstring past 120 columns +tools/zig/prefill_swap.py:1: docstring past 120 columns +tools/zig/reuse_served.py:133: docstring past 120 columns +tools/zig/reuse_served.py:232: comment past 120 columns +tools/zig/reuse_served.py:254: docstring past 120 columns +tools/zig/reuse_served.py:290: docstring past 120 columns +tools/zig/reuse_served.py:375: multi-line docstring +tools/zig/reuse_served.py:401: docstring past 120 columns +tools/zig/reuse_served.py:423: docstring past 120 columns +tools/zig/reuse_served.py:440: docstring past 120 columns +tools/zig/test_fabric.sh:2: comment past 120 columns +tools/zig/test_lanes.sh:2: comment past 120 columns +tools/zig/tokenizer_parity.py:1: docstring past 120 columns +tools/zig/triton_aot_manifest.py:2: docstring past 120 columns +tools/zig/unicode_tables.py:1: docstring past 120 columns +3599 files, 438 problems diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/build/native.log b/docs/receipts/glm-kv-pages-8e705835/tp2/build/native.log new file mode 100644 index 0000000000..1e36c8df90 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/build/native.log @@ -0,0 +1,126 @@ +native ++- install tensorfold-native + +- compile exe tensorfold-native safe native + +- run /usr/local/cuda/bin/nvcc (prefill_attention.fatbin) w +./zig/kernels/cuda/prefill_attention.cu(18): warning #177-D: variable "tf_prefill_attention::SMEM_MAX" was declared but never referenced + constexpr int SMEM_MAX = 96 * 1024; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +native ++- install tensorfold-native + +- compile exe tensorfold-native safe native + +- run /usr/local/cuda/bin/nvcc (drama3_fp8.fatbin) w +./zig/kernels/cuda/drama3_fp8.cu(188): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +native ++- install tensorfold-native + +- compile exe tensorfold-native safe native + +- run /usr/local/cuda/bin/nvcc (fp8_experts.fatbin) w +./zig/kernels/cuda/fp8_experts.cu(153): warning #177-D: variable "RW" was declared but never referenced + constexpr int RW = PromptCfg::RW; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + + +native ++- install tensorfold-native + +- compile exe tensorfold-native safe native + +- run /usr/local/cuda/bin/nvcc (flashnext_attn.fatbin) w +./zig/kernels/cuda/flashnext_attn.cu(394): warning #128-D: loop is not reachable + for (int h = 0; h < 2; ++h) { + ^ + detected during instantiation of "void attn_chunks(const __nv_bfloat16 *, const __nv_bfloat16 *, const __nv_bfloat16 *, const int *, float *, float *, float *, const int *, const int *, const int *, int, int, int, int, int, float, int, int) [with NB=1, ONE=true, FUSE=true]" at line 441 + +Remark: The warnings can be suppressed with "-diag-suppress " + + +native ++- install tensorfold-native + +- compile exe tensorfold-native safe native + +- run /usr/local/cuda/bin/nvcc (glm_mm.fatbin) w +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +Remark: The warnings can be suppressed with "-diag-suppress " + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=8, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=6, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=64, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=32, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=4, BM=16, WM=1, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + +./zig/kernels/cuda/glm_mm.cu(84): warning #177-D: member "::Tile::SMEM [with BITS=16, BM=128, WM=2, STAGES=3]" was declared but never referenced + static constexpr int SMEM = STAGES * STAGE; + ^ + + +native ++- install tensorfold-native + +- compile exe tensorfold-native safe native + +- run /usr/local/cuda/bin/nvcc (flashnext_fact.fatbin) w +./zig/kernels/cuda/flashnext_fact.cu(175): warning #128-D: loop is not reachable + for (int kt = 0; kt < GS / 16; ++kt) { + ^ + detected during instantiation of "void fact::fact_kernel(const __nv_bfloat16 *, const float *, const uint32_t *, const __nv_bfloat16 *, const __nv_bfloat16 *, void *, float *, int, int, int, int, int, int, int) [with BM=64, BN=128, WM=2, WN=4, STAGES=3, GPS=2, DBUF=false, MINB=2, FOLD=true, OUT16=true, BLOCKED=true]" at line 262 + +Remark: The warnings can be suppressed with "-diag-suppress " diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/build/test.log b/docs/receipts/glm-kv-pages-8e705835/tp2/build/test.log new file mode 100644 index 0000000000..5ba0d8c1f4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/build/test.log @@ -0,0 +1,723 @@ +test ++- run test w +[default] (warn): a round could not be recorded: OutOfDeviceMemory +failed command: ./.zig-cache/o/a54a0865e59274669004a7f13c694324/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- run test w +[default] (warn): XPU: a command list did not drain at close; its memory and context stay until the process ends +failed command: ./.zig-cache/o/edf396673e708e86d364a030b3d10882/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- run test w +[default] (warn): glm: restoring 1500 tokens failed (PoolMoveNotDrained); pool relocation aborted +failed command: ./.zig-cache/o/ac7a1d93ef13200ea44a510c4feba393/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- run test w +[default] (warn): flash next: rotary cache table unavailable: OutOfMemory +failed command: ./.zig-cache/o/092a376cafb6e37fb30f5cb9a2b1ab18/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +discovery verbs: retained through TLS, atexit and destructor; child exit 0 +discovery grammar: retained through TLS, atexit and destructor; child exit 0 +test ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +failed command: ./.zig-cache/o/aa7f512ae499b3ab99989273a7914c6a/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zig-cache/o/0332a591e5dff57aacddfc6db8f86b02/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +failed command: ./.zig-cache/o/af5184cd4832ac36660fa25b70afbb1b/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +failed command: ./.zig-cache/o/bbb0959f6b656a233c811ff375f41ecf/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zig-cache/o/d242521578f45aef572937875f2b8cfc/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +GLM_KV_BYTES context=8192 capacity=8256 main_layers=11 bf16=296101376 fp8=245772800 +GLM_KV_BYTES context=65536 capacity=65600 main_layers=11 bf16=1044784640 fp8=644887040 +GLM_KV_BYTES context=199307 capacity=199360 main_layers=11 bf16=2791155200 fp8=1575856640 +GLM_KV_BYTES context=245760 capacity=245824 main_layers=11 bf16=3397789184 fp8=1899246080 +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zig-cache/o/95e65b516076e9d2332d3f70ac56d448/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zig-cache/o/0affd09f202bd260849de9b155a7099f/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +FAIL kernel-contract: unknown image name: unknown_image +FAIL kernel-contract: unknown image name: glm_glu +FAIL kernel-contract: unknown image name: +FAIL kernel-contract: unknown image name: contract_control +FAIL kernel-contract: unknown image name: GLM_GLUE +failed command: ./.zig-cache/o/ff8534b4c1ac5ae35036760b98b83976/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zig-cache/o/8a4d02bb9d3c4e71d1e5e01e68a5f951/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): libcublasLt.so.13 is required for CUDA vision towers (--vision) and CUDA cuBLASLt projections; install the NVIDIA CUDA 13 cuBLAS runtime; on Linux, add its directory to LD_LIBRARY_PATH; a driver-only host does not include it +[default] (warn): libnccl.so.2 is required for CUDA ranks (including the GLM pair); install the NVIDIA NCCL 2 runtime compatible with CUDA 13; on Linux, add its directory to LD_LIBRARY_PATH +[default] (warn): libibverbs.so.1 is required for RoCE/RDMA gathers; install rdma-core's libibverbs runtime and the provider for the network adapter +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +[default] (warn): libmcdma-rpc.so is required for MCDMA mailbox and Metal speed-up communication; build and install MCDMA's RPC helper and set MCDMA_RPC_LIBRARY to its file +failed command: ./.zig-cache/o/8c9e9beb5d041b4487c34483255a79b3/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +DEPTH_REPLAY n=2 intercept=50 slope=3 head=1 probe=false deepest=6 crossed={ 62, 62 } last={ 6, 6 } trimmed=0 +DEPTH_REPLAY n=2 intercept=50 slope=3 head=1 probe=true deepest=8 crossed={ 59, 57 } last={ 7, 8 } trimmed=11 +DEPTH_REPLAY n=4 intercept=50 slope=3 head=1 probe=false deepest=4 crossed={ 83, 83, 83, 83 } last={ 4, 4, 4, 4 } trimmed=0 +DEPTH_REPLAY n=4 intercept=50 slope=3 head=1 probe=true deepest=5 crossed={ 75, 83, 83, 76 } last={ 5, 5, 5, 5 } trimmed=9 +DEPTH_REPLAY n=2 intercept=55 slope=2.8 head=1 probe=false deepest=6 crossed={ 61, 61 } last={ 6, 6 } trimmed=0 +DEPTH_REPLAY n=2 intercept=55 slope=2.8 head=1 probe=true deepest=8 crossed={ 54, 56 } last={ 8, 8 } trimmed=2 +DEPTH_REPLAY n=4 intercept=55 slope=2.8 head=1 probe=false deepest=4 crossed={ 82, 82, 82, 82 } last={ 4, 4, 4, 4 } trimmed=0 +DEPTH_REPLAY n=4 intercept=55 slope=2.8 head=1 probe=true deepest=6 crossed={ 75, 71, 72, 74 } last={ 5, 6, 5, 5 } trimmed=9 +DEPTH_REPLAY n=8 intercept=100 slope=0 head=0 probe=false deepest=3 crossed={ 100, 100, 100, 100, 100, 100, 100, 100 } last={ 3, 3, 3, 3, 3, 3, 3, 3 } trimmed=0 +DEPTH_REPLAY n=8 intercept=100 slope=0 head=0 probe=true deepest=3 crossed={ 100, 100, 100, 100, 100, 100, 100, 100 } last={ 3, 3, 3, 3, 3, 3, 3, 3 } trimmed=0 +failed command: ./.zig-cache/o/0a17fa5c7393aa452f1f424982eda498/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +VISION_LFM2 markers=104; every omitted entry refused +VISION_LFM2_AUTO valid stays enabled; missing and non-added disable pictures with warning +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zig-cache/o/73032413fa2cbd32f6d0043fbdc810a6/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- test-cuda-host + +- run test w +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libwebp 1.6.0 not found: WebP pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +failed command: ./.zig-cache/o/dcbf5137ce9f9d7cba1a4343a320edac/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +test ++- run test w +[default] (warn): lane host: stopping at a round boundary: 2 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): 4 prompts together: SamplingRefused; each goes alone +[default] (warn): lane host: a round failed (the planner refused): every admitted stream fails +[default] (warn): lane host: a round failed (PlannerFull): every admitted stream fails +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (drain deadline) +[default] (warn): lane host: stopping at a round boundary: 0 running and 1 queued request(s) end (NoContext) +[default] (warn): lane host: a round failed (OutOfMemory): every admitted stream fails +[default] (warn): lane host: a round failed (AdmittedFailed): every admitted stream fails +[default] (warn): lane host: a round failed (FinalsFailed): every admitted stream fails +failed command: ./.zig-cache/o/33d3a117abb9ad8e9ed4ec4bcd11c3a0/test --cache-dir=./.zig-cache --seed=0x2fc29d03 --listen=- + +Build Summary: 276/276 steps succeeded; 2023/2037 tests passed (14 skipped) +test success ++- test-kernel-symbols-host success +| +- run test 3 pass (3 total) 4s MaxRSS:82M +| +- compile test debug native success 2s MaxRSS:339M ++- test-checkpoint-security success +| +- run test 32 pass (32 total) 77ms MaxRSS:17M +| | +- compile test safe native success 13s MaxRSS:730M +| +- run test 5 pass (5 total) 3ms MaxRSS:14M +| | +- compile test safe native success 10s MaxRSS:624M +| +- run test 6 pass (6 total) 2ms MaxRSS:14M +| +- compile test safe native success 9s MaxRSS:542M ++- run test 37 pass (37 total) 78ms MaxRSS:24M +| +- compile test debug native success 3s MaxRSS:393M ++- run test 4 pass (4 total) 42ms MaxRSS:25M +| +- compile test fast aarch64-linux-gnu success 10s MaxRSS:551M ++- run test 24 pass (24 total) 784ms MaxRSS:75M +| +- compile test debug native success 2s MaxRSS:402M ++- run test 13 pass (13 total) 458ms MaxRSS:10M +| +- compile test debug native success 2s MaxRSS:372M +| +- options success +| +- compile exe digest-writer debug native success 1s MaxRSS:323M ++- run test 6 pass (6 total) 8ms MaxRSS:24M +| +- compile test debug native success 2s MaxRSS:343M ++- test-flashnext-scheduling-host success +| +- run test 30 pass, 1 skip (31 total) 13s MaxRSS:147M +| +- compile test debug native success 4s MaxRSS:524M +| +- WriteFile flashnext-test-empty.zig success ++- run test 15 pass (15 total) 32ms MaxRSS:12M +| +- compile test debug native success 3s MaxRSS:348M +| +- WriteFile flashnext-test-empty.zig (reused) ++- test-flashnext-batch-host success +| +- run test 14 pass (14 total) 1s MaxRSS:33M +| +- compile test debug native success 3s MaxRSS:487M +| +- WriteFile flashnext-test-empty.zig (reused) ++- run test 2 pass (2 total) 31ms MaxRSS:8M +| +- compile test debug native success 1s MaxRSS:328M ++- run test 4 pass (4 total) 11ms MaxRSS:8M +| +- compile test debug native success 1s MaxRSS:321M ++- run test 4 pass (4 total) 13ms MaxRSS:9M +| +- compile test debug native success 2s MaxRSS:349M ++- run test 3 pass (3 total) 11ms MaxRSS:8M +| +- compile test debug native success 1s MaxRSS:327M ++- run test 23 pass (23 total) 13ms MaxRSS:8M +| +- compile test debug native success 2s MaxRSS:352M ++- run test 29 pass (29 total) 2s MaxRSS:23M +| +- compile test debug native success 2s MaxRSS:423M ++- run test 2 pass (2 total) 4ms MaxRSS:12M +| +- compile test fast native success 9s MaxRSS:516M ++- run test 59 pass, 1 skip (60 total) 29s MaxRSS:49M +| +- compile test debug native success 3s MaxRSS:424M +| | +- options cached +| +- compile obj tp_host_compile debug aarch64-linux-gnu success 227ms MaxRSS:108M +| +- options cached ++- run exe tf-tp-test success 2ms +| +- compile exe tf-tp-test fast native success 19s MaxRSS:855M +| +- WriteFile tp_kernel.zig cached +| +- options cached ++- run exe tf-tp-test success 5ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 7ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 7ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 6ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 5ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 3ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 3ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 3ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 5ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 4ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 8ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 5ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- run exe tf-tp-test success 6ms +| +- compile exe tf-tp-test fast native (+2 more reused dependencies) ++- test-cuda-host success +| +- run test cached +| | +- compile test debug native cached 26ms MaxRSS:39M +| +- run test 1 pass (1 total) 17ms MaxRSS:12M +| | +- compile test debug native success 3s MaxRSS:328M +| +- run test 10 pass (10 total) 28ms MaxRSS:18M +| | +- compile test debug native success 3s MaxRSS:370M +| +- run test cached +| | +- compile test debug native cached 8ms MaxRSS:39M +| +- run exe runtime-discovery-exit success 17ms +| | +- compile exe runtime-discovery-exit debug native success 3s MaxRSS:330M +| | +- options cached +| | +- compile lib runtime-discovery-mock debug native cached 20ms MaxRSS:38M +| +- run exe rpc-warning-check success 4ms +| | +- compile exe rpc-warning-check debug native success 2s MaxRSS:316M +| | +- options success +| | +- compile lib rpc-mock-complete debug native cached 29ms MaxRSS:37M +| | +- compile lib rpc-mock-abi debug native success 46ms MaxRSS:62M +| | +- compile lib rpc-mock-wait debug native cached 21ms MaxRSS:38M +| | +- compile lib rpc-mock-store debug native cached 21ms MaxRSS:38M +| | +- compile lib rpc-mock-wrong debug native cached 15ms MaxRSS:38M +| +- run test 11 pass (11 total) 15ms MaxRSS:24M +| | +- compile test debug native success 3s MaxRSS:357M +| | +- options (+5 more reused dependencies) +| +- run test 1 pass (1 total) 12ms MaxRSS:17M +| | +- compile test debug native success 2s MaxRSS:326M +| +- run test 30 pass (30 total) 68ms MaxRSS:16M +| | +- compile test debug native success 4s MaxRSS:469M +| | +- WriteFile h3-compile-only.fatbin cached +| | +- options cached +| +- run test cached +| | +- compile test debug native cached 25ms MaxRSS:38M +| +- run test 3 pass (3 total) 15ms MaxRSS:23M +| | +- compile test debug native success 5s MaxRSS:357M +| | +- WriteFile attn_train.fatbin cached +| | +- WriteFile dsv41_exl3_experts.fatbin cached +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- WriteFile dsv41_exl3_experts.fatbin (reused) +| | +- options cached +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- options cached +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| | +- WriteFile attn_train.fatbin (reused) +| +- run test 3 pass (3 total) 108ms MaxRSS:15M +| | +- compile test debug native success 3s MaxRSS:409M +| | +- options success +| | | +- compile exe qwen-round-timing-probe debug native success 4s MaxRSS:439M +| | | +- WriteFile draft_ids.txt cached +| | | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 10 pass (10 total) 12ms MaxRSS:17M +| | +- compile test debug native success 2s MaxRSS:355M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 25ms MaxRSS:39M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test 6 pass (6 total) 45ms MaxRSS:17M +| | +- compile test debug native success 3s MaxRSS:381M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test 10 pass (10 total) 365ms MaxRSS:22M +| | +- compile test debug native success 5s MaxRSS:533M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 9 pass (9 total) 59ms MaxRSS:21M +| | +- compile test debug native success 3s MaxRSS:372M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- compile obj exl3_host_compile debug native success 425ms MaxRSS:118M +| | +- options (reused) +| +- run test 6 pass (6 total) 17ms MaxRSS:21M +| | +- compile test debug native success 3s MaxRSS:393M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 25ms MaxRSS:39M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 36ms MaxRSS:39M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 24ms MaxRSS:39M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 17ms MaxRSS:39M +| | +- options (reused) +| +- run test 3 pass (3 total) 31ms MaxRSS:15M +| | +- compile test debug native success 2s MaxRSS:345M +| | +- options (reused) +| +- run test 8 pass (8 total) 17ms MaxRSS:22M +| | +- compile test debug native success 2s MaxRSS:348M +| +- run test 3 pass (3 total) 99ms MaxRSS:24M +| | +- compile test debug native success 2s MaxRSS:336M +| +- run test 8 pass (8 total) 25ms MaxRSS:13M +| | +- compile test debug native success 3s MaxRSS:346M +| +- run test cached +| | +- compile test debug native cached 28ms MaxRSS:39M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 22ms MaxRSS:39M +| | +- options (reused) +| +- run test cached +| | +- compile test fast native cached 22ms MaxRSS:39M +| | +- options (reused) +| +- run test 1 pass (1 total) 13ms MaxRSS:22M +| | +- compile test fast native success 10s MaxRSS:557M +| | +- options (reused) +| +- compile obj bolt_host_compile debug aarch64-linux-gnu cached 8ms MaxRSS:39M +| | +- options cached +| | +- WriteFile draft_ids.txt (reused) +| +- run test 26 pass (26 total) 4s MaxRSS:21M +| | +- compile test debug native cached 21ms MaxRSS:39M +| | +- options (reused) +| +- run test 4 pass (4 total) 21ms MaxRSS:20M +| | +- compile test debug native success 3s MaxRSS:355M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 11ms MaxRSS:39M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run exe qwen-cost-key-process success 1s +| | +- compile exe qwen-cost-key-process debug native cached 23ms MaxRSS:38M +| | | +- WriteFile draft_ids.txt (reused) +| | | +- options (reused) +| | +- WriteFile target/config.json cached +| +- run test cached +| | +- compile test debug native cached 18ms MaxRSS:38M +| +- compile obj drama_cuda_native_host debug native cached 17ms MaxRSS:38M +| | +- options (reused) +| +- run test 5 pass (5 total) 17ms MaxRSS:18M +| | +- compile test debug native success 3s MaxRSS:335M +| +- run test cached +| | +- compile test debug native cached 26ms MaxRSS:39M +| +- run test 4 pass (4 total) 15ms MaxRSS:23M +| | +- compile test debug native success 3s MaxRSS:383M +| +- run test cached +| | +- compile test debug native cached 27ms MaxRSS:39M +| +- run test cached +| | +- compile test debug native cached 19ms MaxRSS:39M +| +- run test cached +| | +- compile test debug native cached 19ms MaxRSS:39M +| | +- options (reused) +| +- run test 352 pass (352 total) 6s MaxRSS:133M +| | +- compile test debug native success 10s MaxRSS:832M +| +- run test 212 pass (212 total) 6s MaxRSS:46M +| | +- compile test debug native success 5s MaxRSS:610M +| +- run test cached +| | +- compile test debug native cached 34ms MaxRSS:39M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 58 pass, 2 skip (60 total) 8s MaxRSS:25M +| | +- compile test debug native success 3s MaxRSS:458M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 25ms MaxRSS:39M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 26ms MaxRSS:39M +| | +- options (reused) +| +- run test 15 pass (15 total) 4s MaxRSS:23M +| | +- compile test debug native success 3s MaxRSS:495M +| | +- options (reused) +| +- run test 29 pass (29 total) 4s MaxRSS:1G +| | +- compile test debug native success 4s MaxRSS:422M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 19ms MaxRSS:39M +| +- run test 36 pass (36 total) 135ms MaxRSS:24M +| | +- compile test debug native success 3s MaxRSS:425M +| +- run test cached +| | +- compile test debug native cached 26ms MaxRSS:39M +| +- run test 73 pass (73 total) 128ms MaxRSS:30M +| | +- compile test debug native success 10s MaxRSS:1G +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test cached +| | +- compile test debug native cached 23ms MaxRSS:39M +| +- run test cached +| | +- compile test debug native cached 29ms MaxRSS:39M +| +- run test 12 pass (12 total) 32ms MaxRSS:20M +| | +- compile test debug native success 3s MaxRSS:371M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 20ms MaxRSS:39M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 25ms MaxRSS:39M +| | +- options (reused) +| +- run test 88 pass (88 total) 88ms MaxRSS:22M +| | +- compile test debug native success 5s MaxRSS:612M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 22ms MaxRSS:39M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 27ms MaxRSS:39M +| +- run test cached +| | +- compile test debug native cached 33ms MaxRSS:39M +| +- run test cached +| | +- compile test debug native cached 24ms MaxRSS:39M +| +- run test 49 pass, 2 skip (51 total) 1s MaxRSS:15M +| | +- compile test debug native success 4s MaxRSS:495M +| +- run test cached +| +- compile test debug native cached 42ms MaxRSS:39M +| +- options cached +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile exlr8_alias.zig cached ++- run test 4 pass (4 total) 15ms MaxRSS:10M +| +- compile test debug native success 2s MaxRSS:328M +| +- options cached ++- run test 13 pass, 1 skip (14 total) 46ms MaxRSS:14M +| +- compile test debug native success 3s MaxRSS:384M +| +- options (reused) ++- run test 3 pass (3 total) 15ms MaxRSS:12M +| +- compile test debug native success 3s MaxRSS:344M +| +- options (reused) ++- run test 1 pass (1 total) 11ms MaxRSS:10M +| +- compile test debug native success 2s MaxRSS:329M ++- run test 238 pass (238 total) 5s MaxRSS:25M +| +- compile test fast native success 34s MaxRSS:1G ++- test-kv-pool success +| +- run test cached +| | +- compile test debug native cached 25ms MaxRSS:39M +| +- run test cached +| +- compile test debug native cached 21ms MaxRSS:39M ++- run test 51 pass (51 total) 73ms MaxRSS:9M +| +- compile test debug native success 2s MaxRSS:399M +| +- options success +| | +- compile lib hip-mock-success debug native success 66ms MaxRSS:62M +| | +- compile lib hip-mock-failed debug native success 65ms MaxRSS:62M +| | +- compile lib hip-mock-missing debug native success 64ms MaxRSS:62M +| +- WriteFile kernels.zig cached ++- run test 23 pass, 1 skip (24 total) 101ms MaxRSS:14M +| +- compile test debug native success 4s MaxRSS:436M ++- run test 49 pass (49 total) 65ms MaxRSS:11M +| +- compile test debug native success 3s MaxRSS:442M +| +- WriteFile kernels.zig success ++- run test 35 pass, 1 skip (36 total) 119ms MaxRSS:14M +| +- compile test debug native success 6s MaxRSS:598M +| +- WriteFile kernels.zig (reused) ++- run test 19 pass (19 total) 973ms MaxRSS:20M +| +- compile test debug native success 7s MaxRSS:607M +| +- WriteFile kernels.zig (reused) ++- run test 4 pass (4 total) 22ms MaxRSS:10M +| +- compile test debug native success 2s MaxRSS:358M +| +- WriteFile kernels.zig (reused) ++- run test 4 pass (4 total) 14ms MaxRSS:17M +| +- compile test debug native success 2s MaxRSS:358M +| +- WriteFile kernels.zig (reused) ++- compile exe tf-qwen35-test debug native success 3s MaxRSS:170M +| +- WriteFile kernels.zig (reused) ++- compile exe tensorfold-native safe native success 2s MaxRSS:254M +| +- options (reused) +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile kernels.zig (reused) ++- run test 31 pass, 5 skip (36 total) 24ms MaxRSS:17M +| +- compile test safe native success 16s MaxRSS:794M ++- run test 4 pass (4 total) 14ms MaxRSS:24M +| +- compile test debug native success 2s MaxRSS:329M ++- run test 121 pass (121 total) 7s MaxRSS:41M +| +- compile test debug native success 4s MaxRSS:482M ++- run test 22 pass (22 total) 220ms MaxRSS:13M +| +- compile test debug native success 4s MaxRSS:406M ++- run exe learned-warning-test success 27ms + +- compile exe learned-warning-test debug native success 2s MaxRSS:360M diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/check/glm-paged-check-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/check/glm-paged-check-rank0.log new file mode 100644 index 0000000000..437bba8420 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/check/glm-paged-check-rank0.log @@ -0,0 +1,9 @@ +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +PASS pages at TP=2: on the engine's own sequence, 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/check/glm-paged-check-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/check/glm-paged-check-rank1.log new file mode 100644 index 0000000000..2a1f2a7088 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/check/glm-paged-check-rank1.log @@ -0,0 +1,2 @@ +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +PASS rank 1 made rank 0's pool and mapped its pages through every step diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/compare/auto-against-all.log b/docs/receipts/glm-kv-pages-8e705835/tp2/compare/auto-against-all.log new file mode 100644 index 0000000000..e460a23f51 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/compare/auto-against-all.log @@ -0,0 +1,83 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.25 s vs fresh 5.94 s ( 4.8x) + S1 t2 drafted prompt 15550 cached 6144: ttft 7.23 s vs fresh 11.87 s ( 1.6x) + S1 t2 plain prompt 15550 cached 14336: ttft 1.02 s vs fresh 11.86 s ( 11.6x) + S1 t3 drafted prompt 19460 cached 14336: ttft 4.06 s vs fresh 15.17 s ( 3.7x) + S1 t3 plain prompt 19460 cached 18432: ttft 0.93 s vs fresh 15.10 s ( 16.3x) + S1 t4 drafted prompt 21874 cached 18432: ttft 2.73 s vs fresh 16.85 s ( 6.2x) + S1 t4 plain prompt 21874 cached 20480: ttft 1.12 s vs fresh 16.85 s ( 15.0x) + S1 t5 drafted prompt 26786 cached 20480: ttft 5.10 s vs fresh 20.81 s ( 4.1x) + S1 t5 plain prompt 26786 cached 26624: ttft 0.43 s vs fresh 20.53 s ( 48.1x) + S1 t6 drafted prompt 29293 cached 26624: ttft 2.31 s vs fresh 22.38 s ( 9.7x) + S1 t6 plain prompt 29293 cached 28672: ttft 0.71 s vs fresh 22.37 s ( 31.3x) + S1 t7 drafted prompt 32216 cached 28672: ttft 2.83 s vs fresh 24.45 s ( 8.7x) + S1 t7 plain prompt 32216 cached 30720: ttft 1.20 s vs fresh 24.38 s ( 20.3x) + S1 t8 drafted prompt 35142 cached 30720: ttft 3.72 s vs fresh 26.89 s ( 7.2x) + S1 t8 plain prompt 35142 cached 34816: ttft 0.56 s vs fresh 26.86 s ( 48.1x) + S1 t9 drafted prompt 37860 cached 34816: ttft 2.56 s vs fresh 28.83 s ( 11.2x) + S1 t9 plain prompt 37860 cached 36864: ttft 0.93 s vs fresh 28.81 s ( 30.9x) + A t1 drafted prompt 7412 cached 6144: ttft 1.10 s vs fresh 5.79 s ( 5.3x) + A t1 plain prompt 7412 cached 6144: ttft 1.05 s vs fresh 5.76 s ( 5.5x) + B t1 drafted prompt 7474 cached 6144: ttft 1.13 s vs fresh 5.84 s ( 5.2x) + B t1 plain prompt 7474 cached 6144: ttft 1.08 s vs fresh 5.79 s ( 5.3x) + A t2 drafted prompt 12623 cached 6144: ttft 5.18 s vs fresh 9.86 s ( 1.9x) + A t2 plain prompt 12623 cached 12288: ttft 0.55 s vs fresh 9.86 s ( 17.9x) + B t2 drafted prompt 12720 cached 6144: ttft 5.23 s vs fresh 9.93 s ( 1.9x) + B t2 plain prompt 12720 cached 12288: ttft 0.60 s vs fresh 9.90 s ( 16.6x) + A t3 drafted prompt 18063 cached 12288: ttft 4.45 s vs fresh 13.73 s ( 3.1x) + A t3 plain prompt 18063 cached 16384: ttft 1.29 s vs fresh 13.69 s ( 10.6x) + B t3 drafted prompt 15032 cached 12288: ttft 2.32 s vs fresh 11.60 s ( 5.0x) + B t3 plain prompt 15032 cached 14336: ttft 0.74 s vs fresh 11.57 s ( 15.6x) + E t1 drafted prompt 40609 cached 6144: ttft 26.23 s vs fresh 30.81 s ( 1.2x) + E t1 plain prompt 40609 cached 38912: ttft 1.34 s vs fresh 30.77 s ( 22.9x) + T t2 plain prompt 4208 cached 4096: ttft 0.37 s vs fresh 3.56 s ( 9.6x) + C t1 plain prompt 9702 cached 8192: ttft 1.17 s vs fresh 7.40 s ( 6.3x) + C t2 drafted prompt 15152 cached 8192: ttft 5.48 s vs fresh 11.69 s ( 2.1x) + C t2 plain prompt 15152 cached 14336: ttft 0.82 s vs fresh 11.66 s ( 14.2x) + A t4 drafted prompt 20872 cached 16384: ttft 6.56 s vs fresh 30.54 s ( 4.7x) + B t4 drafted prompt 17550 cached 14336: ttft 2.61 s vs fresh 13.45 s ( 5.1x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/compare/auto-against-on.log b/docs/receipts/glm-kv-pages-8e705835/tp2/compare/auto-against-on.log new file mode 100644 index 0000000000..2dcfc22654 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/compare/auto-against-on.log @@ -0,0 +1,83 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.25 s vs fresh 5.94 s ( 4.8x) + S1 t2 drafted prompt 15550 cached 6144: ttft 7.20 s vs fresh 11.87 s ( 1.6x) + S1 t2 plain prompt 15550 cached 14336: ttft 1.03 s vs fresh 11.86 s ( 11.6x) + S1 t3 drafted prompt 19460 cached 14336: ttft 4.06 s vs fresh 15.17 s ( 3.7x) + S1 t3 plain prompt 19460 cached 18432: ttft 0.93 s vs fresh 15.10 s ( 16.3x) + S1 t4 drafted prompt 21874 cached 18432: ttft 2.73 s vs fresh 16.85 s ( 6.2x) + S1 t4 plain prompt 21874 cached 20480: ttft 1.12 s vs fresh 16.85 s ( 15.1x) + S1 t5 drafted prompt 26786 cached 20480: ttft 5.10 s vs fresh 20.81 s ( 4.1x) + S1 t5 plain prompt 26786 cached 26624: ttft 0.43 s vs fresh 20.53 s ( 48.0x) + S1 t6 drafted prompt 29293 cached 26624: ttft 2.31 s vs fresh 22.38 s ( 9.7x) + S1 t6 plain prompt 29293 cached 28672: ttft 0.71 s vs fresh 22.37 s ( 31.4x) + S1 t7 drafted prompt 32216 cached 28672: ttft 2.82 s vs fresh 24.45 s ( 8.7x) + S1 t7 plain prompt 32216 cached 30720: ttft 1.20 s vs fresh 24.38 s ( 20.3x) + S1 t8 drafted prompt 35142 cached 30720: ttft 3.71 s vs fresh 26.89 s ( 7.2x) + S1 t8 plain prompt 35142 cached 34816: ttft 0.56 s vs fresh 26.86 s ( 48.0x) + S1 t9 drafted prompt 37860 cached 34816: ttft 2.56 s vs fresh 28.83 s ( 11.3x) + S1 t9 plain prompt 37860 cached 36864: ttft 0.93 s vs fresh 28.81 s ( 30.9x) + A t1 drafted prompt 7412 cached 6144: ttft 1.09 s vs fresh 5.79 s ( 5.3x) + A t1 plain prompt 7412 cached 6144: ttft 1.05 s vs fresh 5.76 s ( 5.5x) + B t1 drafted prompt 7474 cached 6144: ttft 1.13 s vs fresh 5.84 s ( 5.2x) + B t1 plain prompt 7474 cached 6144: ttft 1.09 s vs fresh 5.79 s ( 5.3x) + A t2 drafted prompt 12623 cached 6144: ttft 5.17 s vs fresh 9.86 s ( 1.9x) + A t2 plain prompt 12623 cached 12288: ttft 0.55 s vs fresh 9.86 s ( 18.0x) + B t2 drafted prompt 12720 cached 6144: ttft 5.23 s vs fresh 9.93 s ( 1.9x) + B t2 plain prompt 12720 cached 12288: ttft 0.60 s vs fresh 9.90 s ( 16.5x) + A t3 drafted prompt 18063 cached 12288: ttft 4.44 s vs fresh 13.73 s ( 3.1x) + A t3 plain prompt 18063 cached 16384: ttft 1.29 s vs fresh 13.69 s ( 10.6x) + B t3 drafted prompt 15032 cached 12288: ttft 2.32 s vs fresh 11.60 s ( 5.0x) + B t3 plain prompt 15032 cached 14336: ttft 0.74 s vs fresh 11.57 s ( 15.7x) + E t1 drafted prompt 40609 cached 6144: ttft 26.18 s vs fresh 30.81 s ( 1.2x) + E t1 plain prompt 40609 cached 38912: ttft 1.34 s vs fresh 30.77 s ( 22.9x) + T t2 plain prompt 4208 cached 4096: ttft 0.37 s vs fresh 3.56 s ( 9.6x) + C t1 plain prompt 9702 cached 8192: ttft 1.17 s vs fresh 7.40 s ( 6.3x) + C t2 drafted prompt 15152 cached 8192: ttft 5.47 s vs fresh 11.69 s ( 2.1x) + C t2 plain prompt 15152 cached 14336: ttft 0.83 s vs fresh 11.66 s ( 14.1x) + A t4 drafted prompt 20872 cached 16384: ttft 6.56 s vs fresh 30.54 s ( 4.7x) + B t4 drafted prompt 17550 cached 14336: ttft 2.61 s vs fresh 13.45 s ( 5.1x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/compare/main-against-all.log b/docs/receipts/glm-kv-pages-8e705835/tp2/compare/main-against-all.log new file mode 100644 index 0000000000..53364c30c1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/compare/main-against-all.log @@ -0,0 +1,83 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.25 s vs fresh 5.93 s ( 4.8x) + S1 t2 drafted prompt 15550 cached 6144: ttft 7.23 s vs fresh 11.89 s ( 1.6x) + S1 t2 plain prompt 15550 cached 14336: ttft 1.02 s vs fresh 11.86 s ( 11.6x) + S1 t3 drafted prompt 19460 cached 14336: ttft 4.06 s vs fresh 14.88 s ( 3.7x) + S1 t3 plain prompt 19460 cached 18432: ttft 0.93 s vs fresh 14.85 s ( 16.0x) + S1 t4 drafted prompt 21874 cached 18432: ttft 2.73 s vs fresh 16.66 s ( 6.1x) + S1 t4 plain prompt 21874 cached 20480: ttft 1.12 s vs fresh 16.59 s ( 14.8x) + S1 t5 drafted prompt 26786 cached 20480: ttft 5.10 s vs fresh 20.59 s ( 4.0x) + S1 t5 plain prompt 26786 cached 26624: ttft 0.43 s vs fresh 20.54 s ( 48.1x) + S1 t6 drafted prompt 29293 cached 26624: ttft 2.31 s vs fresh 22.41 s ( 9.7x) + S1 t6 plain prompt 29293 cached 28672: ttft 0.71 s vs fresh 22.38 s ( 31.4x) + S1 t7 drafted prompt 32216 cached 28672: ttft 2.83 s vs fresh 24.48 s ( 8.7x) + S1 t7 plain prompt 32216 cached 30720: ttft 1.20 s vs fresh 24.42 s ( 20.3x) + S1 t8 drafted prompt 35142 cached 30720: ttft 3.72 s vs fresh 26.92 s ( 7.2x) + S1 t8 plain prompt 35142 cached 34816: ttft 0.56 s vs fresh 26.88 s ( 48.1x) + S1 t9 drafted prompt 37860 cached 34816: ttft 2.56 s vs fresh 28.86 s ( 11.3x) + S1 t9 plain prompt 37860 cached 36864: ttft 0.93 s vs fresh 28.82 s ( 30.9x) + A t1 drafted prompt 7412 cached 6144: ttft 1.10 s vs fresh 5.81 s ( 5.3x) + A t1 plain prompt 7412 cached 6144: ttft 1.05 s vs fresh 5.76 s ( 5.5x) + B t1 drafted prompt 7474 cached 6144: ttft 1.13 s vs fresh 5.84 s ( 5.2x) + B t1 plain prompt 7474 cached 6144: ttft 1.08 s vs fresh 5.79 s ( 5.3x) + A t2 drafted prompt 12623 cached 6144: ttft 5.18 s vs fresh 9.87 s ( 1.9x) + A t2 plain prompt 12623 cached 12288: ttft 0.55 s vs fresh 9.85 s ( 17.9x) + B t2 drafted prompt 12720 cached 6144: ttft 5.23 s vs fresh 9.93 s ( 1.9x) + B t2 plain prompt 12720 cached 12288: ttft 0.60 s vs fresh 9.90 s ( 16.6x) + A t3 drafted prompt 18063 cached 12288: ttft 4.45 s vs fresh 13.73 s ( 3.1x) + A t3 plain prompt 18063 cached 16384: ttft 1.29 s vs fresh 13.67 s ( 10.6x) + B t3 drafted prompt 15032 cached 12288: ttft 2.32 s vs fresh 11.62 s ( 5.0x) + B t3 plain prompt 15032 cached 14336: ttft 0.74 s vs fresh 11.59 s ( 15.6x) + E t1 drafted prompt 40609 cached 6144: ttft 26.23 s vs fresh 30.83 s ( 1.2x) + E t1 plain prompt 40609 cached 38912: ttft 1.34 s vs fresh 30.79 s ( 22.9x) + T t2 plain prompt 4208 cached 4096: ttft 0.37 s vs fresh 3.57 s ( 9.6x) + C t1 plain prompt 9702 cached 8192: ttft 1.17 s vs fresh 7.41 s ( 6.3x) + C t2 drafted prompt 15152 cached 8192: ttft 5.48 s vs fresh 11.70 s ( 2.1x) + C t2 plain prompt 15152 cached 14336: ttft 0.82 s vs fresh 11.66 s ( 14.2x) + A t4 drafted prompt 20872 cached 16384: ttft 6.56 s vs fresh 30.51 s ( 4.6x) + B t4 drafted prompt 17550 cached 14336: ttft 2.61 s vs fresh 13.43 s ( 5.1x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/compare/main-against-on.log b/docs/receipts/glm-kv-pages-8e705835/tp2/compare/main-against-on.log new file mode 100644 index 0000000000..594fc66c8e --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/compare/main-against-on.log @@ -0,0 +1,83 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.25 s vs fresh 5.93 s ( 4.8x) + S1 t2 drafted prompt 15550 cached 6144: ttft 7.20 s vs fresh 11.89 s ( 1.7x) + S1 t2 plain prompt 15550 cached 14336: ttft 1.03 s vs fresh 11.86 s ( 11.6x) + S1 t3 drafted prompt 19460 cached 14336: ttft 4.06 s vs fresh 14.88 s ( 3.7x) + S1 t3 plain prompt 19460 cached 18432: ttft 0.93 s vs fresh 14.85 s ( 16.0x) + S1 t4 drafted prompt 21874 cached 18432: ttft 2.73 s vs fresh 16.66 s ( 6.1x) + S1 t4 plain prompt 21874 cached 20480: ttft 1.12 s vs fresh 16.59 s ( 14.9x) + S1 t5 drafted prompt 26786 cached 20480: ttft 5.10 s vs fresh 20.59 s ( 4.0x) + S1 t5 plain prompt 26786 cached 26624: ttft 0.43 s vs fresh 20.54 s ( 48.0x) + S1 t6 drafted prompt 29293 cached 26624: ttft 2.31 s vs fresh 22.41 s ( 9.7x) + S1 t6 plain prompt 29293 cached 28672: ttft 0.71 s vs fresh 22.38 s ( 31.4x) + S1 t7 drafted prompt 32216 cached 28672: ttft 2.82 s vs fresh 24.48 s ( 8.7x) + S1 t7 plain prompt 32216 cached 30720: ttft 1.20 s vs fresh 24.42 s ( 20.3x) + S1 t8 drafted prompt 35142 cached 30720: ttft 3.71 s vs fresh 26.92 s ( 7.3x) + S1 t8 plain prompt 35142 cached 34816: ttft 0.56 s vs fresh 26.88 s ( 48.0x) + S1 t9 drafted prompt 37860 cached 34816: ttft 2.56 s vs fresh 28.86 s ( 11.3x) + S1 t9 plain prompt 37860 cached 36864: ttft 0.93 s vs fresh 28.82 s ( 30.9x) + A t1 drafted prompt 7412 cached 6144: ttft 1.09 s vs fresh 5.81 s ( 5.3x) + A t1 plain prompt 7412 cached 6144: ttft 1.05 s vs fresh 5.76 s ( 5.5x) + B t1 drafted prompt 7474 cached 6144: ttft 1.13 s vs fresh 5.84 s ( 5.2x) + B t1 plain prompt 7474 cached 6144: ttft 1.09 s vs fresh 5.79 s ( 5.3x) + A t2 drafted prompt 12623 cached 6144: ttft 5.17 s vs fresh 9.87 s ( 1.9x) + A t2 plain prompt 12623 cached 12288: ttft 0.55 s vs fresh 9.85 s ( 17.9x) + B t2 drafted prompt 12720 cached 6144: ttft 5.23 s vs fresh 9.93 s ( 1.9x) + B t2 plain prompt 12720 cached 12288: ttft 0.60 s vs fresh 9.90 s ( 16.5x) + A t3 drafted prompt 18063 cached 12288: ttft 4.44 s vs fresh 13.73 s ( 3.1x) + A t3 plain prompt 18063 cached 16384: ttft 1.29 s vs fresh 13.67 s ( 10.6x) + B t3 drafted prompt 15032 cached 12288: ttft 2.32 s vs fresh 11.62 s ( 5.0x) + B t3 plain prompt 15032 cached 14336: ttft 0.74 s vs fresh 11.59 s ( 15.7x) + E t1 drafted prompt 40609 cached 6144: ttft 26.18 s vs fresh 30.83 s ( 1.2x) + E t1 plain prompt 40609 cached 38912: ttft 1.34 s vs fresh 30.79 s ( 22.9x) + T t2 plain prompt 4208 cached 4096: ttft 0.37 s vs fresh 3.57 s ( 9.6x) + C t1 plain prompt 9702 cached 8192: ttft 1.17 s vs fresh 7.41 s ( 6.3x) + C t2 drafted prompt 15152 cached 8192: ttft 5.47 s vs fresh 11.70 s ( 2.1x) + C t2 plain prompt 15152 cached 14336: ttft 0.83 s vs fresh 11.66 s ( 14.1x) + A t4 drafted prompt 20872 cached 16384: ttft 6.56 s vs fresh 30.51 s ( 4.7x) + B t4 drafted prompt 17550 cached 14336: ttft 2.61 s vs fresh 13.43 s ( 5.1x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/client.log new file mode 100644 index 0000000000..3994a345cd --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/client.log @@ -0,0 +1,61 @@ +server up 2026-10-23 10:25:40 +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 39.1, "per_stream_tps": 39.1, "ttft_s_max": 0.15, "timing_valid": true, "end_to_end_tps": 38.39180039650422, "makespan_s": 6.66809051297605, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 45.3, "per_stream_tps": 25.0, "ttft_s_max": 0.36, "timing_valid": true, "end_to_end_tps": 44.86162687215045, "makespan_s": 11.412871884007473, "steady_tps": 48.6, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 61.1, "per_stream_tps": 17.1, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 60.81980109891801, "makespan_s": 16.8366219799791, "steady_tps": 65.0, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 29.1, "per_stream_tps": 29.1, "ttft_s_max": 0.22, "timing_valid": true, "end_to_end_tps": 28.542519010321044, "makespan_s": 8.96907522098627, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 40.4, "per_stream_tps": 20.6, "ttft_s_max": 0.51, "timing_valid": true, "end_to_end_tps": 39.85462641103368, "makespan_s": 12.846689233003417, "steady_tps": 41.2, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 51.0, "per_stream_tps": 13.6, "ttft_s_max": 1.18, "timing_valid": true, "end_to_end_tps": 50.61233434023709, "makespan_s": 20.232222310005454, "steady_tps": 54.0, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 56.1, "per_stream_tps": 56.1, "ttft_s_max": 0.15, "timing_valid": true, "end_to_end_tps": 54.56913804744495, "makespan_s": 4.691296384000452, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 64.0, "per_stream_tps": 32.6, "ttft_s_max": 0.36, "timing_valid": true, "end_to_end_tps": 63.125641826998525, "makespan_s": 8.110808622004697, "steady_tps": 65.0, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 87.0, "per_stream_tps": 22.6, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 86.29338237677625, "makespan_s": 11.866495110007236, "steady_tps": 91.6, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 30.5, "per_stream_tps": 30.5, "ttft_s_max": 0.22, "timing_valid": true, "end_to_end_tps": 29.885191052604196, "makespan_s": 8.566115557012381, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 43.6, "per_stream_tps": 22.2, "ttft_s_max": 0.5, "timing_valid": true, "end_to_end_tps": 42.98004314767995, "makespan_s": 11.91250549099641, "steady_tps": 44.8, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 58.9, "per_stream_tps": 15.2, "ttft_s_max": 1.17, "timing_valid": true, "end_to_end_tps": 58.430742226998426, "makespan_s": 17.52502126400941, "steady_tps": 61.7, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +logs/r2all-conc.json: 12 cells; alone 28/28 equal (0 unequal, 0 failed); serial 30/30 equal (0 unequal, 0 failed) + 0 S1 t1 drafted prompt 7759 history 7757 cached 0 (allowed [0]) ttft 6.05s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok + 1 S1 t1 plain prompt 7759 history 7757 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 1.25s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok +system cuts [4563, 6099, 6611] + 2 S1 t2 drafted prompt 15550 history 15548 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 7.23s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 tool_calls ok + 3 S1 t2 plain prompt 15550 history 15548 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 1.02s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 tool_calls ok + 4 S1 t3 drafted prompt 19460 history 19458 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 4.06s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 tool_calls ok + 5 S1 t3 plain prompt 19460 history 19458 cached 18432 (allowed {'min': 11266, 'max': 19458}) ttft 0.93s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 tool_calls ok + 6 S1 t4 drafted prompt 21874 history 21872 cached 18432 (allowed {'min': 11266, 'max': 19458}) ttft 2.73s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 tool_calls ok + 7 S1 t4 plain prompt 21874 history 21872 cached 20480 (allowed {'min': 13680, 'max': 21872}) ttft 1.12s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 tool_calls ok + 8 S1 t5 drafted prompt 26786 history 26784 cached 20480 (allowed {'min': 13680, 'max': 21872}) ttft 5.10s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c length ok + 9 S1 t5 plain prompt 26786 history 26784 cached 26624 (allowed {'min': 18592, 'max': 26784}) ttft 0.43s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c length ok + 10 S1 t6 drafted prompt 29293 history 29291 cached 26624 (allowed {'min': 18592, 'max': 26784}) ttft 2.31s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 length ok + 11 S1 t6 plain prompt 29293 history 29291 cached 28672 (allowed {'min': 21099, 'max': 29291}) ttft 0.71s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 length ok + 12 S1 t7 drafted prompt 32216 history 32214 cached 28672 (allowed {'min': 21099, 'max': 29291}) ttft 2.83s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e length ok + 13 S1 t7 plain prompt 32216 history 32214 cached 30720 (allowed {'min': 24022, 'max': 32214}) ttft 1.20s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e length ok + 14 S1 t8 drafted prompt 35142 history 35140 cached 30720 (allowed {'min': 24022, 'max': 32214}) ttft 3.72s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 length ok + 15 S1 t8 plain prompt 35142 history 35140 cached 34816 (allowed {'min': 26948, 'max': 35140}) ttft 0.56s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 length ok + 16 S1 t9 drafted prompt 37860 history 37858 cached 34816 (allowed {'min': 26948, 'max': 35140}) ttft 2.56s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 17 S1 t9 plain prompt 37860 history 37858 cached 36864 (allowed {'min': 29666, 'max': 37858}) ttft 0.93s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 18 X t1 drafted prompt 2115 history 2113 cached 0 (allowed [0]) ttft 1.77s sha 9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd length ok + 19 A t1 drafted prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 1.10s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 tool_calls ok + 20 A t1 plain prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 1.05s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 tool_calls ok + 21 B t1 drafted prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 1.13s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a tool_calls ok + 22 B t1 plain prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 1.08s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a tool_calls ok + 23 A t2 drafted prompt 12623 history 12621 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 5.18s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 tool_calls ok + 24 A t2 plain prompt 12623 history 12621 cached 12288 (allowed {'min': 4429, 'max': 12621}) ttft 0.55s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 tool_calls ok + 25 B t2 drafted prompt 12720 history 12718 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 5.23s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c tool_calls ok + 26 B t2 plain prompt 12720 history 12718 cached 12288 (allowed {'min': 4526, 'max': 12718}) ttft 0.60s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c tool_calls ok + 27 A t3 drafted prompt 18063 history 18061 cached 12288 (allowed {'min': 4429, 'max': 12621}) ttft 4.45s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc length ok + 28 A t3 plain prompt 18063 history 18061 cached 16384 (allowed {'min': 9869, 'max': 18061}) ttft 1.29s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc length ok + 29 B t3 drafted prompt 15032 history 15030 cached 12288 (allowed {'min': 4526, 'max': 12718}) ttft 2.32s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa tool_calls ok + 30 B t3 plain prompt 15032 history 15030 cached 14336 (allowed {'min': 6838, 'max': 15030}) ttft 0.74s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa tool_calls ok + 31 E t1 drafted prompt 40609 history 40607 cached 6144 (allowed {'min': 0, 'max': 6611}) ttft 26.23s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 32 E t1 plain prompt 40609 history 40607 cached 38912 (allowed {'min': 32415, 'max': 40607}) ttft 1.34s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 33 T t1 drafted prompt 1520 history 1517 cached 0 (allowed [0]) ttft 1.14s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 stop ok + 34 T t1 plain prompt 1520 history 1517 cached 0 (allowed [0]) ttft 1.10s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 stop ok + 35 T t2 drafted prompt 4208 history 4205 cached 0 (allowed [0]) ttft 3.60s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 36 T t2 plain prompt 4208 history 4205 cached 4096 (allowed {'min': 1, 'max': 4205}) ttft 0.37s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 37 C t1 cancel prompt None history 0 cached None (allowed None) ttft 0.00s sha None None ok + 38 C t1 drafted prompt 9702 history 9700 cached 0 (allowed {'min': 0, 'max': 9688}) ttft 8.21s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 tool_calls ok + 39 C t1 plain prompt 9702 history 9700 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 1.17s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 tool_calls ok + 40 C t2 drafted prompt 15152 history 15150 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 5.48s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 length ok + 41 C t2 plain prompt 15152 history 15150 cached 14336 (allowed {'min': 6958, 'max': 15150}) ttft 0.82s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 length ok + 42 A t4 drafted prompt 20872 history 20870 cached 16384 (allowed {'min': 9869, 'max': 18061}) ttft 6.56s sha eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c length ok + 43 B t4 drafted prompt 17550 history 17548 cached 14336 (allowed {'min': 6838, 'max': 15030}) ttft 2.61s sha fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8 length ok +record: 44 requests, 0 not cached as planned +=== kvp-client r2all rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/conc.json b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/conc.json new file mode 100644 index 0000000000..63680704c0 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/conc.json @@ -0,0 +1,4588 @@ +{ + "label": "", + "tokens": 256, + "mixed": false, + "cells": [ + { + "prompt": "code", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.1, + "per_stream_tps": 39.1, + "ttft_s_max": 0.15, + "timing_valid": true, + "end_to_end_tps": 38.39180039650422, + "makespan_s": 6.66809051297605, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.10193397094204, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 6.66809051297605, + "end_to_end_tps": 38.39180039650422, + "timing_valid": true, + "per_stream_tps": [ + 39.1 + ], + "ttft_s": [ + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + } + ], + "client_intervals": [ + { + "start": 170447.95892942, + "first": 170448.105458566, + "last": 170454.626875133, + "end": 170454.627019933, + "elapsed_s": 6.66809051297605, + "chunk_offsets_s": [ + 0.14652914597536437, + 0.21967058299924247, + 0.2907408739847597, + 0.3665976789779961, + 0.4503898669790942, + 0.5362979689962231, + 0.6146877739811316, + 0.7003472840005998, + 0.7825101490016095, + 0.8649151579884347, + 0.9469813679752406, + 1.031959536980139, + 1.111529307992896, + 1.1939511809905525, + 1.2675389689975418, + 1.3426179369853344, + 1.4191307889996096, + 1.497874017979484, + 1.5838603599986527, + 1.6684588509961031, + 1.755347381986212, + 1.8373706649872474, + 1.9114398599776905, + 1.9826756569964346, + 2.054946618998656, + 2.1320949569926597, + 2.212996291986201, + 2.3013890119909775, + 2.3865551019844133, + 2.4712349849869497, + 2.553733722976176, + 2.6388597810000647, + 2.7224824040022213, + 2.8071582389820833, + 2.889570561994333, + 2.9740911349945236, + 3.0589353229734115, + 3.147320411982946, + 3.2314496740000322, + 3.3174432819942012, + 3.406657423998695, + 3.4966656279866584, + 3.5828196200018283, + 3.6677124479901977, + 3.751482462976128, + 3.8342764169792645, + 3.9185909270017873, + 4.0006381169951055, + 4.082198011979926, + 4.1643251049972605, + 4.246460453985492, + 4.3285698349936865, + 4.415415105991997, + 4.494665583973983, + 4.566571539995493, + 4.640670049993787, + 4.716927272995235, + 4.802973313984694, + 4.8922873139963485, + 4.9767688169959, + 5.059408884990262, + 5.159959204000188, + 5.26923679097672, + 5.356065166997723, + 5.439414480992127, + 5.522457029001089, + 5.60970383498352, + 5.699612153985072, + 5.786893151991535, + 5.874909172998741, + 5.954676370980451, + 6.036873321980238, + 6.1244709759776015, + 6.207638067979133, + 6.292870370001765, + 6.373963387974072, + 6.4600089349842165, + 6.535385602997849, + 6.606197227985831, + 6.667945712979417 + ], + "chunk_gaps_ms": { + "count": 79, + "p50": 83.16709200153127, + "p95": 89.90831900155172, + "max": 109.2775869765319 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 45.3, + "per_stream_tps": 25.0, + "ttft_s_max": 0.36, + "timing_valid": true, + "end_to_end_tps": 44.86162687215045, + "makespan_s": 11.412871884007473, + "steady_tps": 48.6, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 45.26752074793841, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 11.412871884007473, + "end_to_end_tps": 44.86162687215045, + "timing_valid": true, + "steady_tps": 48.56046464645542, + "per_stream_tps": [ + 27.3, + 22.6 + ], + "ttft_s": [ + 0.36, + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + } + ], + "client_intervals": [ + { + "start": 170454.628666344, + "first": 170454.990171554, + "last": 170464.31456905, + "end": 170464.314737081, + "elapsed_s": 9.686070736992406, + "chunk_offsets_s": [ + 0.36150520999217406, + 0.4619800270011183, + 0.5780720759939868, + 0.6970274449849967, + 0.8157342709891964, + 0.9308761149877682, + 1.0429777289973572, + 1.163922323990846, + 1.2796676399884745, + 1.3964333519979846, + 1.5171668929979205, + 1.636920052987989, + 1.7553551209857687, + 1.8736489259754308, + 1.9917045539768878, + 2.1111332029977348, + 2.2318404889956582, + 2.3519922889827285, + 2.4727782309928443, + 2.590768117981497, + 2.712318649981171, + 2.8330330079770647, + 2.952746633993229, + 3.0720606919785496, + 3.194849013001658, + 3.322219864989165, + 3.4447246989875566, + 3.5641377349966206, + 3.685002045996953, + 3.8071088979777414, + 3.930723904981278, + 4.052230118977604, + 4.168103100993903, + 4.285660223977175, + 4.405105980986264, + 4.527814975997899, + 4.644741204974707, + 4.765884075983195, + 4.89283249899745, + 5.017288224975346, + 5.136525838985108, + 5.258122582978103, + 5.3783156509744, + 5.5008951439813245, + 5.623943131999113, + 5.743517561990302, + 5.866398061974905, + 5.990991420985665, + 6.106849269999657, + 6.227397873997688, + 6.349988536996534, + 6.46375722499215, + 6.582274891989073, + 6.704313971975353, + 6.825293470988981, + 6.947151575994212, + 7.072724293975625, + 7.191108056984376, + 7.310517417994561, + 7.425682535977103, + 7.539016522991005, + 7.6570322569750715, + 7.780060774995945, + 7.903527932998259, + 8.0257052059751, + 8.148943563981447, + 8.274417932989309, + 8.393629615980899, + 8.52042612497462, + 8.637936500977958, + 8.759648960985942, + 8.881015597988153, + 9.001292749977438, + 9.119769715995062, + 9.237537084001815, + 9.361112979997415, + 9.47819807898486, + 9.581984240998281, + 9.68590270599816 + ], + "chunk_gaps_ms": { + "count": 78, + "p50": 120.17243399168365, + "p95": 125.572717981413, + "max": 127.3708519875072 + } + }, + { + "start": 170454.6285734, + "first": 170454.774782685, + "last": 170466.041138597, + "end": 170466.041445284, + "elapsed_s": 11.412871884007473, + "chunk_offsets_s": [ + 0.14620928501244634, + 0.21680249500786886, + 0.4619137870031409, + 0.5782792599929962, + 0.6973118130117655, + 0.8157060630037449, + 0.9311392189993057, + 1.0430303210159764, + 1.1642730589956045, + 1.2799243110057432, + 1.396696392010199, + 1.5174215490114875, + 1.6372373170161154, + 1.7556511689908803, + 1.8737799969967455, + 1.9918395619897638, + 2.111159652005881, + 2.2317656569939572, + 2.3521667850145604, + 2.4728355589904822, + 2.590830469998764, + 2.7123770659964066, + 2.8330421450082213, + 2.9528018340060953, + 3.0721871400019154, + 3.1947633489908185, + 3.3224048080155626, + 3.4450840420031454, + 3.5645419899956323, + 3.6852125099976547, + 3.8074999370146543, + 3.9310280329955276, + 4.052512277994538, + 4.1685830040078145, + 4.285934943007305, + 4.405393371998798, + 4.528253663011128, + 4.645002164004836, + 4.766267180006253, + 4.893155043013394, + 5.017547761002788, + 5.136793790006777, + 5.258579462009948, + 5.37818168301601, + 5.501285431004362, + 5.62431617101538, + 5.743750488996739, + 5.866682093997952, + 5.991181980993133, + 6.107111413992243, + 6.2276048499916214, + 6.35018269601278, + 6.463992072996916, + 6.582151403999887, + 6.704248691996327, + 6.825116608000826, + 6.9474854950094596, + 7.072986740997294, + 7.191323223989457, + 7.311066280992236, + 7.425921432004543, + 7.539300347008975, + 7.657271503994707, + 7.780303590989206, + 7.903716219996568, + 8.025981829006923, + 8.149194235011237, + 8.274609084008262, + 8.393809264001902, + 8.520752796001034, + 8.638119045004714, + 8.759801248990698, + 8.881281662004767, + 9.001653101993725, + 9.119978708011331, + 9.237368300993694, + 9.361354212014703, + 9.478498557989951, + 9.582169247994898, + 9.68613828899106, + 9.764470349007752, + 9.82961227101623, + 9.892303161002928, + 9.956535214005271, + 10.021666335989721, + 10.087037216988392, + 10.159876749006798, + 10.233911654009717, + 10.306252756010508, + 10.372044468997046, + 10.439307538006688, + 10.513660169992363, + 10.588707153016003, + 10.66282621701248, + 10.734452537988545, + 10.799450652993983, + 10.8663317229948, + 10.940791394998087, + 11.018689841002924, + 11.096756093989825, + 11.177027892990736, + 11.261886480002431, + 11.341416137991473, + 11.412565197009826 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 119.03255301876925, + "p95": 125.41484899702482, + "max": 245.11129199527204 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 61.1, + "per_stream_tps": 17.1, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 60.81980109891801, + "makespan_s": 16.8366219799791, + "steady_tps": 65.0, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 61.11732353322164, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 16.8366219799791, + "end_to_end_tps": 60.81980109891801, + "timing_valid": true, + "steady_tps": 64.9735972188965, + "per_stream_tps": [ + 17.8, + 15.7, + 16.5, + 17.8 + ], + "ttft_s": [ + 0.15, + 0.61, + 0.36, + 0.89 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6c97574b6d3c", + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692" + } + ], + "client_intervals": [ + { + "start": 170466.043629117, + "first": 170466.190154294, + "last": 170480.528844698, + "end": 170480.528942618, + "elapsed_s": 14.485313501005294, + "chunk_offsets_s": [ + 0.1465251769986935, + 0.21967358901747502, + 0.4634273810079321, + 0.7474677280115429, + 1.0597155170107726, + 1.2337451119965408, + 1.3943322180130053, + 1.565455677016871, + 1.7338568400009535, + 1.906280841998523, + 2.07827186799841, + 2.2483632689982187, + 2.422745889023645, + 2.6068254570127465, + 2.7910885930177756, + 2.9741497980139684, + 3.152267274010228, + 3.330734476010548, + 3.5140376650088, + 3.697756292007398, + 3.8822863580135163, + 4.062255062017357, + 4.243755041999975, + 4.424698928021826, + 4.602919861004921, + 4.78719357700902, + 4.971896269998979, + 5.1591556270141155, + 5.347762245015474, + 5.5303269760042895, + 5.715129764023004, + 5.908092819008743, + 6.089199620008003, + 6.269847895018756, + 6.457268469996052, + 6.639228373009246, + 6.820240439003101, + 7.002514166000765, + 7.188318700005766, + 7.372975637001218, + 7.558850555011304, + 7.746520012005931, + 7.93657087199972, + 8.11723703800817, + 8.298320194997359, + 8.481083443999523, + 8.662395353021566, + 8.84854028900736, + 9.029870855010813, + 9.212395099020796, + 9.4013459639973, + 9.578035743004875, + 9.760933473997284, + 9.942737305012997, + 10.120746363012586, + 10.306029081024462, + 10.492682563024573, + 10.675603655021405, + 10.85138248000294, + 11.029868865996832, + 11.209420961997239, + 11.396184045996051, + 11.582268315018155, + 11.76575220801169, + 11.945248802017886, + 12.126949310011696, + 12.322797651024302, + 12.514405283000087, + 12.704337592003867, + 12.8905830970034, + 13.066600246005692, + 13.254368596011773, + 13.428694294008892, + 13.607253627997125, + 13.784584214008646, + 13.967782193998573, + 14.145051374012837, + 14.316727721015923, + 14.485215581022203 + ], + "chunk_gaps_ms": { + "count": 78, + "p50": 182.3989855038235, + "p95": 195.8483410126064, + "max": 312.24778899922967 + } + }, + { + "start": 170466.044415819, + "first": 170466.652497454, + "last": 170482.879366796, + "end": 170482.88015465, + "elapsed_s": 16.83573883099598, + "chunk_offsets_s": [ + 0.608081635000417, + 0.7470271050115116, + 1.0594502690073568, + 1.2337741999945138, + 1.3940153540170286, + 1.565481613011798, + 1.7337545680056792, + 1.9060287940083072, + 2.078134013019735, + 2.248140917014098, + 2.4223253460077103, + 2.6066696170018986, + 2.790776978014037, + 2.9739423590071965, + 3.152131322014611, + 3.330590588011546, + 3.5139902560040355, + 3.697408356994856, + 3.8817806470033247, + 4.061685560009209, + 4.242728837009054, + 4.424170609010616, + 4.601891975005856, + 4.786659450008301, + 4.970852783997543, + 5.158639884000877, + 5.346726920019137, + 5.529288163001183, + 5.714574757992523, + 5.907056662021205, + 6.088663302012719, + 6.269101849000435, + 6.456520887993975, + 6.638528181996662, + 6.819407321017934, + 7.001763528009178, + 7.187623422010802, + 7.372232951020123, + 7.557814222003799, + 7.745778317999793, + 7.935746953997295, + 8.116490095999325, + 8.297569909016602, + 8.48033397400286, + 8.66155189901474, + 8.847789171006298, + 9.02904165702057, + 9.211646556010237, + 9.400623326015193, + 9.577403697010595, + 9.760110483999597, + 9.941988811013289, + 10.119996957015246, + 10.305293401994277, + 10.492133588006254, + 10.674852905009175, + 10.850639537995448, + 11.029033844999503, + 11.208684932003962, + 11.395444304012926, + 11.581533996999497, + 11.764918450004188, + 11.944424835994141, + 12.126198336016387, + 12.32204758899752, + 12.513653221016284, + 12.703596714010928, + 12.889832379005384, + 13.065856568020536, + 13.253544709994458, + 13.427993318997324, + 13.606501902017044, + 13.7835469370184, + 13.966735861002235, + 14.144223295996198, + 14.315902091009775, + 14.484512351016747, + 14.639354807994096, + 14.790308732015546, + 14.94518315899768, + 15.094041778007522, + 15.233089272020152, + 15.361408251017565, + 15.481774548010435, + 15.594857376010623, + 15.714344619016629, + 15.820144060009625, + 15.900555330998031, + 15.965903297998011, + 16.032510045013623, + 16.10731846699491, + 16.18235608699615, + 16.25351866299752, + 16.331149731995538, + 16.41782261800836, + 16.50713186399662, + 16.597738306998508, + 16.683686714008218, + 16.763202610018197, + 16.834950977005064 + ], + "chunk_gaps_ms": { + "count": 99, + "p50": 180.4385469877161, + "p95": 189.96863599750213, + "max": 312.42316399584524 + } + }, + { + "start": 170466.043895053, + "first": 170466.406972099, + "last": 170481.864644311, + "end": 170481.864717687, + "elapsed_s": 15.820822634006618, + "chunk_offsets_s": [ + 0.3630770460003987, + 0.4636121639923658, + 0.747724141983781, + 1.0603243299992755, + 1.2341929499816615, + 1.3945746319950558, + 1.565773594978964, + 1.734308549988782, + 1.9065225519880187, + 2.0785821229801513, + 2.248462916002609, + 2.4229014559823554, + 2.6069809919863474, + 2.7912696480052546, + 2.974434709001798, + 3.1525777200004086, + 3.3310368899838068, + 3.514236062997952, + 3.698019409988774, + 3.8824233649938833, + 4.062461541005177, + 4.243665793997934, + 4.424750895006582, + 4.602752851991681, + 4.787286503997166, + 4.971698188979644, + 5.159369673987385, + 5.347804179997183, + 5.5302799829805735, + 5.715333106985781, + 5.907963698991807, + 6.089305746980244, + 6.270152693003183, + 6.457477796997409, + 6.639335539977765, + 6.820329093985492, + 7.002680132980458, + 7.188470155000687, + 7.373049571993761, + 7.558768763003172, + 7.7466186989913695, + 7.936691238981439, + 8.117512300988892, + 8.29845723399194, + 8.481339043006301, + 8.662212519993773, + 8.848628287989413, + 9.029930774006061, + 9.212695480993716, + 9.401513819000684, + 9.578078414982883, + 9.761165375995915, + 9.94280805598828, + 10.120876105997013, + 10.30592952799634, + 10.492965152981924, + 10.675832117005484, + 10.85148361499887, + 11.029255795001518, + 11.209453920979286, + 11.396205661003478, + 11.582520058000227, + 11.765934078983264, + 11.945378176984377, + 12.12699642899679, + 12.32300217699958, + 12.514533265988575, + 12.704533686977811, + 12.890775943989865, + 13.066715556982672, + 13.254629057977581, + 13.428965859988239, + 13.60719215599238, + 13.784632644994417, + 13.967335121997166, + 14.145199420978315, + 14.316801367996959, + 14.485425067978213, + 14.640127828984987, + 14.790890201984439, + 14.945864501001779, + 15.094647375983186, + 15.233746949990746, + 15.362022840999998, + 15.482234145980328, + 15.59543951798696, + 15.715066360979108, + 15.820749257982243 + ], + "chunk_gaps_ms": { + "count": 87, + "p50": 181.3024860166479, + "p95": 191.53108898899518, + "max": 312.6001880154945 + } + }, + { + "start": 170466.04353267, + "first": 170466.936194443, + "last": 170481.277607059, + "end": 170481.277703762, + "elapsed_s": 15.234171091986354, + "chunk_offsets_s": [ + 0.8926617729885038, + 1.0605264579935465, + 1.234295670001302, + 1.394696200004546, + 1.566042363003362, + 1.7344061659823637, + 1.9068397989904042, + 2.0788366019842215, + 2.2489791059924755, + 2.42341292600031, + 2.6075087659992278, + 2.791587070998503, + 2.9746896519791335, + 3.152836438996019, + 3.3312898659787606, + 3.514830925996648, + 3.6983371370006353, + 3.8829983879986685, + 4.06300429199473, + 4.244292447983753, + 4.425160062004579, + 4.603158018988324, + 4.787691926991101, + 4.972104315995239, + 5.15982623299351, + 5.348277602985036, + 5.530595213989727, + 5.71579248199123, + 5.908373281999957, + 6.089714065979933, + 6.270334420987638, + 6.457746467989637, + 6.639744209998753, + 6.820791284000734, + 7.003140435990645, + 7.188878729997668, + 7.373458514979575, + 7.5592275289818645, + 7.747152152995113, + 7.93710173398722, + 8.11770234798314, + 8.298864401003812, + 8.481535825994797, + 8.662625575001584, + 8.849036750994856, + 9.030341044999659, + 9.212847144983243, + 9.401925993995974, + 9.578609724994749, + 9.761455887986813, + 9.943218262982555, + 10.121284776978428, + 10.306347462988924, + 10.493525470985333, + 10.676149028993677, + 10.85206652499619, + 11.029852690000553, + 11.210033167997608, + 11.396734587993706, + 11.583001705002971, + 11.766260685981251, + 11.945706864004023, + 12.127395291987341, + 12.323397935979301, + 12.5149304009974, + 12.704966549994424, + 12.891219285986153, + 13.067041923990473, + 13.254907904978609, + 13.429268034989946, + 13.607149883988313, + 13.784957475989358, + 13.967482913983986, + 14.145642795978347, + 14.317197622993262, + 14.485748331004288, + 14.640406180988066, + 14.79128887297702, + 14.946193140000105, + 15.095042574976105, + 15.234074388979934 + ], + "chunk_gaps_ms": { + "count": 80, + "p50": 181.51460598164704, + "p95": 189.94958099210635, + "max": 196.0026439919602 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 29.1, + "per_stream_tps": 29.1, + "ttft_s_max": 0.22, + "timing_valid": true, + "end_to_end_tps": 28.542519010321044, + "makespan_s": 8.96907522098627, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 29.132837058407393, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 8.96907522098627, + "end_to_end_tps": 28.542519010321044, + "timing_valid": true, + "per_stream_tps": [ + 29.1 + ], + "ttft_s": [ + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + } + ], + "client_intervals": [ + { + "start": 170577.552871268, + "first": 170577.768798061, + "last": 170586.521807577, + "end": 170586.521946489, + "elapsed_s": 8.96907522098627, + "chunk_offsets_s": [ + 0.21592679299646989, + 0.29451083298772573, + 0.3683192019816488, + 0.4331080819829367, + 0.5002501739945728, + 0.5731434879999142, + 0.6382865749765188, + 0.7029159999801777, + 0.7678254559868947, + 0.8329436949861702, + 0.8969077609945089, + 0.9615399529866409, + 1.0262098099919967, + 1.092086175980512, + 1.1557692179922014, + 1.2197299870022107, + 1.285783425992122, + 1.3628015809808858, + 1.4385016429878306, + 1.511567693989491, + 1.5872299000038765, + 1.6763176989916246, + 1.7591125489852857, + 1.8369223829940893, + 1.9008741449797526, + 1.9649971390026622, + 2.0276245989953168, + 2.092417559993919, + 2.1584075589780696, + 2.234618660004344, + 2.3080843740026467, + 2.38173645597999, + 2.455541898001684, + 2.540298952982994, + 2.6266669819888193, + 2.7074552119884174, + 2.793463416979648, + 2.8751756609999575, + 2.9598980929877143, + 3.0359744589950424, + 3.111073049978586, + 3.184574076993158, + 3.2587839659827296, + 3.3419812489883043, + 3.4251968349853996, + 3.509060052980203, + 3.5773445449885912, + 3.6542042219953146, + 3.7282263359811623, + 3.8022041469812393, + 3.8808317729854025, + 3.961701523978263, + 4.0282055549905635, + 4.100802166998619, + 4.165061705978587, + 4.228046222997364, + 4.294292702979874, + 4.3708978059876245, + 4.443482050992316, + 4.518734643992502, + 4.59196552800131, + 4.657010745984735, + 4.724614584003575, + 4.798148492001928, + 4.871227791998535, + 4.946576496993657, + 5.021725650003646, + 5.111237689998234, + 5.187882825004635, + 5.254080968996277, + 5.3267411509878, + 5.391473330004374, + 5.4543371759937145, + 5.519153211993398, + 5.585036460979609, + 5.650766045990167, + 5.7161499520007055, + 5.779624325979967, + 5.845324134978, + 5.911139576986898, + 5.975565276981797, + 6.038927905989112, + 6.101573512976756, + 6.165604588983115, + 6.231021022977075, + 6.302200056001311, + 6.377665657986654, + 6.4528138199821115, + 6.531554568995489, + 6.611193524004193, + 6.676094342983561, + 6.740713466977468, + 6.807089915993856, + 6.871982511976967, + 6.9377853619807865, + 7.012489988992456, + 7.086507672996959, + 7.162878504983382, + 7.236082318995614, + 7.29906843797653, + 7.364264809992164, + 7.428223023976898, + 7.492260948987678, + 7.556385770993074, + 7.621729357982986, + 7.687684543983778, + 7.752124228980392, + 7.815713946998585, + 7.879843873000937, + 7.9445438609982375, + 8.008623562986031, + 8.073307726997882, + 8.13758777998737, + 8.206698769994546, + 8.282252835982945, + 8.356934776995331, + 8.429101344983792, + 8.5062737139815, + 8.58673766098218, + 8.651946912985295, + 8.715783224004554, + 8.779871837992687, + 8.844413602986606, + 8.908270824991632, + 8.968936308985576 + ], + "chunk_gaps_ms": { + "count": 124, + "p50": 66.82306151196826, + "p95": 83.86321799480356, + "max": 89.51203999458812 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 40.4, + "per_stream_tps": 20.6, + "ttft_s_max": 0.51, + "timing_valid": true, + "end_to_end_tps": 39.85462641103368, + "makespan_s": 12.846689233003417, + "steady_tps": 41.2, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 40.381896491585174, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 12.846689233003417, + "end_to_end_tps": 39.85462641103368, + "timing_valid": true, + "steady_tps": 41.249907483073315, + "per_stream_tps": [ + 20.6, + 20.7 + ], + "ttft_s": [ + 0.22, + 0.51 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + } + ], + "client_intervals": [ + { + "start": 170586.523619158, + "first": 170586.740536767, + "last": 170599.102485474, + "end": 170599.10260941, + "elapsed_s": 12.578990252019139, + "chunk_offsets_s": [ + 0.21691760901012458, + 0.29498684001737274, + 0.6134830850060098, + 0.7288597430160735, + 0.8491547610028647, + 0.9678163260105066, + 1.0824770019971766, + 1.2012050150078721, + 1.3177472070092335, + 1.42934548901394, + 1.5478616150212474, + 1.6726008990081027, + 1.7868152080045547, + 1.909997167007532, + 2.0279004460026044, + 2.1470607160008512, + 2.2688502439996228, + 2.3902255349967163, + 2.5109486510045826, + 2.629232745995978, + 2.749814310023794, + 2.86926702799974, + 2.9920133569976315, + 3.116623937996337, + 3.2375300450075883, + 3.3593325040128548, + 3.4830638560233638, + 3.6092918119975366, + 3.7288897950202227, + 3.8482770420087036, + 3.967206849018112, + 4.091157033020863, + 4.2118996220233385, + 4.323849345004419, + 4.439869015011936, + 4.55819626399898, + 4.674266574002104, + 4.7872645840107, + 4.909975939022843, + 5.028611123998417, + 5.145964071009075, + 5.26461220101919, + 5.381930029019713, + 5.50492039200617, + 5.628827826003544, + 5.753914458007785, + 5.871336751006311, + 5.993480171018746, + 6.111897918017348, + 6.234435642021708, + 6.3514658080239315, + 6.473354957997799, + 6.597083016997203, + 6.7202024060243275, + 6.840990134020103, + 6.965768623020267, + 7.0865864630031865, + 7.209312060003867, + 7.3256147720094305, + 7.446400453016395, + 7.564052667003125, + 7.68578528999933, + 7.808936408022419, + 7.930421400000341, + 8.047619710996514, + 8.164973557024496, + 8.28279316300177, + 8.400500370014925, + 8.521559556014836, + 8.644860066007823, + 8.766086323012132, + 8.89228876499692, + 9.01552729101968, + 9.138518521998776, + 9.264002901007188, + 9.387934771017171, + 9.508086007001111, + 9.631473846005974, + 9.74901142300223, + 9.865990056016017, + 9.99057581301895, + 10.111480136023602, + 10.227587986999424, + 10.344174301018938, + 10.462939781020395, + 10.581420493021142, + 10.701349314011168, + 10.820486825017724, + 10.937690163002117, + 11.054572398017626, + 11.178729341016151, + 11.29786549200071, + 11.424056737014325, + 11.547135475004325, + 11.664137278014096, + 11.785747137997532, + 11.907467526005348, + 12.02418249801849, + 12.143374298000708, + 12.263163153023925, + 12.377500913018594, + 12.477668775012717, + 12.578866316005588 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 119.85883800662123, + "p95": 125.08663200424053, + "max": 318.49624498863705 + } + }, + { + "start": 170586.52352463, + "first": 170587.031800912, + "last": 170599.369958471, + "end": 170599.370213863, + "elapsed_s": 12.846689233003417, + "chunk_offsets_s": [ + 0.5082762819947675, + 0.6135079170053359, + 0.7288748789869715, + 0.849014345993055, + 0.967795158008812, + 1.0823730819975026, + 1.2011291270027868, + 1.3180061189923435, + 1.4295589929970447, + 1.5477181439928245, + 1.6723600680124946, + 1.787077176006278, + 1.9100364150071982, + 2.0282524620124605, + 2.1474262189876754, + 2.269112756010145, + 2.390354926988948, + 2.5111000110046007, + 2.629262986010872, + 2.7498746780038346, + 2.8693264840112533, + 2.992145244992571, + 3.1166789139970206, + 3.237464814010309, + 3.3592580239928793, + 3.482989023992559, + 3.6091843240137678, + 3.728818099014461, + 3.848452576989075, + 3.9671382420056034, + 4.091091417009011, + 4.211767350003356, + 4.3239058410108555, + 4.4399308229913, + 4.5582559120084625, + 4.674327741988236, + 4.787299560004612, + 4.910106786992401, + 5.028798307990655, + 5.146023495995905, + 5.264672280987725, + 5.382061373005854, + 5.50506512800348, + 5.6289573299873155, + 5.754097817989532, + 5.8713962550100405, + 5.993629050994059, + 6.112026477989275, + 6.234495849988889, + 6.351594767998904, + 6.473370974010322, + 6.59721461700974, + 6.720261637994554, + 6.841008054005215, + 6.9658985910064075, + 7.086646767013008, + 7.209372156008612, + 7.325673843995901, + 7.446439380990341, + 7.5641120270011015, + 7.685914089990547, + 7.808995416009566, + 7.930485096003395, + 8.047679390991107, + 8.165102644998115, + 8.282782443013275, + 8.40063081801054, + 8.521620499988785, + 8.644917906000046, + 8.766216482996242, + 8.892348557012156, + 9.015583163010888, + 9.1386487139971, + 9.264059589011595, + 9.38799469100195, + 9.508106310997391, + 9.631620245985687, + 9.74906809499953, + 9.866051224002149, + 9.990706660988508, + 10.111541688005673, + 10.227716914989287, + 10.344307308987482, + 10.46299947699299, + 10.581618828990031, + 10.701483442011522, + 10.820637080993038, + 10.937869315006537, + 11.054628829995636, + 11.178785564989084, + 11.297924932005117, + 11.424112432985567, + 11.547193331003655, + 11.664283118007006, + 11.785880178009393, + 11.907644613995217, + 12.02431406598771, + 12.143431209988194, + 12.263173473009374, + 12.377630192990182, + 12.477799031010363, + 12.579000219993759, + 12.658407634007744, + 12.721710752986837, + 12.786332685005618, + 12.846433841012185 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 119.63377500069328, + "p95": 124.89053700119257, + "max": 126.19530002120882 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 51.0, + "per_stream_tps": 13.6, + "ttft_s_max": 1.18, + "timing_valid": true, + "end_to_end_tps": 50.61233434023709, + "makespan_s": 20.232222310005454, + "steady_tps": 54.0, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.96240078918947, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 20.232222310005454, + "end_to_end_tps": 50.61233434023709, + "timing_valid": true, + "steady_tps": 53.97544303065838, + "per_stream_tps": [ + 13.8, + 13.3, + 13.8, + 13.4 + ], + "ttft_s": [ + 0.51, + 0.22, + 0.83, + 1.18 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2e4cb369c515", + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8" + } + ], + "client_intervals": [ + { + "start": 170599.372593923, + "first": 170599.878813674, + "last": 170618.409095991, + "end": 170618.409370871, + "elapsed_s": 19.03677694799262, + "chunk_offsets_s": [ + 0.5062197509978432, + 0.6130643719807267, + 0.9674195270054042, + 1.3374602260009851, + 1.4961685439920984, + 1.6628760029852856, + 1.8264118029910605, + 1.9828785739955492, + 2.1543272729904857, + 2.3216314509918448, + 2.4926993679837324, + 2.668717310996726, + 2.8386545259854756, + 3.0180653909919783, + 3.195804691000376, + 3.3818565539841074, + 3.5597303510003258, + 3.7405904319894034, + 3.914272363006603, + 4.086200841004029, + 4.2648762690078, + 4.443858129001455, + 4.623426004982321, + 4.801536458980991, + 4.982983452995541, + 5.163429840002209, + 5.344353873981163, + 5.524727973999688, + 5.701729135005735, + 5.881379508995451, + 6.067151266004657, + 6.254809370992007, + 6.436822093004594, + 6.6220289889897685, + 6.8052539670025, + 6.98792209700332, + 7.167960952006979, + 7.343922164989635, + 7.527445782994619, + 7.705420882004546, + 7.8830065579968505, + 8.059623723005643, + 8.239040165004553, + 8.425008388003334, + 8.611577459989348, + 8.796352982986718, + 8.979227868985618, + 9.164462398999603, + 9.352667340979679, + 9.533775830001105, + 9.715027647995157, + 9.899925667996285, + 10.093935035984032, + 10.279452062997734, + 10.468481085990788, + 10.649763416993665, + 10.832587490003789, + 11.012959805986611, + 11.193321946979268, + 11.37204828200629, + 11.559874428989133, + 11.738575869007036, + 11.92508136300603, + 12.100675080000656, + 12.27715036299196, + 12.454442557995208, + 12.632276176998857, + 12.818589047994465, + 12.99627417200827, + 13.181189605005784, + 13.36209664400667, + 13.541336069989484, + 13.72343365298002, + 13.914138294989243, + 14.093461305979872, + 14.275217321002856, + 14.447488645993872, + 14.62191169700236, + 14.79503807798028, + 14.975991968007293, + 15.15563734798343, + 15.333837257989217, + 15.512538735987619, + 15.691219414002262, + 15.871463130984921, + 16.053628908994142, + 16.230213365983218, + 16.40775023898459, + 16.58918894699309, + 16.76901498498046, + 16.947026673995424, + 17.12726687299437, + 17.307295214995975, + 17.48543549599708, + 17.65524731000187, + 17.81930311600445, + 17.99716373498086, + 18.17551203299081, + 18.352331597998273, + 18.532351045985706, + 18.704796121979598, + 18.869490985001903, + 19.036502067989204 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 179.49215898988768, + "p95": 188.2049419800751, + "max": 370.0406989955809 + } + }, + { + "start": 170599.372434691, + "first": 170599.589718118, + "last": 170618.79559727, + "end": 170618.795705734, + "elapsed_s": 19.423271042993292, + "chunk_offsets_s": [ + 0.21728342698770575, + 0.2948528429842554, + 0.61352336400887, + 0.9677522949932609, + 1.3373808340111282, + 1.4963601919880603, + 1.663006115006283, + 1.8266017549904063, + 1.9830068619921803, + 2.154527289007092, + 2.321744059008779, + 2.492893975984771, + 2.6688451830123086, + 2.8390975020010956, + 3.0183839029923547, + 3.196264178986894, + 3.3819470510061365, + 3.560125630989205, + 3.740695215994492, + 3.9149857859883923, + 4.086461800994584, + 4.264755693991901, + 4.444069536984898, + 4.6234094130049925, + 4.80153853999218, + 4.9829327809857205, + 5.163541104004253, + 5.344224481988931, + 5.525002420996316, + 5.701986158994259, + 5.881490052997833, + 6.06724150601076, + 6.255028315004893, + 6.437025757011725, + 6.622249420004664, + 6.805481902003521, + 6.9881207369908225, + 7.168034472008003, + 7.344133509002859, + 7.52750754301087, + 7.705525969999144, + 7.883255997992819, + 8.05985103501007, + 8.23923843598459, + 8.425248596002348, + 8.61178077198565, + 8.796465750987409, + 8.979438940994442, + 9.164753230987117, + 9.353000044997316, + 9.533998405997409, + 9.715232640010072, + 9.90002488400205, + 10.093997740012128, + 10.279657119011972, + 10.468682926002657, + 10.649877721007215, + 10.83265776198823, + 11.01305351901101, + 11.193397274997551, + 11.37216693800292, + 11.559895132988458, + 11.738637308997568, + 11.925327297998592, + 12.100877094984753, + 12.277391690993682, + 12.454670701990835, + 12.63235310500022, + 12.818558616010705, + 12.996390668005915, + 13.18140866101021, + 13.362203875993146, + 13.541775429999689, + 13.723638564988505, + 13.914346198987914, + 14.093677881988697, + 14.275165737984935, + 14.447355718002655, + 14.62212312099291, + 14.795009310008027, + 14.976255968009355, + 15.155837811995298, + 15.333884922001744, + 15.512650015996769, + 15.691280230006669, + 15.871718746988336, + 16.053832061006688, + 16.230483077990357, + 16.407844558998477, + 16.589412626984995, + 16.769216296990635, + 16.947125090984628, + 17.127479304996086, + 17.307534719002433, + 17.48585024799104, + 17.655694669985678, + 17.81941897200886, + 17.99736747899442, + 18.175963665009476, + 18.352440766000655, + 18.532246246992145, + 18.70500765799079, + 18.869698536989745, + 19.036911331990268, + 19.185994951985776, + 19.318428801983828, + 19.423162578983465 + ], + "chunk_gaps_ms": { + "count": 106, + "p50": 179.36363849730697, + "p95": 189.02580699068494, + "max": 369.6285390178673 + } + }, + { + "start": 170599.37313997, + "first": 170600.19980728, + "last": 170618.690666742, + "end": 170618.690998677, + "elapsed_s": 19.317858706985135, + "chunk_offsets_s": [ + 0.8266673100006301, + 0.9672233680030331, + 1.337355858006049, + 1.4960854570090305, + 1.6627975399896968, + 1.8262914829829242, + 1.9826928300026339, + 2.1541470020019915, + 2.321490875998279, + 2.4925691289827228, + 2.6686241909919772, + 2.838465646986151, + 3.0178463039919734, + 3.1957602909824345, + 3.3809974519826937, + 3.559634814999299, + 3.740107152989367, + 3.9141294519940857, + 4.0860103300074115, + 4.264391870005056, + 4.4434494099987205, + 4.623008597991429, + 4.801115948008373, + 4.982379453984322, + 5.163337200006936, + 5.34390801898553, + 5.52453190999222, + 5.701558047992876, + 5.881148485001177, + 6.066975425987039, + 6.254632044001482, + 6.436718637996819, + 6.621652076981263, + 6.804985246999422, + 6.987825746007729, + 7.167682233004598, + 7.343766884994693, + 7.527208599989535, + 7.705240210983902, + 7.8828373109863605, + 8.059352412004955, + 8.23878667698591, + 8.424837717000628, + 8.61131035699509, + 8.796187943982659, + 8.978893246006919, + 9.164268560009077, + 9.35249638199457, + 9.533689286996378, + 9.714892463991418, + 9.899622628989164, + 10.093777756002964, + 10.279183776001446, + 10.468389597983332, + 10.649608840991277, + 10.832357314007822, + 11.012652542995056, + 11.192996427998878, + 11.371999737981241, + 11.559573276987066, + 11.738264557992807, + 11.925030674989102, + 12.100493831996573, + 12.277008172008209, + 12.454229901981307, + 12.631912449985975, + 12.81815008900594, + 12.996072587993694, + 13.180886918009492, + 13.361879443982616, + 13.541206837981008, + 13.723317188996589, + 13.913962296006503, + 14.09323077899171, + 14.274852058006218, + 14.447032934986055, + 14.621605505992193, + 14.794713582989061, + 14.97587025599205, + 15.15544581200811, + 15.333574346004752, + 15.512282176001463, + 15.691048902983312, + 15.871450042992365, + 16.053358076984296, + 16.23013341499609, + 16.407597310986603, + 16.58895309100626, + 16.76872600999195, + 16.946714290999807, + 17.127078632998746, + 17.30722463899292, + 17.485454712994397, + 17.655091391003225, + 17.81913654098753, + 17.996869623981183, + 18.175452338007744, + 18.35214383798302, + 18.531917126994813, + 18.704421690985328, + 18.86884889000794, + 19.035717892984394, + 19.185120120993815, + 19.317526772007113 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 179.3806570058223, + "p95": 187.65661801444367, + "max": 370.13249000301585 + } + }, + { + "start": 170599.372353747, + "first": 170600.553684308, + "last": 170619.604473785, + "end": 170619.604576057, + "elapsed_s": 20.232222310005454, + "chunk_offsets_s": [ + 1.1813305609975941, + 1.3378908500017133, + 1.4966731680033263, + 1.6633398749982007, + 1.826917995000258, + 1.9833138690155465, + 2.154775305010844, + 2.32203468302032, + 2.4931565040023997, + 2.6691852300136816, + 2.8389313100196887, + 3.0183667830133345, + 3.196094979008194, + 3.381897339015268, + 3.5603165430075023, + 3.740517024009023, + 3.914755915000569, + 4.086481897014892, + 4.264877198002068, + 4.443870529008564, + 4.623390757013112, + 4.801519675995223, + 4.982753150019562, + 5.163918927020859, + 5.344543265993707, + 5.525016326020705, + 5.702010575012537, + 5.881678773002932, + 6.067216177994851, + 6.255002523015719, + 6.437023453996517, + 6.62212983702193, + 6.805445295001846, + 6.9880676490138285, + 7.168160264001926, + 7.344265189021826, + 7.527630694996333, + 7.705565058015054, + 7.883286526019219, + 8.059822219016496, + 8.239232741005253, + 8.425287827994907, + 8.611714372993447, + 8.796499718999257, + 8.979411692998838, + 9.164663135015871, + 9.352951997017954, + 9.533973542012973, + 9.715228512010071, + 9.900002901005792, + 10.094127756019589, + 10.279589375015348, + 10.468680926016532, + 10.649917721020756, + 10.832779874006519, + 11.013034959003562, + 11.193521003006026, + 11.372340250003617, + 11.559932317002676, + 11.738763597008074, + 11.925359810993541, + 12.100826968002366, + 12.277429562993348, + 12.454627725994214, + 12.632556001015473, + 12.81854196099448, + 12.996376827999484, + 13.1813145490014, + 13.362232228013454, + 13.541804406006122, + 13.72361059699324, + 13.914340023009572, + 14.093594282021513, + 14.275416969001526, + 14.447679622011492, + 14.622056177002378, + 14.795139518013457, + 14.97626995202154, + 15.155961028009187, + 15.334015625994653, + 15.512820799020119, + 15.691406502010068, + 15.871743195020827, + 16.053819837019546, + 16.230492694012355, + 16.407882543018786, + 16.589335074997507, + 16.76920474501094, + 16.94715377900866, + 17.12746821701876, + 17.307574014994316, + 17.485869096009992, + 17.655724334006663, + 17.819459004007513, + 17.997308999008965, + 18.17599523300305, + 18.352419438015204, + 18.532533671008423, + 18.70498285800568, + 18.869985975994496, + 19.036952947993996, + 19.186133512004744, + 19.318455169996014, + 19.42318800301291, + 19.50115015299525, + 19.564852993993554, + 19.631356472003972, + 19.706970225000987, + 19.780679532006616, + 19.85078607700416, + 19.915127925021807, + 19.978551759006223, + 20.04315675899852, + 20.1080992630159, + 20.172208024014253, + 20.23212003801018 + ], + "chunk_gaps_ms": { + "count": 115, + "p50": 178.68623399408534, + "p95": 187.59206699905917, + "max": 194.12485501379706 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 56.1, + "per_stream_tps": 56.1, + "ttft_s_max": 0.15, + "timing_valid": true, + "end_to_end_tps": 54.56913804744495, + "makespan_s": 4.691296384000452, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 56.108754540087794, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 4.691296384000452, + "end_to_end_tps": 54.56913804744495, + "timing_valid": true, + "per_stream_tps": [ + 56.1 + ], + "ttft_s": [ + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 170638.857358259, + "first": 170639.003808804, + "last": 170643.548554131, + "end": 170643.548654643, + "elapsed_s": 4.691296384000452, + "chunk_offsets_s": [ + 0.1464505449985154, + 0.22084432799601927, + 0.2930342739855405, + 0.3718359080085065, + 0.4555544320028275, + 0.5422356729977764, + 0.628910914005246, + 0.7127977419877425, + 0.7948775159893557, + 0.8794060549989808, + 0.9607185820059385, + 1.0424150769831613, + 1.1277710559952538, + 1.2097125430009328, + 1.291842398000881, + 1.3762550019891933, + 1.458604376995936, + 1.5311645150068216, + 1.5954374790017027, + 1.6587007640046068, + 1.7259619320102502, + 1.8023178899893537, + 1.8780891280039214, + 1.9502043550019152, + 2.0250350190035533, + 2.1093474320077803, + 2.1923412379983347, + 2.272731239994755, + 2.3513327309919987, + 2.45443492999766, + 2.557295176986372, + 2.6642342999984976, + 2.772959260008065, + 2.8771731849992648, + 2.9872616159846075, + 3.0934187719831243, + 3.203949154994916, + 3.3144693140056916, + 3.4247877770103514, + 3.533252130000619, + 3.6430629460082855, + 3.74951768599567, + 3.8594131889985874, + 3.9689242299937177, + 4.0793243740045, + 4.184218876005616, + 4.29286478200811, + 4.400618608982768, + 4.507156468986068, + 4.617385861987714, + 4.691195872001117 + ], + "chunk_gaps_ms": { + "count": 50, + "p50": 84.47057149896864, + "p95": 110.40014401078224, + "max": 110.53038301179186 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 64.0, + "per_stream_tps": 32.6, + "ttft_s_max": 0.36, + "timing_valid": true, + "end_to_end_tps": 63.125641826998525, + "makespan_s": 8.110808622004697, + "steady_tps": 65.0, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 64.03685521427283, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 8.110808622004697, + "end_to_end_tps": 63.125641826998525, + "timing_valid": true, + "steady_tps": 64.99252195995876, + "per_stream_tps": [ + 32.9, + 32.3 + ], + "ttft_s": [ + 0.36, + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 170643.550157169, + "first": 170643.914599867, + "last": 170651.660517711, + "end": 170651.660689231, + "elapsed_s": 8.110532062011771, + "chunk_offsets_s": [ + 0.3644426980172284, + 0.4681995389983058, + 0.5763190140132792, + 0.6833740269939881, + 0.7912139669933822, + 0.8998603060026653, + 1.0071285020094365, + 1.1122695970116183, + 1.2151724389987066, + 1.3121488710166886, + 1.4137277950067073, + 1.5186665710061789, + 1.6351842459989712, + 1.7468414140166715, + 1.859048326004995, + 1.9739188670064323, + 2.088346767995972, + 2.2038714039954357, + 2.3167405069980305, + 2.4323414800164755, + 2.548528276005527, + 2.6616633480007295, + 2.771080200007418, + 2.882921432988951, + 2.99515368900029, + 3.1054757880046964, + 3.2113737000036053, + 3.3189534500124864, + 3.42690681599197, + 3.5353677819948643, + 3.6426003169908654, + 3.7525610089942347, + 3.8624658930057194, + 3.9720596260158345, + 4.085117242007982, + 4.199169322004309, + 4.306554689013865, + 4.417122790007852, + 4.531985237001209, + 4.646437268995214, + 4.7559714340022765, + 4.867436238011578, + 4.982739277009387, + 5.095360831008293, + 5.210200589994201, + 5.325606653001159, + 5.440500574011821, + 5.554740815015975, + 5.66800689700176, + 5.783045616990421, + 5.896805155003676, + 6.010243252996588, + 6.121830329007935, + 6.235908858012408, + 6.349433693016181, + 6.462716336012818, + 6.577446017006878, + 6.692743858002359, + 6.806397476990242, + 6.919607576011913, + 7.028534879995277, + 7.143013105989667, + 7.25626002100762, + 7.36362217500573, + 7.475941044016508, + 7.5898386310145725, + 7.7009143980103545, + 7.813268483005231, + 7.927688133990159, + 8.036315342993475, + 8.110360541992122 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 112.27556251105852, + "p95": 115.5246359994635, + "max": 116.51767499279231 + } + }, + { + "start": 170643.549880609, + "first": 170643.696353976, + "last": 170651.585722369, + "end": 170651.585823536, + "elapsed_s": 8.035942927002907, + "chunk_offsets_s": [ + 0.1464733669999987, + 0.22076120000565425, + 0.46824569898308255, + 0.5762096869875677, + 0.6832173079892527, + 0.7912167829927057, + 0.8999316179833841, + 1.006916710990481, + 1.1121309099835344, + 1.2151581509970129, + 1.3123102159879636, + 1.41355734798708, + 1.5186343640089035, + 1.6350458629895002, + 1.746760551002808, + 1.8589559909887612, + 1.973730338999303, + 2.0882183680078015, + 2.2038376280106604, + 2.316602940001758, + 2.432344615983311, + 2.548622339993017, + 2.661638324003434, + 2.771079992002342, + 2.8828615930106025, + 2.9950650009850506, + 3.105247227998916, + 3.211195763986325, + 3.318943321995903, + 3.4268343529838603, + 3.535311318002641, + 3.642556142003741, + 3.752504465985112, + 3.8622660219843965, + 3.971996793989092, + 4.085031755006639, + 4.198997402010718, + 4.306461601983756, + 4.417105158005143, + 4.531927189003909, + 4.646540804998949, + 4.7560327789979056, + 4.867514398007188, + 4.982870492996881, + 5.095332703000167, + 5.21031566199963, + 5.3256839659879915, + 5.440488813997945, + 5.5545833589858375, + 5.667845090007177, + 5.782983473996865, + 5.896857507003006, + 6.010298148990842, + 6.122392872988712, + 6.235936426994158, + 6.349355980986729, + 6.4626516480057035, + 6.577304065984208, + 6.692553186992882, + 6.80635606899159, + 6.919572199985851, + 7.028404879994923, + 7.1429976820072625, + 7.256184997007949, + 7.36360787198646, + 7.475925156002631, + 7.589816183986841, + 7.700838333985303, + 7.813162418984575, + 7.927680997992866, + 8.035841759992763 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 112.32068450772204, + "p95": 115.74167598155327, + "max": 247.4844989774283 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 87.0, + "per_stream_tps": 22.6, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 86.29338237677625, + "makespan_s": 11.866495110007236, + "steady_tps": 91.6, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 87.0269768356712, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 11.866495110007236, + "end_to_end_tps": 86.29338237677625, + "timing_valid": true, + "steady_tps": 91.55965811457078, + "per_stream_tps": [ + 22.5, + 23.2, + 22.8, + 22.4 + ], + "ttft_s": [ + 0.36, + 0.89, + 0.61, + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 170651.66302918, + "first": 170652.027367103, + "last": 170663.34647492, + "end": 170663.346581896, + "elapsed_s": 11.683552716014674, + "chunk_offsets_s": [ + 0.3643379230052233, + 0.468715985014569, + 0.7459381480002776, + 1.0467113410122693, + 1.2018251249974128, + 1.3545613599999342, + 1.50805121101439, + 1.660701926011825, + 1.8064691289910115, + 1.9455513629945926, + 2.087908793007955, + 2.2339851169963367, + 2.394314609002322, + 2.560249695001403, + 2.728770459012594, + 2.891556443995796, + 3.0586943780072033, + 3.222994314011885, + 3.389476537005976, + 3.557488087011734, + 3.7140121289994568, + 3.882338894007262, + 4.049900732003152, + 4.210010481998324, + 4.368752570997458, + 4.52674256399041, + 4.682721487013623, + 4.83779108399176, + 4.989093308016891, + 5.137165248015663, + 5.291133135004202, + 5.449320840998553, + 5.604668295010924, + 5.759439796995139, + 5.917169855994871, + 6.074604010995245, + 6.228361498011509, + 6.384030521003297, + 6.545118625013856, + 6.705600891000358, + 6.866816259018378, + 7.027152941009263, + 7.1874966460163705, + 7.345559059991501, + 7.50606204601354, + 7.673368706018664, + 7.838811991008697, + 8.003606029000366, + 8.166284102015197, + 8.331412845000159, + 8.492147960001603, + 8.651183622016106, + 8.813189984008204, + 8.974638571002288, + 9.135727029992267, + 9.298064161004731, + 9.462487370008603, + 9.62652043599519, + 9.78507501099375, + 9.947222029994009, + 10.109029001992894, + 10.268317320995266, + 10.428206840006169, + 10.587995623005554, + 10.749949171993649, + 10.905707671015989, + 11.065488854015712, + 11.228220307006268, + 11.38886326699867, + 11.547511940007098, + 11.683445739996387 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.21962100057863, + "p95": 168.32676500780508, + "max": 300.7731930119917 + } + }, + { + "start": 170651.663792155, + "first": 170652.553679283, + "last": 170663.52935013, + "end": 170663.529436882, + "elapsed_s": 11.865644726989558, + "chunk_offsets_s": [ + 0.8898871279961895, + 1.045704318996286, + 1.2008800379990134, + 1.3538710409775376, + 1.5071243479906116, + 1.6601628389908, + 1.8057960899895988, + 1.9449548839766067, + 2.087253529985901, + 2.2334887809993234, + 2.3937558099860325, + 2.5597535999841057, + 2.7282470839854795, + 2.890626781998435, + 3.057980122976005, + 3.2221475309925154, + 3.38875853799982, + 3.556658951973077, + 3.7131780659838114, + 3.881690062000416, + 4.0492062689736485, + 4.209434658987448, + 4.3680599789950065, + 4.526180804998148, + 4.681992543977685, + 4.837058460980188, + 4.98836132499855, + 5.136478720989544, + 5.2901895999966655, + 5.4483626819856, + 5.603949655982433, + 5.758875236992026, + 5.916440112981945, + 6.073923147981986, + 6.227486219984712, + 6.383190489985282, + 6.544534433982335, + 6.704873051989125, + 6.866250691993628, + 7.026464077993296, + 7.186888278985862, + 7.34476234097383, + 7.505267438973533, + 7.672576354991179, + 7.838011127983918, + 8.002962269994896, + 8.16545493499143, + 8.330680893996032, + 8.491421480983263, + 8.65053023098153, + 8.812285184976645, + 8.973842459992738, + 9.134893350972561, + 9.297242481989088, + 9.461790170986205, + 9.62590947697754, + 9.784281075990293, + 9.946381166984793, + 10.108335770986741, + 10.267522057984024, + 10.42732773697935, + 10.587278983992292, + 10.749260756972944, + 10.9050806479936, + 11.064660039992305, + 11.227422515978105, + 11.388252979988465, + 11.546683028980624, + 11.682608492992586, + 11.791489376977552, + 11.865557974990224 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 159.87846300413366, + "p95": 167.51620697323233, + "max": 168.51199601660483 + } + }, + { + "start": 170651.663635515, + "first": 170652.275757281, + "last": 170663.454935452, + "end": 170663.455026668, + "elapsed_s": 11.791391153004952, + "chunk_offsets_s": [ + 0.6121217659965623, + 0.7451249489968177, + 1.0455918229999952, + 1.2011695099936333, + 1.353587649995461, + 1.5068815799895674, + 1.6598657830036245, + 1.8054989379888866, + 1.944718243990792, + 2.0869130670034792, + 2.2331387820013333, + 2.393170449999161, + 2.5594423199945595, + 2.72798074799357, + 2.8904487020045053, + 3.0579462670139037, + 3.2222145390114747, + 3.3886861389910337, + 3.5565473200113047, + 3.7131029939919244, + 3.8815536469919607, + 4.049122700991575, + 4.209790018998319, + 4.367823116015643, + 4.525969076988986, + 4.681919296999695, + 4.836916252999799, + 4.988171309989411, + 5.1362561770074535, + 5.290198400005465, + 5.448923354008002, + 5.603904376010178, + 5.7586024700140115, + 5.91633560101036, + 6.073823084007017, + 6.2272718680032995, + 6.383147849992383, + 6.544395714998245, + 6.704787468013819, + 6.86604859601357, + 7.026385837991256, + 7.1867922800011, + 7.3446935410029255, + 7.505194622994168, + 7.672471763013164, + 7.837961784010986, + 8.002713630005019, + 8.165419863013085, + 8.33062806201633, + 8.49134770600358, + 8.650439863005886, + 8.812115840992192, + 8.973774604004575, + 9.134744375012815, + 9.297053090995178, + 9.461605324002448, + 9.62564459699206, + 9.784165908000432, + 9.946202752005775, + 10.108243611000944, + 10.267551130003994, + 10.427184842003044, + 10.58722504900652, + 10.748933989991201, + 10.904775177012198, + 11.064530616014963, + 11.227329732995713, + 11.38803113999893, + 11.546487990010064, + 11.682283550006105, + 11.791299937001895 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.03593750065193, + "p95": 167.86118102027103, + "max": 300.4668740031775 + } + }, + { + "start": 170651.662941772, + "first": 170651.808846476, + "last": 170663.210508784, + "end": 170663.210614384, + "elapsed_s": 11.547672612010501, + "chunk_offsets_s": [ + 0.14590470399707556, + 0.22029620700050145, + 0.4686119219986722, + 0.7462784200033639, + 1.046909324999433, + 1.2016351089987438, + 1.3544638890016358, + 1.5078048750001471, + 1.661121910001384, + 1.8063009210163727, + 1.9459758260054514, + 2.0878240570018534, + 2.2342446520051453, + 2.3941657450050116, + 2.56076185501297, + 2.7292542179930024, + 2.8912384129944257, + 3.0587337859906256, + 3.2230334659980144, + 3.389648568991106, + 3.5575449989992194, + 3.7140664650069084, + 3.8824688300082926, + 4.0500237719970755, + 4.209994673990877, + 4.368756570998812, + 4.5269966759951785, + 4.6830065910180565, + 4.837834060017485, + 4.989145819999976, + 5.137514688016381, + 5.291186846996425, + 5.449114729999565, + 5.604960982018383, + 5.75969224399887, + 5.917219808005029, + 6.074884011002723, + 6.228608378005447, + 6.384084729012102, + 6.545244192995597, + 6.7057547949953005, + 6.866938643011963, + 7.02727558100014, + 7.187642678007251, + 7.345676083990838, + 7.506182237993926, + 7.673391457996331, + 7.838971335004317, + 8.003778765007155, + 8.166340518015204, + 8.33145865300321, + 8.492308360000607, + 8.651307910011383, + 8.813214800000424, + 8.974807802995201, + 9.135643029992934, + 9.297995664994232, + 9.462609050009632, + 9.626691123994533, + 9.785054003004916, + 9.947380110010272, + 10.109152474004077, + 10.26843727301457, + 10.428259848005837, + 10.588203446997795, + 10.750076419994002, + 10.905618296004832, + 11.065545846999157, + 11.228199314995436, + 11.389031011000043, + 11.547567011992214 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.15391999098938, + "p95": 168.49236298003234, + "max": 300.63090499606915 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 30.5, + "per_stream_tps": 30.5, + "ttft_s_max": 0.22, + "timing_valid": true, + "end_to_end_tps": 29.885191052604196, + "makespan_s": 8.566115557012381, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 30.538645058563333, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 8.566115557012381, + "end_to_end_tps": 29.885191052604196, + "timing_valid": true, + "per_stream_tps": [ + 30.5 + ], + "ttft_s": [ + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 170686.904119538, + "first": 170687.119850706, + "last": 170695.469926471, + "end": 170695.470235095, + "elapsed_s": 8.566115557012381, + "chunk_offsets_s": [ + 0.21573116799117997, + 0.2931882250122726, + 0.369309603003785, + 0.4482809139881283, + 0.5303749270096887, + 0.615533898002468, + 0.6966745200043079, + 0.7774385980155785, + 0.8581587400112767, + 0.9401295380084775, + 1.014350357989315, + 1.0858536600135267, + 1.1497445210115984, + 1.2143693969992455, + 1.2797955679998267, + 1.3432275480008684, + 1.410389717988437, + 1.4747576020017732, + 1.5383895669947378, + 1.604075178009225, + 1.677099727006862, + 1.7536541449953802, + 1.8283377969928551, + 1.9000533549988177, + 1.9628343289950863, + 2.0283971079916228, + 2.104929638997419, + 2.1778886530082673, + 2.2522252810013015, + 2.3272769809991587, + 2.4100988509890158, + 2.488286837004125, + 2.5531754409894347, + 2.624599622999085, + 2.699667242995929, + 2.765590407012496, + 2.8312529459944926, + 2.900079034996452, + 2.976136415003566, + 3.0505592199915554, + 3.1247732249903493, + 3.203721227007918, + 3.284898026002338, + 3.350593109993497, + 3.4220807820092887, + 3.487065977999009, + 3.5521066790097393, + 3.619275553006446, + 3.695401525008492, + 3.7724763920123223, + 3.846754605008755, + 3.910105802991893, + 3.9724756260111462, + 4.035093481012154, + 4.099466309999116, + 4.164098740002373, + 4.2298341279965825, + 4.295796924008755, + 4.360968921013409, + 4.426437910005916, + 4.490878178999992, + 4.554529729008209, + 4.618284063006286, + 4.682101468992187, + 4.746459899004549, + 4.811818887013942, + 4.875165206001839, + 4.939099364011781, + 5.005897024006117, + 5.081272981013171, + 5.158522120997077, + 5.237179980002111, + 5.316688334016362, + 5.404024729999946, + 5.485755066998536, + 5.568748057994526, + 5.650445483013755, + 5.7415610600146465, + 5.8257549940026365, + 5.894877309008734, + 5.958953722991282, + 6.021824251016369, + 6.085424074000912, + 6.150000504014315, + 6.214272439014167, + 6.279307877004612, + 6.345546194002964, + 6.4158363309979904, + 6.486762485001236, + 6.551317347009899, + 6.615990737016546, + 6.682070845999988, + 6.755242549988907, + 6.82756150199566, + 6.9026097010064404, + 6.974568781995913, + 7.037720046006143, + 7.103107708011521, + 7.1670903470076155, + 7.231557338003768, + 7.298214807000477, + 7.373110654007178, + 7.44854845199734, + 7.520747197006131, + 7.594772356998874, + 7.670261899009347, + 7.7545330509892665, + 7.839221497997642, + 7.927721927000675, + 8.009420040994883, + 8.091213515988784, + 8.173489166016225, + 8.25729338999372, + 8.34442529201624, + 8.428545149014099, + 8.504957522993209, + 8.565806933009299 + ], + "chunk_gaps_ms": { + "count": 116, + "p50": 71.60943001508713, + "p95": 84.68844700837508, + "max": 91.11557700089179 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 43.6, + "per_stream_tps": 22.2, + "ttft_s_max": 0.5, + "timing_valid": true, + "end_to_end_tps": 42.98004314767995, + "makespan_s": 11.91250549099641, + "steady_tps": 44.8, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 43.603089958840165, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 11.91250549099641, + "end_to_end_tps": 42.98004314767995, + "timing_valid": true, + "steady_tps": 44.82518314667624, + "per_stream_tps": [ + 22.1, + 22.4 + ], + "ttft_s": [ + 0.22, + 0.5 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 170695.471885574, + "first": 170695.687566819, + "last": 170707.247057637, + "end": 170707.247160069, + "elapsed_s": 11.775274495012127, + "chunk_offsets_s": [ + 0.2156812449975405, + 0.29346289800014347, + 0.6117105320154224, + 0.7259263030136935, + 0.8349601410154719, + 0.9451702189981006, + 1.0632381140021607, + 1.1803584750159644, + 1.2930643900181167, + 1.4074605130008422, + 1.5232607320067473, + 1.6353304730146192, + 1.7521306099952199, + 1.8652954380086157, + 1.9845992060145363, + 2.105464253021637, + 2.2265891870192718, + 2.3467732580029406, + 2.4658595390210394, + 2.584087436000118, + 2.704193636018317, + 2.822832397010643, + 2.94384662801167, + 3.0628060290182475, + 3.187140434019966, + 3.304525259998627, + 3.4205271280079614, + 3.5378890900174156, + 3.6537020140094683, + 3.767023276013788, + 3.8913633140036836, + 4.015306712011807, + 4.139658989995951, + 4.262224741018144, + 4.382823534018826, + 4.498982794000767, + 4.618511908018263, + 4.739005038019968, + 4.85502377001103, + 4.973134005005704, + 5.090302401018562, + 5.210967243008781, + 5.330834405001951, + 5.447199857997475, + 5.568714043009095, + 5.682931642018957, + 5.802833429013845, + 5.926370589004364, + 6.044341641012579, + 6.1654444510058966, + 6.284969694010215, + 6.403919625998242, + 6.518849145009881, + 6.636462006019428, + 6.754563827009406, + 6.864831670012791, + 6.974339784996118, + 7.091307462018449, + 7.21434472000692, + 7.33691821701359, + 7.462262034008745, + 7.584932333003962, + 7.706291800015606, + 7.815440603997558, + 7.938240390998544, + 8.059468434017617, + 8.180180030001793, + 8.301388570020208, + 8.413272908015642, + 8.532253929995932, + 8.648157418007031, + 8.767605720000574, + 8.885824119002791, + 9.004821653012186, + 9.115834120020736, + 9.226231228007236, + 9.342481500003487, + 9.459551580017433, + 9.574007229995914, + 9.690914766018977, + 9.803363378014183, + 9.92085985801532, + 10.038696577015799, + 10.15680636902107, + 10.2760749910085, + 10.393666607007617, + 10.508720130019356, + 10.625722291006241, + 10.736205224005971, + 10.85731445500278, + 10.976338806009153, + 11.094528471003287, + 11.208846506022383, + 11.327348346007057, + 11.44273257200257, + 11.558957999019185, + 11.673721138009569, + 11.775172063003993 + ], + "chunk_gaps_ms": { + "count": 97, + "p50": 118.06789500406012, + "p95": 124.33440500171855, + "max": 318.2476340152789 + } + }, + { + "start": 170695.471792294, + "first": 170695.976394282, + "last": 170707.383985593, + "end": 170707.384297785, + "elapsed_s": 11.91250549099641, + "chunk_offsets_s": [ + 0.5046019880101085, + 0.6120435080083553, + 0.7258395509852562, + 0.8348925249883905, + 0.944839770992985, + 1.0629136670031585, + 1.18036173901055, + 1.2929848860076163, + 1.4074062569998205, + 1.5228981079999357, + 1.635234216984827, + 1.7521477139962371, + 1.865312941983575, + 1.9848340539901983, + 2.1057929399830755, + 2.2265952989982907, + 2.346691385988379, + 2.4661359069868922, + 2.5843611799937207, + 2.7044697479868773, + 2.8231765890086535, + 2.944128356000874, + 3.0630699330067728, + 3.187409089994617, + 3.304663340008119, + 3.4206640390038956, + 3.5380263699917123, + 3.6538371820061, + 3.7671584759955294, + 3.891497617994901, + 4.015445080003701, + 4.139799566008151, + 4.262276964989724, + 4.383135117008351, + 4.4993173539987765, + 4.61856525199255, + 4.739271213009488, + 4.85528138600057, + 4.973403445008444, + 5.090574497007765, + 5.211247209983412, + 5.330888341006357, + 5.447257889987668, + 5.568978650990175, + 5.683202905987855, + 5.803100933000678, + 5.926607500994578, + 6.044606520998059, + 6.165706819010666, + 6.285248653992312, + 6.4041819460107945, + 6.51885652099736, + 6.6367266940069385, + 6.754833794984734, + 6.865093430009438, + 6.974613047990715, + 7.09157325400156, + 7.214396063995082, + 7.337182424991624, + 7.462534433987457, + 7.585205037001288, + 7.7065561519993935, + 7.8157089879969135, + 7.938293254992459, + 8.059737169998698, + 8.180235453997739, + 8.30165412198403, + 8.413446043996373, + 8.532483738003066, + 8.648294505983358, + 8.767870855983347, + 8.886093206994701, + 9.00471091698273, + 9.116092231997754, + 9.226094348006882, + 9.342748331982875, + 9.459813883993775, + 9.574274173995946, + 9.691181022004457, + 9.803378418000648, + 9.92112579298555, + 10.038960496982327, + 10.157072528993012, + 10.276345199003117, + 10.393939375004265, + 10.508747985993978, + 10.625989522988675, + 10.736476808000589, + 10.857581286982168, + 10.976607046002755, + 11.094797639001627, + 11.209116680984152, + 11.327621240983717, + 11.442733661009697, + 11.559233150997898, + 11.673966178001137, + 11.775095566990785, + 11.852190506993793, + 11.912193299009232 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 117.93459650652949, + "p95": 123.94746200880036, + "max": 125.35200899583288 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 58.9, + "per_stream_tps": 15.2, + "ttft_s_max": 1.17, + "timing_valid": true, + "end_to_end_tps": 58.430742226998426, + "makespan_s": 17.52502126400941, + "steady_tps": 61.7, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 58.93091030880808, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 17.52502126400941, + "end_to_end_tps": 58.430742226998426, + "timing_valid": true, + "steady_tps": 61.7029929997579, + "per_stream_tps": [ + 15.6, + 15.1, + 15.3, + 15.1 + ], + "ttft_s": [ + 1.17, + 0.22, + 0.83, + 0.51 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 170707.386906983, + "first": 170708.561057101, + "last": 170724.91105084, + "end": 170724.911352264, + "elapsed_s": 17.524445280985674, + "chunk_offsets_s": [ + 1.174150117993122, + 1.3358936430013273, + 1.4988163999805693, + 1.6620652379933745, + 1.8228264130011667, + 1.9901057429960929, + 2.163494077976793, + 2.3254030759853777, + 2.4866012439888436, + 2.6528303519880865, + 2.8148426639963873, + 2.985280584980501, + 3.1520797580014914, + 3.3252820939815138, + 3.502770827995846, + 3.6835467759810854, + 3.8568641539895907, + 4.031074650003575, + 4.204813962976914, + 4.380081786977826, + 4.549703247990692, + 4.728881950984942, + 4.906659742991906, + 5.082322639995255, + 5.247054904000834, + 5.412214622978354, + 5.579339046991663, + 5.752773338987026, + 5.922469040990109, + 6.102072544978, + 6.284321262995945, + 6.465415885992115, + 6.6349013169819955, + 6.8067947309755255, + 6.97629152299487, + 7.144930665992433, + 7.316944911988685, + 7.480503115977626, + 7.650907811999787, + 7.821878426999319, + 7.990390500985086, + 8.163158042996656, + 8.33262155597913, + 8.500384109996958, + 8.670316055999137, + 8.842216367978835, + 9.012750856985804, + 9.179887763981242, + 9.354629594978178, + 9.520932630985044, + 9.68819365199306, + 9.852615937998053, + 10.014338273002068, + 10.177987951989053, + 10.332292291976046, + 10.490200261992868, + 10.662677120999433, + 10.838057784974808, + 11.01672891198541, + 11.20056539599318, + 11.373593021999113, + 11.55036408698652, + 11.722732258989708, + 11.8990900600038, + 12.081466689996887, + 12.25614649199997, + 12.430340614984743, + 12.600884004990803, + 12.771133330999874, + 12.941077024996048, + 13.113970973994583, + 13.284003212000243, + 13.450333580985898, + 13.612113424984273, + 13.775194643996656, + 13.94438617999549, + 14.113227155990899, + 14.27132911499939, + 14.428239489992848, + 14.590637846995378, + 14.757685224991292, + 14.929999175976263, + 15.099432936985977, + 15.268649657984497, + 15.43507362899254, + 15.592213605996221, + 15.762272823980311, + 15.931259625998791, + 16.10661069699563, + 16.28256906400202, + 16.45732642299845, + 16.625184028001968, + 16.793632846995024, + 16.957518581999466, + 17.113800463994266, + 17.252147348975996, + 17.37375057800091, + 17.4636705610028, + 17.524143856979208 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 169.4911115046125, + "p95": 180.77594798523933, + "max": 183.8364840077702 + } + }, + { + "start": 170707.386444856, + "first": 170707.602646903, + "last": 170724.500051224, + "end": 170724.500322568, + "elapsed_s": 17.113877711992245, + "chunk_offsets_s": [ + 0.21620204698410816, + 0.2924298199941404, + 0.6132731600082479, + 0.9624664809962269, + 1.33680985000683, + 1.4994332790083718, + 1.6626783079991583, + 1.823720251006307, + 1.9909787020005751, + 2.1643706210015807, + 2.325626147008734, + 2.4868443149898667, + 2.6534979669959284, + 2.8156046780059114, + 2.9860356559802312, + 3.1526566210086457, + 3.326007149007637, + 3.5030820909887552, + 3.683711880003102, + 3.8574593359953724, + 4.032189449004363, + 4.205484826001339, + 4.3808230819995515, + 4.550333694991423, + 4.729570445982972, + 4.9074360609811265, + 5.083237982005812, + 5.247715559002245, + 5.412849485990591, + 5.579905061982572, + 5.752920937986346, + 5.923023759998614, + 6.102240175998304, + 6.284634398005437, + 6.465835164999589, + 6.635198628006037, + 6.807006361981621, + 6.976878193003358, + 7.145137929997873, + 7.317629166995175, + 7.4807779469992965, + 7.651206562994048, + 7.822184682998341, + 7.990651747997617, + 8.163378665980417, + 8.333212498982903, + 8.501029932987876, + 8.670582502993057, + 8.84277155098971, + 9.013039799989201, + 9.180171474989038, + 9.355238921998534, + 9.521197141992161, + 9.688421347003896, + 9.852913968992652, + 10.014628703997005, + 10.178243678994477, + 10.332850706996396, + 10.490497460996266, + 10.663179600000149, + 10.838347431999864, + 11.017086175008444, + 11.201067843008786, + 11.373978268995415, + 11.550527974002762, + 11.72302841799683, + 11.898995578987524, + 12.08175740900333, + 12.256443578982726, + 12.430875557998661, + 12.601189267996233, + 12.77143165000598, + 12.941404976008926, + 13.114279052999336, + 13.284619115001988, + 13.450496492005186, + 13.612397311982932, + 13.775503426993964, + 13.944693875004305, + 14.11343945900444, + 14.271747385995695, + 14.428741217008792, + 14.591178069007583, + 14.758098791993689, + 14.93039703098475, + 15.099852168001235, + 15.269051576993661, + 15.43557911599055, + 15.592580260999966, + 15.76259288698202, + 15.93176040900289, + 16.10703496800852, + 16.282902678998653, + 16.457586150005227, + 16.625533515005372, + 16.793924686004175, + 16.958013124996796, + 17.113606367987813 + ], + "chunk_gaps_ms": { + "count": 97, + "p50": 169.97332600294612, + "p95": 182.76183001580648, + "max": 374.34336901060306 + } + }, + { + "start": 170707.387030951, + "first": 170708.213042627, + "last": 170724.850052777, + "end": 170724.850135401, + "elapsed_s": 17.463104449998355, + "chunk_offsets_s": [ + 0.8260116759920493, + 0.961950833996525, + 1.336055291001685, + 1.498928831977537, + 1.6621661329991184, + 1.823019787989324, + 1.9902936150028836, + 2.163669661997119, + 2.3249389799893834, + 2.4861104599840473, + 2.6530172159837093, + 2.815200886980165, + 2.9851898810011335, + 3.1518992460041773, + 3.3255903979879804, + 3.5022481879859697, + 3.682995144976303, + 3.8566754179773852, + 4.031163001985988, + 4.204637018992798, + 4.380140794994077, + 4.550027407996822, + 4.72862207898288, + 4.9065789750020485, + 5.08214240000234, + 5.2468843440001365, + 5.412034862994915, + 5.578959254984511, + 5.752455770998495, + 5.922278785001254, + 6.101779649005039, + 6.283805326995207, + 6.464852973993402, + 6.634425173979253, + 6.806168202980189, + 6.975799298990751, + 7.144644026993774, + 7.316425295983208, + 7.479864331980934, + 7.650329587981105, + 7.821425692003686, + 7.9899523409840185, + 8.162553564005066, + 8.332141491991933, + 8.499974557984388, + 8.669803400000092, + 8.841772447980475, + 9.012185064988444, + 9.179460596002173, + 9.35435365900048, + 9.520429080002941, + 9.687663667980814, + 9.85210637000273, + 10.013855441997293, + 10.17743815999711, + 10.331951044005109, + 10.489726614003303, + 10.662252224981785, + 10.837601977982558, + 11.016330496000592, + 11.200124131981283, + 11.373276158003137, + 11.549793686979683, + 11.722247475001495, + 11.898781708005117, + 12.080879410001216, + 12.255673179985024, + 12.430011414980981, + 12.600469684985, + 12.770676290994743, + 12.940474242001073, + 13.113574429997243, + 13.283624475996476, + 13.449423452984774, + 13.611618976981845, + 13.7747732999851, + 13.943991491978522, + 14.112699924997287, + 14.271000651002396, + 14.427954081998905, + 14.590439542982494, + 14.75719840900274, + 14.929530631983653, + 15.099097464990336, + 15.268226265994599, + 15.434833356994204, + 15.591781941999216, + 15.761804935988039, + 15.930863433983177, + 16.106295320991194, + 16.282140135997906, + 16.45685698298621, + 16.624746123998193, + 16.793138576002093, + 16.95716549397912, + 17.11288500900264, + 17.25184506099322, + 17.373458401998505, + 17.46302182599902 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 169.7145235084463, + "p95": 181.04764699819498, + "max": 374.10445700515993 + } + }, + { + "start": 170707.386331, + "first": 170707.891997831, + "last": 170724.760689129, + "end": 170724.760783337, + "elapsed_s": 17.374452337011462, + "chunk_offsets_s": [ + 0.5056668310135137, + 0.6137140870268922, + 0.962625233019935, + 1.336544522026088, + 1.499599519011099, + 1.6628372360137291, + 1.823513580020517, + 1.9907857420039363, + 2.1641965890012216, + 2.3255086120043416, + 2.486744523019297, + 2.653657471004408, + 2.815825413999846, + 2.9862146000086796, + 3.1528124770266004, + 3.3259576450218447, + 3.503402779024327, + 3.6840371750004124, + 3.8576926960085984, + 4.032062153011793, + 4.205557498004055, + 4.380896458023926, + 4.550398863008013, + 4.729886654007714, + 4.907608973007882, + 5.083189085999038, + 5.247918214008678, + 5.412920878006844, + 5.580074230005266, + 5.753206474008039, + 5.9228736640070565, + 6.102547472022707, + 6.284936894022394, + 6.465746909001609, + 6.635549876024015, + 6.807031914009713, + 6.976693282020278, + 7.145426346018212, + 7.317351871024584, + 7.480720379011473, + 7.651158963009948, + 7.822493738000048, + 7.990799124003388, + 8.16356413802714, + 8.333061603014357, + 8.501028141006827, + 8.67093155902694, + 8.8426050710259, + 9.013117240014253, + 9.180248851014767, + 9.355386266019195, + 9.521548662014538, + 9.688809491024585, + 9.853230561013333, + 10.01470742400852, + 10.17839319902123, + 10.332995683012996, + 10.490575669013197, + 10.663103296014015, + 10.838536632014439, + 11.01725849500508, + 11.2009853470081, + 11.37423828500323, + 11.550768086017342, + 11.723105937999208, + 11.898910651012557, + 12.081836224999279, + 12.256761003023712, + 12.430957590026082, + 12.601499908021651, + 12.771749186009401, + 12.941481376008596, + 13.114585900999373, + 13.284669163025683, + 13.450318843999412, + 13.612472928012721, + 13.775810467021074, + 13.945001843007049, + 14.113586467021378, + 14.271944970008917, + 14.428769345016917, + 14.59126002099947, + 14.758077608013991, + 14.930428119027056, + 15.100049048021901, + 15.269265529001132, + 15.435724572016625, + 15.592837829026394, + 15.762769271008437, + 15.931685065006604, + 16.107113224017667, + 16.283098807005445, + 16.457735830015736, + 16.625799339002697, + 16.79436836601235, + 16.958161509013735, + 17.113968496007146, + 17.252804644027492, + 17.37435812901822 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 169.69968999910634, + "p95": 182.38942199968733, + "max": 373.9192890061531 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + } + ] +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/rows-without-bodies.jsonl b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/rows-without-bodies.jsonl new file mode 100644 index 0000000000..1ee124b362 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/rows-without-bodies.jsonl @@ -0,0 +1,44 @@ +{"i": 0, "conv": "S1", "turn": 1, "arm": "drafted", "group": "", "history": 7757, "allowed": [0], "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 6.047555812, "prefill": 6.046134564, "wall": 8.068726478988538, "finish": "tool_calls"}, "ok": true, "body_sha256": "c89863023849e1c523580f91d6e3e557e187ee1345ec141afe8843f12f6a48bb", "body_bytes": 29036} +{"i": 1, "conv": "S1", "turn": 1, "arm": "plain", "group": "", "history": 7757, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 6144, "prompt": 7759, "tokens": 61, "ttft": 1.245775049, "prefill": 1.243725769, "wall": 4.647700744011672, "finish": "tool_calls"}, "ok": true, "body_sha256": "990b29e03db5234f33dad6e1fa5929871234fa91c7f57ae49f27a1167d79423a", "body_bytes": 29050} +{"i": 2, "conv": "S1", "turn": 2, "arm": "drafted", "group": "", "history": 15548, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 6144, "prompt": 15550, "tokens": 44, "ttft": 7.227874416, "prefill": 7.224967458, "wall": 8.295700771006523, "finish": "tool_calls"}, "ok": true, "body_sha256": "3030f3e1aca4248ddc3774913c645b201b7bc6edb33e47ebedef2891434a88a5", "body_bytes": 52453} +{"i": 3, "conv": "S1", "turn": 2, "arm": "plain", "group": "", "history": 15548, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 14336, "prompt": 15550, "tokens": 44, "ttft": 1.023516748, "prefill": 1.020054685, "wall": 3.4715008189959917, "finish": "tool_calls"}, "ok": true, "body_sha256": "b6811d3189d1ae811d5f81cdb8f790400d28440a34e1b73f2a7a81489769d4a1", "body_bytes": 52467} +{"i": 4, "conv": "S1", "turn": 3, "arm": "drafted", "group": "", "history": 19458, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 14336, "prompt": 19460, "tokens": 80, "ttft": 4.063079524, "prefill": 4.059537077, "wall": 6.660844339989126, "finish": "tool_calls"}, "ok": true, "body_sha256": "9da958ca324cb77d807ad9ba42f41bbb135d47bc4142bef90bacc958f2e22c9e", "body_bytes": 64230} +{"i": 5, "conv": "S1", "turn": 3, "arm": "plain", "group": "", "history": 19458, "allowed": {"min": 11266, "max": 19458}, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 18432, "prompt": 19460, "tokens": 80, "ttft": 0.9264888, "prefill": 0.922550514, "wall": 5.380387098994106, "finish": "tool_calls"}, "ok": true, "body_sha256": "39a863ce5a23cb6f7909374db5d30c9c9c26bff13c16fe62adfd9780ccfe0df8", "body_bytes": 64244} +{"i": 6, "conv": "S1", "turn": 4, "arm": "drafted", "group": "", "history": 21872, "allowed": {"min": 11266, "max": 19458}, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 18432, "prompt": 21874, "tokens": 148, "ttft": 2.732135937, "prefill": 2.728273218, "wall": 7.868745972024044, "finish": "tool_calls"}, "ok": true, "body_sha256": "a3c446d988de5857c2455e8abd4e64dda25437840cd5ef8fa31475db42204e87", "body_bytes": 76137} +{"i": 7, "conv": "S1", "turn": 4, "arm": "plain", "group": "", "history": 21872, "allowed": {"min": 13680, "max": 21872}, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 20480, "prompt": 21874, "tokens": 148, "ttft": 1.124659329, "prefill": 1.120052946, "wall": 9.389180733996909, "finish": "tool_calls"}, "ok": true, "body_sha256": "e853c96ae111ba87a6a8bfee0d9a791bd4059f5f4b382fd8efc7a85448011220", "body_bytes": 76151} +{"i": 8, "conv": "S1", "turn": 5, "arm": "drafted", "group": "", "history": 26784, "allowed": {"min": 13680, "max": 21872}, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 20480, "prompt": 26786, "tokens": 160, "ttft": 5.104997505, "prefill": 5.099753906, "wall": 10.39790081299725, "finish": "length"}, "ok": true, "body_sha256": "9125338f0509b19bd85a205c72a3b5d5f491710a037369d44cb0111da1010a4d", "body_bytes": 99864} +{"i": 9, "conv": "S1", "turn": 5, "arm": "plain", "group": "", "history": 26784, "allowed": {"min": 18592, "max": 26784}, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 26624, "prompt": 26786, "tokens": 160, "ttft": 0.42686806, "prefill": 0.421169069, "wall": 9.331210053001996, "finish": "length"}, "ok": true, "body_sha256": "f6a2bb40f1670b03676781b0fc481a2f2578a329e650208bd8a9612fdc0f0446", "body_bytes": 99878} +{"i": 10, "conv": "S1", "turn": 6, "arm": "drafted", "group": "", "history": 29291, "allowed": {"min": 18592, "max": 26784}, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 26624, "prompt": 29293, "tokens": 160, "ttft": 2.313441485, "prefill": 2.307361694, "wall": 7.683533178002108, "finish": "length"}, "ok": true, "body_sha256": "979b3fcfc7a0533423064b402ba077df0cc024cbf265caafa1534f25e7fef313", "body_bytes": 112228} +{"i": 11, "conv": "S1", "turn": 6, "arm": "plain", "group": "", "history": 29291, "allowed": {"min": 21099, "max": 29291}, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 28672, "prompt": 29293, "tokens": 160, "ttft": 0.713579061, "prefill": 0.707409973, "wall": 9.64847477900912, "finish": "length"}, "ok": true, "body_sha256": "e091089f70c0886b8f09b6e0dfd3d01eecc55f40e1c2ae2375d48307f583e6d1", "body_bytes": 112242} +{"i": 12, "conv": "S1", "turn": 7, "arm": "drafted", "group": "", "history": 32214, "allowed": {"min": 21099, "max": 29291}, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 28672, "prompt": 32216, "tokens": 160, "ttft": 2.825893639, "prefill": 2.819053943, "wall": 8.442102285014698, "finish": "length"}, "ok": true, "body_sha256": "6a0e3cfe44e411f0fa67b37e7632a57c17f4bfe43b7261be77dffa95caeb38b4", "body_bytes": 124515} +{"i": 13, "conv": "S1", "turn": 7, "arm": "plain", "group": "", "history": 32214, "allowed": {"min": 24022, "max": 32214}, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 30720, "prompt": 32216, "tokens": 160, "ttft": 1.20103659, "prefill": 1.193098413, "wall": 10.156732073024614, "finish": "length"}, "ok": true, "body_sha256": "66758474fdfd9722a714cc2f7da6049b35bec1f51591926f7ce78b98aa67ed14", "body_bytes": 124529} +{"i": 14, "conv": "S1", "turn": 8, "arm": "drafted", "group": "", "history": 35140, "allowed": {"min": 24022, "max": 32214}, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 30720, "prompt": 35142, "tokens": 160, "ttft": 3.722613064, "prefill": 3.714775335, "wall": 9.283397983992472, "finish": "length"}, "ok": true, "body_sha256": "41ea3d382376d4805e41f62081965e40c2099e79df3bc5a66affd16da3827729", "body_bytes": 136847} +{"i": 15, "conv": "S1", "turn": 8, "arm": "plain", "group": "", "history": 35140, "allowed": {"min": 26948, "max": 35140}, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 34816, "prompt": 35142, "tokens": 160, "ttft": 0.55875432, "prefill": 0.550394782, "wall": 9.491786833008518, "finish": "length"}, "ok": true, "body_sha256": "818535227039566804c3dd06547e10fd3f8eaae63f7812338c29a5ce6b30155d", "body_bytes": 136861} +{"i": 16, "conv": "S1", "turn": 9, "arm": "drafted", "group": "", "history": 37858, "allowed": {"min": 26948, "max": 35140}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 34816, "prompt": 37860, "tokens": 160, "ttft": 2.562848435, "prefill": 2.553385089, "wall": 5.2841357169963885, "finish": "length"}, "ok": true, "body_sha256": "a593f947b081d93b91a3270d18d644b28553545788f70fd4193accd9859882d6", "body_bytes": 149173} +{"i": 17, "conv": "S1", "turn": 9, "arm": "plain", "group": "", "history": 37858, "allowed": {"min": 29666, "max": 37858}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 36864, "prompt": 37860, "tokens": 160, "ttft": 0.933882647, "prefill": 0.925092852, "wall": 9.856686992017785, "finish": "length"}, "ok": true, "body_sha256": "c2b6ee13c01c3c91490d367095de69ab581bf131e3df637d00f7d95a9026b64b", "body_bytes": 149187} +{"i": 18, "conv": "X", "turn": 1, "arm": "drafted", "group": "", "history": 2113, "allowed": [0], "got": {"sha": "9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.768296691, "prefill": 1.767687042, "wall": 6.9896358359837905, "finish": "length"}, "ok": true, "body_sha256": "ea7f1fb5d610d52f549d66c8d5bac3876b0c32a1f369f7230266491a9a674737", "body_bytes": 9524} +{"i": 19, "conv": "A", "turn": 1, "arm": "drafted", "group": "", "history": 7410, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 6144, "prompt": 7412, "tokens": 95, "ttft": 1.095696678, "prefill": 1.093878885, "wall": 3.786942471022485, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a153933bfbacdc64dd54d832df32d21139497927a2527fd60217851614cf603", "body_bytes": 28006} +{"i": 20, "conv": "A", "turn": 1, "arm": "plain", "group": "", "history": 7410, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 6144, "prompt": 7412, "tokens": 95, "ttft": 1.049438026, "prefill": 1.047655113, "wall": 6.307847138989018, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a6c1cda2af13b7752b30514bbb91dc76381167e03b207540e31a0f810a462b3", "body_bytes": 28020} +{"i": 21, "conv": "B", "turn": 1, "arm": "drafted", "group": "", "history": 7472, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 6144, "prompt": 7474, "tokens": 72, "ttft": 1.131981135, "prefill": 1.130454509, "wall": 3.0356573299795855, "finish": "tool_calls"}, "ok": true, "body_sha256": "6685678d8326dff3c264f306810b45e76303da9dbc2ff27f9b4eec69f8a6384c", "body_bytes": 27976} +{"i": 22, "conv": "B", "turn": 1, "arm": "plain", "group": "", "history": 7472, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 6144, "prompt": 7474, "tokens": 72, "ttft": 1.084359099, "prefill": 1.081830346, "wall": 5.084812888002489, "finish": "tool_calls"}, "ok": true, "body_sha256": "ac1e48a68c55011216bc0008268ca26a6a788c311372fed6bc6dc9ed62769bf4", "body_bytes": 27990} +{"i": 23, "conv": "A", "turn": 2, "arm": "drafted", "group": "", "history": 12621, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 6144, "prompt": 12623, "tokens": 120, "ttft": 5.181636777, "prefill": 5.178572552, "wall": 9.14657586600515, "finish": "tool_calls"}, "ok": true, "body_sha256": "2a8662abd5d5ea461caf6ddb2bdd02f69c0cee1c40d40e997b6b28b8a371dfa7", "body_bytes": 51797} +{"i": 24, "conv": "A", "turn": 2, "arm": "plain", "group": "", "history": 12621, "allowed": {"min": 4429, "max": 12621}, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 12288, "prompt": 12623, "tokens": 120, "ttft": 0.550844168, "prefill": 0.547213622, "wall": 7.209120162995532, "finish": "tool_calls"}, "ok": true, "body_sha256": "e4bfefdf9fcd99dedc4439686a63627164213dd7a346fdce1b4c28e6088bd888", "body_bytes": 51811} +{"i": 25, "conv": "B", "turn": 2, "arm": "drafted", "group": "", "history": 12718, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 6144, "prompt": 12720, "tokens": 79, "ttft": 5.230536277, "prefill": 5.227394179, "wall": 8.00823299001786, "finish": "tool_calls"}, "ok": true, "body_sha256": "9639c99eab3ccb2dd5acc5878eb2dbf30e0b4d9ff3870280f572f37ea1705757", "body_bytes": 51398} +{"i": 26, "conv": "B", "turn": 2, "arm": "plain", "group": "", "history": 12718, "allowed": {"min": 4526, "max": 12718}, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 12288, "prompt": 12720, "tokens": 79, "ttft": 0.597088443, "prefill": 0.594037929, "wall": 4.954973621992394, "finish": "tool_calls"}, "ok": true, "body_sha256": "662c9ef37adbd864a24e8bd04c2df05b99e6fe2536f1f6e60b549505d65228bb", "body_bytes": 51412} +{"i": 27, "conv": "A", "turn": 3, "arm": "drafted", "group": "", "history": 18061, "allowed": {"min": 4429, "max": 12621}, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 12288, "prompt": 18063, "tokens": 160, "ttft": 4.445921036, "prefill": 4.442511514, "wall": 10.018717103987, "finish": "length"}, "ok": true, "body_sha256": "6b0dbffbcf639e8c4bb6ff1c21339ad472a641a3fe5912f53cfd3845f6c00c7d", "body_bytes": 75423} +{"i": 28, "conv": "A", "turn": 3, "arm": "plain", "group": "", "history": 18061, "allowed": {"min": 9869, "max": 18061}, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 16384, "prompt": 18063, "tokens": 160, "ttft": 1.292750094, "prefill": 1.28821465, "wall": 10.222894193022512, "finish": "length"}, "ok": true, "body_sha256": "e950b53b8aa35c09f62978dd25d2d47744ee333b9a3b28d1d5118084006221bc", "body_bytes": 75437} +{"i": 29, "conv": "B", "turn": 3, "arm": "drafted", "group": "", "history": 15030, "allowed": {"min": 4526, "max": 12718}, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 12288, "prompt": 15032, "tokens": 85, "ttft": 2.320335391, "prefill": 2.316508557, "wall": 5.257798534003086, "finish": "tool_calls"}, "ok": true, "body_sha256": "387f16258f32e82421ee0d1fa7f6d860b80510467fe0f78cf5889a52283ae121", "body_bytes": 63313} +{"i": 30, "conv": "B", "turn": 3, "arm": "plain", "group": "", "history": 15030, "allowed": {"min": 6838, "max": 15030}, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 14336, "prompt": 15032, "tokens": 85, "ttft": 0.740589848, "prefill": 0.736831622, "wall": 5.456716518005123, "finish": "tool_calls"}, "ok": true, "body_sha256": "43cf677afb67316e1fa1fb6284903a85237b172c09a3b1f0194e49649fc671e1", "body_bytes": 63327} +{"i": 31, "conv": "E", "turn": 1, "arm": "drafted", "group": "", "history": 40607, "allowed": {"min": 0, "max": 6611}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 6144, "prompt": 40609, "tokens": 160, "ttft": 26.23328978, "prefill": 26.223367517, "wall": 28.60576085699722, "finish": "length"}, "ok": true, "body_sha256": "43e5cd87284746532e9a3773ce52f5fd4b183215086f72aa8a1a70f905164d10", "body_bytes": 161550} +{"i": 32, "conv": "E", "turn": 1, "arm": "plain", "group": "", "history": 40607, "allowed": {"min": 32415, "max": 40607}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 38912, "prompt": 40609, "tokens": 160, "ttft": 1.342293285, "prefill": 1.332549996, "wall": 10.322872521006502, "finish": "length"}, "ok": true, "body_sha256": "c912d54535554d90eb45f73fc239ccba03afc5525f169e4759acb86d0fed118e", "body_bytes": 161564} +{"i": 33, "conv": "T", "turn": 1, "arm": "drafted", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.136116528, "prefill": 1.135246672, "wall": 6.139433910982916, "finish": "stop"}, "ok": true, "body_sha256": "45cd3d7f3f990fbf5badf7e4179006aece0dac6ff89e43c1a049c9c47e65c0af", "body_bytes": 6672} +{"i": 34, "conv": "T", "turn": 1, "arm": "plain", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.095218551, "prefill": 1.094098421, "wall": 9.611102762981318, "finish": "stop"}, "ok": true, "body_sha256": "17133648c09b8f37c9e3f3f57cf38d2952a766d4633876fb61d87b50e01f6e13", "body_bytes": 6686} +{"i": 35, "conv": "T", "turn": 2, "arm": "drafted", "group": "", "history": 4205, "allowed": [0], "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.598629209, "prefill": 3.596793238, "wall": 3.9083609389781486, "finish": "stop"}, "ok": true, "body_sha256": "697cbd599a80dd68e03e110992a6355933628f9f0b27307c22cb85d2a1282482", "body_bytes": 19059} +{"i": 36, "conv": "T", "turn": 2, "arm": "plain", "group": "", "history": 4205, "allowed": {"min": 1, "max": 4205}, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 4096, "prompt": 4208, "tokens": 12, "ttft": 0.371525858, "prefill": 0.369513072, "wall": 0.9989264660107438, "finish": "stop"}, "ok": true, "body_sha256": "a477a66d50c218bb9084b3eaa0fb91a039270992aee4478d3ab1de5260bdb370", "body_bytes": 19073} +{"i": 37, "conv": "C", "turn": 1, "arm": "cancel", "group": "", "history": 0, "allowed": null, "got": {"cancelled_after": 4.0}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 38, "conv": "C", "turn": 1, "arm": "drafted", "group": "", "history": 9700, "allowed": {"min": 0, "max": 9688}, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 8.212054533, "prefill": 8.126615166, "wall": 10.97376293700654, "finish": "tool_calls"}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 39, "conv": "C", "turn": 1, "arm": "plain", "group": "", "history": 9700, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 8192, "prompt": 9702, "tokens": 82, "ttft": 1.170101222, "prefill": 1.167648132, "wall": 5.736105783988023, "finish": "tool_calls"}, "ok": true, "body_sha256": "a3d8ec39bf405406cdbde28193f74e4e2ac61b2857b32a308870da9fadd5869f", "body_bytes": 43551} +{"i": 40, "conv": "C", "turn": 2, "arm": "drafted", "group": "", "history": 15150, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 8192, "prompt": 15152, "tokens": 160, "ttft": 5.480542989, "prefill": 5.47720017, "wall": 10.60611721299938, "finish": "length"}, "ok": true, "body_sha256": "4f3116872667bac2957a0f1940d91f710b967440db4635b4694a1ea96c9500e9", "body_bytes": 66946} +{"i": 41, "conv": "C", "turn": 2, "arm": "plain", "group": "", "history": 15150, "allowed": {"min": 6958, "max": 15150}, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 14336, "prompt": 15152, "tokens": 160, "ttft": 0.822615193, "prefill": 0.818845508, "wall": 9.72598850299255, "finish": "length"}, "ok": true, "body_sha256": "a84eade01cad94722aa7a3b3947b57a37f6398c8d22095c55468fda150935ede", "body_bytes": 66960} +{"i": 42, "conv": "A", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 20870, "allowed": {"min": 9869, "max": 18061}, "got": {"sha": "eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c", "cached": 16384, "prompt": 20872, "tokens": 160, "ttft": 6.563061939, "prefill": 5.649587684, "wall": 13.213028155005304, "finish": "length"}, "ok": true, "body_sha256": "ff1f54d8050b169edd0333864471bec51688bba9c4ef6b4fb43fbfbb5f2bc453", "body_bytes": 87155} +{"i": 43, "conv": "B", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 17548, "allowed": {"min": 6838, "max": 15030}, "got": {"sha": "fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8", "cached": 14336, "prompt": 17550, "tokens": 160, "ttft": 2.61344746, "prefill": 2.609495199, "wall": 13.28056809300324, "finish": "length"}, "ok": true, "body_sha256": "a3211ca48119825f3883c3ef935e30f14ef6bce3d5681ba0d74b222d03b8ea1f", "body_bytes": 75226} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/server-rank0.log new file mode 100644 index 0000000000..94bafc1450 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/server-rank0.log @@ -0,0 +1,557 @@ +[kvp-serve] 2026-10-23T01:20:31+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native 0c7727776fda6e28 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 4 +host-d:21999:21999 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:21999:21999 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:21999:21999 [0] NCCL INFO cudaDriverVersion 13030 +host-d:21999:21999 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:21999:21999 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:21999:21999 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:21999:21999 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:21999:21999 [0] NCCL INFO Count not open libibverbs.so +host-d:21999:21999 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:21999:21999 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:21999:21999 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:21999:21999 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:21999:21999 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x2a1bd7e0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:21999:21999 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:21999:21999 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:21999:21999 [0] NCCL INFO Initialized NET plugin IB +host-d:21999:21999 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:21999:21999 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:21999:21999 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:21999:21999 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:21999:21999 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:21999:21999 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:21999:21999 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:21999:21999 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:21999:21999 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:21999:21999 [0] NCCL INFO Using network IB +host-d:21999:21999 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:21999:21999 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x2758b7a0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:21999:21999 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:21999:21999 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:21999:21999 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:21999:21999 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:21999:21999 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:21999:21999 [0] NCCL INFO Rank 0: 1 Net devices +host-d:21999:21999 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:21999:21999 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:21999:21999 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:21999:21999 [0] NCCL INFO comm 0x2758b7a0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:21999:21999 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:21999:21999 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:21999:21999 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:21999:21999 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:21999:21999 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:21999:21999 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:21999:21999 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:21999:21999 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:21999:21999 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:21999:21999 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:21999:22019 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-d:21999:22018 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-d:21999:22018 [0] NCCL INFO proxy listening socket at <41867> +host-d:21999:21999 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21999:21999 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21999:21999 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21999:21999 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:21999:22020 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-d:21999:21999 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21999:21999 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21999:21999 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21999:21999 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf62ec4010000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf62ec4054000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf62ec409c000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf62ec40e0000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf62ec4128000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf62ec416c000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf62ec41b4000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf62ec41f8000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1926 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1927 pkey=0 pd=0xf62ec4053eb0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1928 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1928 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a6563 ctsFifoLkey=0x1a6563 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1928 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1929 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1930 pkey=0 pd=0xf62ec4053eb0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1931 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1931 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a1513 ctsFifoLkey=0x1a1513 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1931 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1932 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1932 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ac9c6 ctsFifoLkey=0x1ac9c6 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1932 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1933 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1933 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1badaa ctsFifoLkey=0x1badaa +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1933 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1928 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1931 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1934 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1935 pkey=0 pd=0xf62ec4053eb0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1932 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1936 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1937 pkey=0 pd=0xf62ec4053eb0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1933 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:21999 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:21999:21999 [0] NCCL INFO Connected all trees +host-d:21999:21999 [0] NCCL INFO Connected binomial trees +host-d:21999:21999 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:21999:21999 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:21999:21999 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:21999:21999 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 93.06 GiB; 4 streams at once, the engine's own and 3 more taking 803 pages of 512 rows (5.00 GiB) as they need them (enough for one at a 131072-token window), 180 MiB fixed each, of 9.5 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 4.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 304.7s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-bc0c6df9b055c3c31d658ade8d1e2fe6 prompt=14 cached=0 thinking=False effort=none tokens=16 sha=cedaf3b614aa finish=length rounds=6 accepted=9/13 ttft=0.148 decode=0.443 elapsed=0.591 queued=0.000 tok/s=33.8 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-345c48bb1f3b5da33cb3eab52b9da992 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.497 elapsed=6.644 queued=0.000 tok/s=39.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-dc4d46e28507eb0ee12a1d7fd468604a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.146 decode=8.524 elapsed=8.670 queued=0.000 tok/s=29.9 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 4, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-a694a3b229d02595cb55e28d6400a673 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.145 decode=7.263 elapsed=7.408 queued=0.000 tok/s=35.1 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 5, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-dd597b0dc1c456ae435d7bd442a40df3 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=77 accepted=178/221 ttft=0.146 decode=6.130 elapsed=6.276 queued=0.000 tok/s=41.6 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-465f03d17dd59e638a2510a67fbdc0d5 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=255 accepted=0/0 ttft=0.152 decode=14.025 elapsed=14.177 queued=0.000 tok/s=18.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-520912205af8becaedef5b88d7a94255 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=255 accepted=0/0 ttft=0.138 decode=13.846 elapsed=13.984 queued=0.000 tok/s=18.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 8, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-42f559c14ff6e57a8ca4f2a3a5d2dcf4 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=255 accepted=0/0 ttft=0.136 decode=13.860 elapsed=13.996 queued=0.000 tok/s=18.4 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 9, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-47cc74e52f500668efd0bd4351661a43 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=255 accepted=0/0 ttft=0.136 decode=13.904 elapsed=14.039 queued=0.000 tok/s=18.3 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 10, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-5655f69900e04a446e0c59ecad8b576a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.522 elapsed=6.667 queued=0.000 tok/s=39.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-6829476bcf7e3ef6f59f67423365561c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=78 accepted=178/229 ttft=0.361 decode=9.325 elapsed=9.685 queued=0.000 tok/s=27.3 seed=1234 +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-bf3af15d4cc56c378328cc4f8ac5ea0a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=103 accepted=152/276 ttft=0.145 decode=11.267 elapsed=11.412 queued=0.000 tok/s=22.6 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 13, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-3a5cba6a4b5cba3de5fbca2e7a804ddb prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=78 accepted=178/228 ttft=0.146 decode=14.339 elapsed=14.484 queued=0.000 tok/s=17.8 seed=1234 +[tensorfold] done cmpl-6440cc959799a73383ad584829c15574 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=80 accepted=175/238 ttft=0.891 decode=14.342 elapsed=15.232 queued=0.218 tok/s=17.8 seed=1237 +[tensorfold] done cmpl-31c223ac9e941adb0ab737ca9003dd4a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=87 accepted=168/258 ttft=0.362 decode=15.458 elapsed=15.820 queued=0.218 tok/s=16.5 seed=1236 +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-e20028ec508f1cb178a75ead5bd9d460 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=99 accepted=156/287 ttft=0.607 decode=16.227 elapsed=16.834 queued=0.218 tok/s=15.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 17, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-80cd7e3fc8b60ab71c3f7ada01f57298 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=3f46b826a241 finish=length rounds=10 accepted=5/13 ttft=0.214 decode=0.674 elapsed=0.887 queued=0.000 tok/s=22.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 18, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-dca5174ec6d15b632943171cb923ec9d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.214 decode=8.702 elapsed=8.916 queued=0.000 tok/s=29.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 19, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-b1c494bfa605b5d49a0f3014ac670dfc prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.215 decode=8.994 elapsed=9.208 queued=0.000 tok/s=28.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-fcf4810c195aa32206f0c3ce6b3c070c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.214 decode=8.956 elapsed=9.169 queued=0.000 tok/s=28.5 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 21, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-83c2894c69f390a3bda3acb1fe41d30b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=143 accepted=113/184 ttft=0.215 decode=9.658 elapsed=9.873 queued=0.000 tok/s=26.4 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 22, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-77ad7d4c52a9b2a88a897e61d8cec020 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=255 accepted=0/0 ttft=0.204 decode=13.946 elapsed=14.150 queued=0.000 tok/s=18.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 23, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-34eeda9236d042c57a970400ad2b6c78 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=255 accepted=0/0 ttft=0.205 decode=13.948 elapsed=14.153 queued=0.000 tok/s=18.3 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-f8c64de85b41cbe93941d0780b74775b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=255 accepted=0/0 ttft=0.205 decode=13.945 elapsed=14.149 queued=0.000 tok/s=18.3 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 25, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-10e7d81b2bd9882e8a413ceaafd9614b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=255 accepted=0/0 ttft=0.205 decode=13.948 elapsed=14.153 queued=0.000 tok/s=18.3 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 26, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-feafd6f4babcc6df0e251921c801bb31 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.215 decode=8.753 elapsed=8.968 queued=0.000 tok/s=29.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 28, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 28, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-7a76c203246bb90a6b96fae256ca78b9 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=102 accepted=154/300 ttft=0.216 decode=12.362 elapsed=12.578 queued=0.000 tok/s=20.6 seed=1234 +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-7f0c2aa081ea4998fa5bd5b6061a2a2f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=105 accepted=150/308 ttft=0.507 decode=12.339 elapsed=12.846 queued=0.000 tok/s=20.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 29, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-8c3746108476df08019e2355b25b003f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=102 accepted=154/301 ttft=0.505 decode=18.530 elapsed=19.035 queued=0.293 tok/s=13.8 seed=1234 +[tensorfold] done chatcmpl-e3c79efbc8c1f73b947ab580661eaa4b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=103 accepted=152/306 ttft=0.826 decode=18.491 elapsed=19.317 queued=0.292 tok/s=13.8 seed=1236 +[tensorfold] done chatcmpl-e3f3b3a38653d90ff81c1257477ec343 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=106 accepted=149/314 ttft=0.216 decode=19.206 elapsed=19.422 queued=0.000 tok/s=13.3 seed=1235 +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-b3d89f5803099150cb30a13a4287c65c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=115 accepted=141/325 ttft=1.180 decode=19.051 elapsed=20.231 queued=0.292 tok/s=13.4 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 33, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-8324d3252c057d6758e0eff603374f4b prompt=14 cached=0 thinking=False effort=none tokens=16 sha=3000ccd48624 finish=length rounds=5 accepted=10/11 ttft=0.146 decode=0.378 elapsed=0.524 queued=0.000 tok/s=39.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 34, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-d83faf4f33bedf365b6dadf50032e260 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.146 decode=4.544 elapsed=4.690 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 35, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-416de49e88537e145cca16c0bef04557 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=255 accepted=0/0 ttft=0.137 decode=13.896 elapsed=14.033 queued=0.000 tok/s=18.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 36, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-c1524f141a6070e609dd701158c84b1e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.545 elapsed=4.690 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 37, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 38, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-02d62655b6f90968d5965ced9beaa1b3 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.146 decode=7.889 elapsed=8.035 queued=0.000 tok/s=32.3 +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-a8ae38708b05807070632f11080cbc53 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.363 decode=7.746 elapsed=8.110 queued=0.219 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 40, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-847ce99c82f88a77afe7b2c4cb007cce prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.145 decode=11.402 elapsed=11.547 queued=0.000 tok/s=22.4 +[tensorfold] done cmpl-f0da9f9c01af86833fe61620ae4e0eb4 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.363 decode=11.319 elapsed=11.682 queued=0.000 tok/s=22.5 +[tensorfold] done cmpl-fa2ae894ed281ac72e516a408458e6a1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.611 decode=11.179 elapsed=11.791 queued=0.218 tok/s=22.8 +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done cmpl-27eb0a0cd5895e54620b17c6acc2ed54 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.889 decode=10.976 elapsed=11.865 queued=0.218 tok/s=23.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 43, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-67078a4f63bf6e3798162e13c14aaf76 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=daface6f9795 finish=length rounds=6 accepted=10/12 ttft=0.214 decode=0.447 elapsed=0.661 queued=0.000 tok/s=33.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 44, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-3235f4e72b7960e6ff13afa8bd0091a5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.385 elapsed=8.600 queued=0.000 tok/s=30.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 45, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-e44573e2b36ffabd9b36e759204b8874 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=255 accepted=0/0 ttft=0.204 decode=13.906 elapsed=14.109 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 46, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-0a06c76c375a7babb1c2fd3536a4389d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.215 decode=8.350 elapsed=8.565 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-6f0b4e881f6cde536e59586156de29b4 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=97 accepted=158/287 ttft=0.215 decode=11.559 elapsed=11.774 queued=0.000 tok/s=22.1 +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-fb93bd6a5b1f6f24a6b7ecedf7a588f4 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.504 decode=11.408 elapsed=11.911 queued=0.000 tok/s=22.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 49, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0ff3a6a18673b572b025568d4f7e9555 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=97 accepted=158/287 ttft=0.215 decode=16.898 elapsed=17.112 queued=0.000 tok/s=15.1 +[tensorfold] done chatcmpl-c75ce2ce9d2ba859d56edd45b9773d73 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.503 decode=16.869 elapsed=17.372 queued=0.290 tok/s=15.1 +[tensorfold] done chatcmpl-05be63d8199586436eb2efd1829fc4c6 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.824 decode=16.637 elapsed=17.462 queued=0.290 tok/s=15.3 +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-0cba8e9ab15a001cefe04cb56c108ba3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=1.172 decode=16.350 elapsed=17.522 queued=0.289 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 0, kept 2; 2 states, 490 MiB and 3606 MiB spare of 4096 MiB (hits 0, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-3e929850162b27cf10be28b472798d29 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=28 accepted=33/46 ttft=6.048 decode=2.020 elapsed=8.067 queued=0.000 tok/s=29.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 6144, kept 0; 2 states, 490 MiB and 3606 MiB spare of 4096 MiB (hits 1, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-2abed5b7c7261ca72ba671557ad2c1f4 prompt=7759 cached=6144 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=1.246 decode=3.400 elapsed=4.646 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 6144, kept 1; 3 states, 850 MiB and 3246 MiB spare of 4096 MiB (hits 2, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-8b17373f3e3a1217341b299c33a8a0eb prompt=15550 cached=6144 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=13 accepted=30/36 ttft=7.228 decode=1.067 elapsed=8.294 queued=0.000 tok/s=40.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 14336, kept 0; 3 states, 850 MiB and 3246 MiB spare of 4096 MiB (hits 3, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-52c90edc91a4262129345678b6a8e239 prompt=15550 cached=14336 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=43 accepted=0/0 ttft=1.024 decode=2.446 elapsed=3.470 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 19460 tokens, resumed at 14336, kept 1; 4 states, 1260 MiB and 2836 MiB spare of 4096 MiB (hits 4, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-e6ad9afa1cafd5edf313f0293142e0e3 prompt=19460 cached=14336 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=35 accepted=45/68 ttft=4.063 decode=2.596 elapsed=6.659 queued=0.000 tok/s=30.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 19460 tokens, resumed at 18432, kept 0; 4 states, 1260 MiB and 2836 MiB spare of 4096 MiB (hits 5, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-fff3359472572b0ab8353a41684075f6 prompt=19460 cached=18432 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=79 accepted=0/0 ttft=0.926 decode=4.452 elapsed=5.379 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 21874 tokens, resumed at 18432, kept 1; 5 states, 1696 MiB and 2400 MiB spare of 4096 MiB (hits 6, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-2769044d10732885d884461e9bdbf218 prompt=21874 cached=18432 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=70 accepted=78/124 ttft=2.732 decode=5.135 elapsed=7.867 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 21874 tokens, resumed at 20480, kept 0; 5 states, 1696 MiB and 2400 MiB spare of 4096 MiB (hits 7, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-0cd56c9f32036deb7c321075bddd79e7 prompt=21874 cached=20480 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=147 accepted=0/0 ttft=1.125 decode=8.263 elapsed=9.387 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 26786 tokens, resumed at 20480, kept 1; 6 states, 2208 MiB and 1888 MiB spare of 4096 MiB (hits 8, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ba497b6f6a44d137907fd6755933c2ce prompt=26786 cached=20480 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=72 accepted=88/126 ttft=5.105 decode=5.291 elapsed=10.396 queued=0.000 tok/s=30.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 26786 tokens, resumed at 26624, kept 0; 6 states, 2208 MiB and 1888 MiB spare of 4096 MiB (hits 9, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-26f252866050478ace0b5eee238d6ddc prompt=26786 cached=26624 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=159 accepted=0/0 ttft=0.427 decode=8.902 elapsed=9.329 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 29293 tokens, resumed at 26624, kept 1; 7 states, 2746 MiB and 1350 MiB spare of 4096 MiB (hits 10, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-e1c61a02e8bced3ceaf96e225b2b465f prompt=29293 cached=26624 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=73 accepted=86/129 ttft=2.313 decode=5.368 elapsed=7.682 queued=0.000 tok/s=29.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 29293 tokens, resumed at 28672, kept 0; 7 states, 2746 MiB and 1350 MiB spare of 4096 MiB (hits 11, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-116fbcacd946e19d983d4509281e34ca prompt=29293 cached=28672 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=159 accepted=0/0 ttft=0.714 decode=8.933 elapsed=9.646 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 32216 tokens, resumed at 28672, kept 1; 8 states, 3310 MiB and 786 MiB spare of 4096 MiB (hits 12, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-b9f70fae6c7638db2b923d64860faf7a prompt=32216 cached=28672 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=78 accepted=82/126 ttft=2.826 decode=5.614 elapsed=8.440 queued=0.000 tok/s=28.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32216 tokens, resumed at 30720, kept 0; 8 states, 3310 MiB and 786 MiB spare of 4096 MiB (hits 13, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-a28bd0a8efdd1ea5b40ea584d822edc1 prompt=32216 cached=30720 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=159 accepted=0/0 ttft=1.201 decode=8.953 elapsed=10.154 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 35142 tokens, resumed at 30720, kept 1; 9 states, 3924 MiB and 172 MiB spare of 4096 MiB (hits 14, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-3d867ce9611e9d6dfc20c40e92f6f51a prompt=35142 cached=30720 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=78 accepted=82/128 ttft=3.723 decode=5.559 elapsed=9.282 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 35142 tokens, resumed at 34816, kept 0; 9 states, 3924 MiB and 172 MiB spare of 4096 MiB (hits 15, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-6cd0c5d39710530bd402adb0afbf3f7d prompt=35142 cached=34816 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=159 accepted=0/0 ttft=0.559 decode=8.931 elapsed=9.489 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 37860 tokens, resumed at 34816, kept 1; 8 states, 3794 MiB and 302 MiB spare of 4096 MiB (hits 16, misses 53, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ec7074655d557a5559de8b6bc675e49b prompt=37860 cached=34816 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=29 accepted=130/135 ttft=2.563 decode=2.719 elapsed=5.282 queued=0.000 tok/s=58.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 37860 tokens, resumed at 36864, kept 0; 8 states, 3794 MiB and 302 MiB spare of 4096 MiB (hits 17, misses 53, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-2dcfb9c5d19bbd80f1f1d46487ba5c05 prompt=37860 cached=36864 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=0.934 decode=8.921 elapsed=9.854 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 2115 tokens, resumed at 0, kept 1; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 17, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-fce9c81893e1c8a189e3b8047838cffc prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=9761cff1a208 finish=length rounds=70 accepted=89/135 ttft=1.768 decode=5.220 elapsed=6.988 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 18, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-1b5f5c5f9303c03384ca30917748a1b5 prompt=7412 cached=6144 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=34 accepted=61/80 ttft=1.096 decode=2.690 elapsed=3.786 queued=0.000 tok/s=34.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 19, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-030d810ee0eb75c126bfd0361b9b6dcc prompt=7412 cached=6144 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=94 accepted=0/0 ttft=1.049 decode=5.257 elapsed=6.306 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 20, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-97b9f3306863979690c6dbabc9c8435d prompt=7474 cached=6144 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=23 accepted=49/62 ttft=1.132 decode=1.902 elapsed=3.034 queued=0.000 tok/s=37.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 21, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-5fb6c2efd06cf0e2441d43e279604c14 prompt=7474 cached=6144 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=71 accepted=0/0 ttft=1.084 decode=3.999 elapsed=5.084 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12623 tokens, resumed at 6144, kept 1; 9 states, 3886 MiB and 210 MiB spare of 4096 MiB (hits 22, misses 54, evicted 3, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-26be93e6b84e992cb903f38c98dac1b6 prompt=12623 cached=6144 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=56 accepted=64/89 ttft=5.182 decode=3.963 elapsed=9.145 queued=0.000 tok/s=30.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12623 tokens, resumed at 12288, kept 0; 9 states, 3886 MiB and 210 MiB spare of 4096 MiB (hits 23, misses 54, evicted 3, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-854de2dc75acea11ba3ddf1c244c821a prompt=12623 cached=12288 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=119 accepted=0/0 ttft=0.551 decode=6.656 elapsed=7.207 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12720 tokens, resumed at 6144, kept 1; 9 states, 3708 MiB and 388 MiB spare of 4096 MiB (hits 24, misses 54, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-2665e13e1b830d80cdda1bc71820155f prompt=12720 cached=6144 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=40 accepted=39/56 ttft=5.231 decode=2.776 elapsed=8.007 queued=0.000 tok/s=28.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12720 tokens, resumed at 12288, kept 0; 9 states, 3708 MiB and 388 MiB spare of 4096 MiB (hits 25, misses 54, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-5f02de9492609bb518cdd89cfa14b659 prompt=12720 cached=12288 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=78 accepted=0/0 ttft=0.597 decode=4.356 elapsed=4.953 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 18063 tokens, resumed at 12288, kept 1; 10 states, 4092 MiB and 4 MiB spare of 4096 MiB (hits 26, misses 54, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-8b9713ce1b630a79a4fd499a741a1eb6 prompt=18063 cached=12288 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=78 accepted=82/123 ttft=4.446 decode=5.571 elapsed=10.017 queued=0.000 tok/s=28.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 18063 tokens, resumed at 16384, kept 0; 10 states, 4092 MiB and 4 MiB spare of 4096 MiB (hits 27, misses 54, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-2f44bc160cc409d89b8c2b96fe8fbfdb prompt=18063 cached=16384 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=159 accepted=0/0 ttft=1.293 decode=8.928 elapsed=10.221 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15032 tokens, resumed at 12288, kept 1; 10 states, 3914 MiB and 182 MiB spare of 4096 MiB (hits 28, misses 54, evicted 5, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-1c3143c162e7368474f6b04d20fdbae0 prompt=15032 cached=12288 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=42 accepted=43/62 ttft=2.320 decode=2.936 elapsed=5.256 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15032 tokens, resumed at 14336, kept 0; 10 states, 3914 MiB and 182 MiB spare of 4096 MiB (hits 29, misses 54, evicted 5, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-d011674b32a62d036e6fa9bd77bb6b3c prompt=15032 cached=14336 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=84 accepted=0/0 ttft=0.741 decode=4.714 elapsed=5.455 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 40609 tokens, resumed at 6144, kept 1; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 30, misses 54, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-62587d761cbfc8d5552389ee6a82db2d prompt=40609 cached=6144 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=23 accepted=136/138 ttft=26.233 decode=2.371 elapsed=28.604 queued=0.000 tok/s=67.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 40609 tokens, resumed at 38912, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 54, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-39434b6bf7dd090c8cd86ff538b507e6 prompt=40609 cached=38912 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=1.342 decode=8.978 elapsed=10.320 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 55, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-457d80cfdd76af88dee7df3c145baa04 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=68 accepted=88/131 ttft=1.136 decode=5.002 elapsed=6.138 queued=0.000 tok/s=31.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 56, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-7beb674f60fc4f98cb5c0e7b572157dc prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=155 accepted=0/0 ttft=1.095 decode=8.515 elapsed=9.610 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 4208 tokens, resumed at 0, kept 1; 10 states, 3634 MiB and 462 MiB spare of 4096 MiB (hits 31, misses 57, evicted 7, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-aacf192397cb3a45ca60840f395efc88 prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.599 decode=0.308 elapsed=3.907 queued=0.000 tok/s=35.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 4208 tokens, resumed at 4096, kept 0; 10 states, 3634 MiB and 462 MiB spare of 4096 MiB (hits 32, misses 57, evicted 7, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-4f388f667918f37f2d574badbbb172f8 prompt=4208 cached=4096 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=0.372 decode=0.626 elapsed=0.998 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] ended chatcmpl-b0828e7db9ee8de7b0d78122ef49fd90 reason=client-left prompt=9702 tokens=0 after=4.11s +info: prompt cache: 9702 tokens, resumed at 0, kept 2; 11 states, 3840 MiB and 256 MiB spare of 4096 MiB (hits 32, misses 59, evicted 8, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-cb511ca61e3129ad5fb9c72072954f2b prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=39 accepted=43/59 ttft=8.212 decode=2.760 elapsed=10.972 queued=0.083 tok/s=29.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 9702 tokens, resumed at 8192, kept 0; 11 states, 3840 MiB and 256 MiB spare of 4096 MiB (hits 33, misses 59, evicted 8, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] done chatcmpl-3ee0e14f93690ed72ac6907a3e9993c3 prompt=9702 cached=8192 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=81 accepted=0/0 ttft=1.170 decode=4.565 elapsed=5.735 queued=0.000 tok/s=17.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15152 tokens, resumed at 8192, kept 1; 11 states, 3866 MiB and 230 MiB spare of 4096 MiB (hits 34, misses 59, evicted 9, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-f7c98bde0449d2975226aec7fb9a72f0 prompt=15152 cached=8192 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=71 accepted=88/117 ttft=5.481 decode=5.124 elapsed=10.605 queued=0.000 tok/s=31.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15152 tokens, resumed at 14336, kept 0; 11 states, 3866 MiB and 230 MiB spare of 4096 MiB (hits 35, misses 59, evicted 9, refused 0, failed 0) +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-aec97252051bf807ea38adc44774ca7d prompt=15152 cached=14336 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=159 accepted=0/0 ttft=0.823 decode=8.902 elapsed=9.724 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 17550 tokens, resumed at 14336, kept 1; 11 states, 4018 MiB and 78 MiB spare of 4096 MiB (hits 36, misses 59, evicted 10, refused 0, failed 0) +info: prompt cache: 20872 tokens, resumed at 16384, kept 2; 11 states, 4094 MiB and 2 MiB spare of 4096 MiB (hits 37, misses 59, evicted 11, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-46f2c4ea4c92acad2284b81588fc288c prompt=20872 cached=16384 thinking=True effort=high tokens=160 sha=eab19e6ff8fd finish=length rounds=54 accepted=105/161 ttft=6.563 decode=6.648 elapsed=13.211 queued=0.909 tok/s=23.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 803 + store 0 == total 803 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-02320129d9ed4a4548cd62410f895a4f prompt=17550 cached=14336 thinking=True effort=high tokens=160 sha=fe9efe1d4302 finish=length rounds=58 accepted=102/167 ttft=2.613 decode=10.665 elapsed=13.279 queued=0.000 tok/s=14.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:21999:21999 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:21999:21999 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:21999:21999 [0] NCCL INFO ncclCommInitRank comm 0x2758b7a0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:21999:21999 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.42 (kernels 0.26, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.12, rest 0.00) +host-d:21999:21999 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:21999:21999 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf62ec4445000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf62ec4486000) +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1938 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1939 pkey=0 pd=0xf62ec4053eb0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1940 pkey=0 pd=0xf62ec4053eb0 oooRq=0 +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1940 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ba29f ctsFifoLkey=0x1ba29f +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1940 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:22018 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1940 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:21999:21999 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/server-rank1.log new file mode 100644 index 0000000000..0d8aaaac2f --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2all/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T01:20:30+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native 0c7727776fda6e28 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 4 +host-e:2141:2141 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2141:2141 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2141:2141 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2141:2141 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2141:2141 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2141:2141 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2141:2141 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2141:2141 [0] NCCL INFO Count not open libibverbs.so +host-e:2141:2141 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2141:2141 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2141:2141 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2141:2141 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2141:2141 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x45df2620 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2141:2141 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2141:2141 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2141:2141 [0] NCCL INFO Initialized NET plugin IB +host-e:2141:2141 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2141:2141 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2141:2141 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2141:2141 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2141:2141 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2141:2141 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2141:2141 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2141:2141 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2141:2141 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2141:2141 [0] NCCL INFO Using network IB +host-e:2141:2141 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2141:2141 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x431c07c0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2141:2141 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2141:2141 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2141:2141 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2141:2141 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2141:2141 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2141:2141 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2141:2141 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2141:2141 [0] NCCL INFO comm 0x431c07c0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2141:2141 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2141:2141 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2141:2141 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2141:2141 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2141:2141 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2141:2141 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2141:2153 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-e:2141:2153 [0] NCCL INFO proxy listening socket at <43161> +host-e:2141:2154 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 11 +host-e:2141:2141 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2141:2141 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2141:2141 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2141:2155 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-e:2141:2141 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2141:2141 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2141:2141 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2141:2141 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2141:2141 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff07d8010000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff07d8054000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff07d809c000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1670 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1670 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1670 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1671 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1671 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x166320 ctsFifoLkey=0x166320 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1671 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff07d814e000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff07d8196000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff07d81da000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff07d8222000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff07d8266000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1670 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1672 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1673 pkey=0 pd=0xff07d8053590 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1674 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1674 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17b06d ctsFifoLkey=0x17b06d +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1674 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1675 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1676 pkey=0 pd=0xff07d8053590 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1677 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1677 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x174502 ctsFifoLkey=0x174502 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1677 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1678 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1679 pkey=0 pd=0xff07d8053590 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1680 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1681 pkey=0 pd=0xff07d8053590 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1671 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1674 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1677 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2141 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2141:2141 [0] NCCL INFO Connected all trees +host-e:2141:2141 [0] NCCL INFO Connected binomial trees +host-e:2141:2141 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2141:2141 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2141:2141 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2141:2141 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2141:2141 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2141:2141 [0] NCCL INFO ncclCommInitRank comm 0x431c07c0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2141:2141 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.42 (kernels 0.24, alloc 0.03, bootstrap 0.03, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.12, rest 0.00) +host-e:2141:2141 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Sharinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +ed +host-e:2141:2141 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff07d8450000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff07d8491000) +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1682 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1682 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1802bf ctsFifoLkey=0x1802bf +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1682 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1682 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1683 pkey=0 pd=0xff07d8053590 oooRq=0 +host-e:2141:2153 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1684 pkey=0 pd=0xff07d8053590 +host-e:2141:2141 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/client.log new file mode 100644 index 0000000000..3c75a0302e --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/client.log @@ -0,0 +1,45 @@ +server up 2026-10-23 11:05:57 + 0 S1 t1 drafted prompt 7759 cached 0 ttft 5.98s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 1 S1 t1 plain prompt 7759 cached 0 ttft 5.94s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 2 S1 t2 drafted prompt 15550 cached 0 ttft 11.87s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 + 3 S1 t2 plain prompt 15550 cached 0 ttft 11.86s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 + 4 S1 t3 drafted prompt 19460 cached 0 ttft 15.17s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 + 5 S1 t3 plain prompt 19460 cached 0 ttft 15.10s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 + 6 S1 t4 drafted prompt 21874 cached 0 ttft 16.85s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 + 7 S1 t4 plain prompt 21874 cached 0 ttft 16.85s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 + 8 S1 t5 drafted prompt 26786 cached 0 ttft 20.81s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c + 9 S1 t5 plain prompt 26786 cached 0 ttft 20.53s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c + 10 S1 t6 drafted prompt 29293 cached 0 ttft 22.38s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 + 11 S1 t6 plain prompt 29293 cached 0 ttft 22.37s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 + 12 S1 t7 drafted prompt 32216 cached 0 ttft 24.45s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e + 13 S1 t7 plain prompt 32216 cached 0 ttft 24.38s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e + 14 S1 t8 drafted prompt 35142 cached 0 ttft 26.89s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 + 15 S1 t8 plain prompt 35142 cached 0 ttft 26.86s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 + 16 S1 t9 drafted prompt 37860 cached 0 ttft 28.83s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 17 S1 t9 plain prompt 37860 cached 0 ttft 28.81s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 18 X t1 drafted prompt 2115 cached 0 ttft 1.76s sha 9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd + 19 A t1 drafted prompt 7412 cached 0 ttft 5.79s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 + 20 A t1 plain prompt 7412 cached 0 ttft 5.76s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 + 21 B t1 drafted prompt 7474 cached 0 ttft 5.84s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a + 22 B t1 plain prompt 7474 cached 0 ttft 5.79s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a + 23 A t2 drafted prompt 12623 cached 0 ttft 9.86s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 + 24 A t2 plain prompt 12623 cached 0 ttft 9.86s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 + 25 B t2 drafted prompt 12720 cached 0 ttft 9.93s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c + 26 B t2 plain prompt 12720 cached 0 ttft 9.90s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c + 27 A t3 drafted prompt 18063 cached 0 ttft 13.73s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc + 28 A t3 plain prompt 18063 cached 0 ttft 13.69s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc + 29 B t3 drafted prompt 15032 cached 0 ttft 11.60s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa + 30 B t3 plain prompt 15032 cached 0 ttft 11.57s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa + 31 E t1 drafted prompt 40609 cached 0 ttft 30.81s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 32 E t1 plain prompt 40609 cached 0 ttft 30.77s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 33 T t1 drafted prompt 1520 cached 0 ttft 1.14s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 + 34 T t1 plain prompt 1520 cached 0 ttft 1.10s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 + 35 T t2 drafted prompt 4208 cached 0 ttft 3.58s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 36 T t2 plain prompt 4208 cached 0 ttft 3.56s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 38 C t1 drafted prompt 9702 cached 0 ttft 7.46s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 + 39 C t1 plain prompt 9702 cached 0 ttft 7.40s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 + 40 C t2 drafted prompt 15152 cached 0 ttft 11.69s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 + 41 C t2 plain prompt 15152 cached 0 ttft 11.66s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 + 42 A t4 drafted prompt 20872 cached 0 ttft 30.54s sha eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c + 43 B t4 drafted prompt 17550 cached 0 ttft 13.45s sha fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8 +=== kvp-client r2auto rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/fresh.jsonl b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/fresh.jsonl new file mode 100644 index 0000000000..ac3b2bfe57 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/fresh.jsonl @@ -0,0 +1,43 @@ +{"i": 0, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.984090404, "prefill": 5.978912482, "wall": 8.005074482993223, "finish": "tool_calls"}} +{"i": 1, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.938037266, "prefill": 5.936473352, "wall": 9.296283559000585, "finish": "tool_calls"}} +{"i": 2, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 0, "prompt": 15550, "tokens": 44, "ttft": 11.86781066, "prefill": 11.863809355, "wall": 12.929991206008708, "finish": "tool_calls"}} +{"i": 3, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 0, "prompt": 15550, "tokens": 44, "ttft": 11.858223945, "prefill": 11.855935082, "wall": 14.277693955024006, "finish": "tool_calls"}} +{"i": 4, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 0, "prompt": 19460, "tokens": 80, "ttft": 15.166698738, "prefill": 15.164170274, "wall": 17.824139886011835, "finish": "tool_calls"}} +{"i": 5, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 0, "prompt": 19460, "tokens": 80, "ttft": 15.095493702, "prefill": 15.093115975, "wall": 19.62794748498709, "finish": "tool_calls"}} +{"i": 6, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 0, "prompt": 21874, "tokens": 148, "ttft": 16.849633649, "prefill": 16.845350808, "wall": 22.13820195000153, "finish": "tool_calls"}} +{"i": 7, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 0, "prompt": 21874, "tokens": 148, "ttft": 16.846997058, "prefill": 16.843865056, "wall": 25.353281667019473, "finish": "tool_calls"}} +{"i": 8, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 0, "prompt": 26786, "tokens": 160, "ttft": 20.811906339, "prefill": 20.806210481, "wall": 26.09721311798785, "finish": "length"}} +{"i": 9, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 0, "prompt": 26786, "tokens": 160, "ttft": 20.534354359, "prefill": 20.530642593, "wall": 29.369869871996343, "finish": "length"}} +{"i": 10, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 0, "prompt": 29293, "tokens": 160, "ttft": 22.376227942, "prefill": 22.370141236, "wall": 27.729063394013792, "finish": "length"}} +{"i": 11, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 0, "prompt": 29293, "tokens": 160, "ttft": 22.370201816, "prefill": 22.36556155, "wall": 31.228896635991987, "finish": "length"}} +{"i": 12, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 0, "prompt": 32216, "tokens": 160, "ttft": 24.454420429, "prefill": 24.447268214, "wall": 30.058980555011658, "finish": "length"}} +{"i": 13, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 0, "prompt": 32216, "tokens": 160, "ttft": 24.380423516, "prefill": 24.375145871, "wall": 33.298186659027124, "finish": "length"}} +{"i": 14, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 0, "prompt": 35142, "tokens": 160, "ttft": 26.887939044, "prefill": 26.881068048, "wall": 32.4310892749927, "finish": "length"}} +{"i": 15, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 0, "prompt": 35142, "tokens": 160, "ttft": 26.856805968, "prefill": 26.851395523, "wall": 35.71393566500046, "finish": "length"}} +{"i": 16, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 37860, "tokens": 160, "ttft": 28.826593836, "prefill": 28.81850013, "wall": 31.539591145992745, "finish": "length"}} +{"i": 17, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 37860, "tokens": 160, "ttft": 28.812659417, "prefill": 28.80613316, "wall": 37.731250907003414, "finish": "length"}} +{"i": 18, "got": {"sha": "9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.756637934, "prefill": 1.754571428, "wall": 6.976849082013359, "finish": "length"}} +{"i": 19, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 0, "prompt": 7412, "tokens": 95, "ttft": 5.790881681, "prefill": 5.788634294, "wall": 8.477557703998173, "finish": "tool_calls"}} +{"i": 20, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 0, "prompt": 7412, "tokens": 95, "ttft": 5.759927557, "prefill": 5.758358444, "wall": 11.008075324993115, "finish": "tool_calls"}} +{"i": 21, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 0, "prompt": 7474, "tokens": 72, "ttft": 5.842472095, "prefill": 5.840378035, "wall": 7.745596574008232, "finish": "tool_calls"}} +{"i": 22, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 0, "prompt": 7474, "tokens": 72, "ttft": 5.791040168, "prefill": 5.789874674, "wall": 9.75541304302169, "finish": "tool_calls"}} +{"i": 23, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 0, "prompt": 12623, "tokens": 120, "ttft": 9.859310522, "prefill": 9.855826553, "wall": 13.81142160098534, "finish": "tool_calls"}} +{"i": 24, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 0, "prompt": 12623, "tokens": 120, "ttft": 9.85723453, "prefill": 9.854875974, "wall": 16.480016143003013, "finish": "tool_calls"}} +{"i": 25, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 0, "prompt": 12720, "tokens": 79, "ttft": 9.927223775, "prefill": 9.924009231, "wall": 12.684954163996736, "finish": "tool_calls"}} +{"i": 26, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 0, "prompt": 12720, "tokens": 79, "ttft": 9.89914057, "prefill": 9.896831295, "wall": 14.253472626995062, "finish": "tool_calls"}} +{"i": 27, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 0, "prompt": 18063, "tokens": 160, "ttft": 13.728104224, "prefill": 13.723552122, "wall": 19.291420816007303, "finish": "length"}} +{"i": 28, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 0, "prompt": 18063, "tokens": 160, "ttft": 13.685211949, "prefill": 13.682288575, "wall": 22.579421768023167, "finish": "length"}} +{"i": 29, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 0, "prompt": 15032, "tokens": 85, "ttft": 11.604019548, "prefill": 11.600249658, "wall": 14.532118183007697, "finish": "tool_calls"}} +{"i": 30, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 0, "prompt": 15032, "tokens": 85, "ttft": 11.573820331, "prefill": 11.571656769, "wall": 16.244663532997947, "finish": "tool_calls"}} +{"i": 31, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 40609, "tokens": 160, "ttft": 30.81248579, "prefill": 30.803259059, "wall": 33.175757516990416, "finish": "length"}} +{"i": 32, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 40609, "tokens": 160, "ttft": 30.768794484, "prefill": 30.762211397, "wall": 39.67683417801163, "finish": "length"}} +{"i": 33, "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.139363979, "prefill": 1.137747621, "wall": 6.107359255984193, "finish": "stop"}} +{"i": 34, "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.09647395, "prefill": 1.095625978, "wall": 9.560968227975536, "finish": "stop"}} +{"i": 35, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.582237881, "prefill": 3.579394652, "wall": 3.8909900470171124, "finish": "stop"}} +{"i": 36, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.563903076, "prefill": 3.562050604, "wall": 4.190598534012679, "finish": "stop"}} +{"i": 38, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 7.464113972, "prefill": 7.458297273, "wall": 10.206775508006103, "finish": "tool_calls"}} +{"i": 39, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 7.403247107, "prefill": 7.401360459, "wall": 11.946995311998762, "finish": "tool_calls"}} +{"i": 40, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 0, "prompt": 15152, "tokens": 160, "ttft": 11.69126701, "prefill": 11.687394112, "wall": 16.800499706005212, "finish": "length"}} +{"i": 41, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 0, "prompt": 15152, "tokens": 160, "ttft": 11.662982706, "prefill": 11.660532455, "wall": 20.52391352399718, "finish": "length"}} +{"i": 42, "got": {"sha": "eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c", "cached": 0, "prompt": 20872, "tokens": 160, "ttft": 30.539433006, "prefill": 29.698019169, "wall": 36.99724469400826, "finish": "length"}} +{"i": 43, "got": {"sha": "fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8", "cached": 0, "prompt": 17550, "tokens": 160, "ttft": 13.451075415, "prefill": 13.446887492, "wall": 36.62121066299733, "finish": "length"}} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/server-rank0.log new file mode 100644 index 0000000000..df36d03ceb --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/server-rank0.log @@ -0,0 +1,231 @@ +[kvp-serve] 2026-10-23T02:00:48+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native 0c7727776fda6e28 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:23101:23101 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:23101:23101 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:23101:23101 [0] NCCL INFO cudaDriverVersion 13030 +host-d:23101:23101 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:23101:23101 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:23101:23101 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:23101:23101 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:23101:23101 [0] NCCL INFO Count not open libibverbs.so +host-d:23101:23101 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:23101:23101 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:23101:23101 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:23101:23101 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:23101:23101 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x865a9d0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:23101:23101 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:23101:23101 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:23101:23101 [0] NCCL INFO Initialized NET plugin IB +host-d:23101:23101 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:23101:23101 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:23101:23101 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:23101:23101 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:23101:23101 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:23101:23101 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:23101:23101 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:23101:23101 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:23101:23101 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:23101:23101 [0] NCCL INFO Using network IB +host-d:23101:23101 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:23101:23101 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x5a28970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:23101:23101 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:23101:23101 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:23101:23101 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:23101:23101 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:23101:23101 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:23101:23101 [0] NCCL INFO Rank 0: 1 Net devices +host-d:23101:23101 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:23101:23101 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:23101:23101 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:23101:23101 [0] NCCL INFO comm 0x5a28970 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:23101:23101 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:23101:23101 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:23101:23101 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:23101:23101 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:23101:23101 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:23101:23101 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:23101:23101 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:23101:23101 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:23101:23101 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:23101:23101 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:23101:23121 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 1 +host-d:23101:23120 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-d:23101:23120 [0] NCCL INFO proxy listening socket at <38183> +host-d:23101:23122 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-d:23101:23101 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:23101:23101 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:23101:23101 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:23101:23101 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:23101:23101 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:23101:23101 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:23101:23101 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:23101:23101 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe66a34010000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe66a34054000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe66a3409c000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1958 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1958 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5e5c ctsFifoLkey=0x1b5e5c +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1958 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1959 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1959 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b0200 ctsFifoLkey=0x1b0200 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1959 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe66a3414e000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe66a34196000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe66a341da000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe66a34222000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe66a34266000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1958 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1959 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1960 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1961 pkey=0 pd=0xe66a34053590 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1962 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1962 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9391 ctsFifoLkey=0x1b9391 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1962 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1963 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1964 pkey=0 pd=0xe66a34053590 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1965 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1965 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9e9b ctsFifoLkey=0x1b9e9b +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1965 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1966 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1967 pkey=0 pd=0xe66a34053590 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1968 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1969 pkey=0 pd=0xe66a34053590 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1962 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1965 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23101 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:23101:23101 [0] NCCL INFO Connected all trees +host-d:23101:23101 [0] NCCL INFO Connected binomial trees +host-d:23101:23101 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:23101:23101 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:23101:23101 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:23101:23101 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.7 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 306.1s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b7bc04a508813558bf64a74061f7f896 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=28 accepted=33/46 ttft=5.984 decode=1.995 elapsed=7.979 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9224cc867e278a06f5b0a952abeb4994 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=5.938 decode=3.356 elapsed=9.294 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-be5966aa0db7336073769d8819dbb876 prompt=15550 cached=0 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=13 accepted=30/36 ttft=11.868 decode=1.061 elapsed=12.928 queued=0.000 tok/s=40.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-503d80e238240842e2f4df9ff56070d2 prompt=15550 cached=0 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=43 accepted=0/0 ttft=11.858 decode=2.418 elapsed=14.276 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-150e4f3227144e15ca96dd499995de6a prompt=19460 cached=0 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=35 accepted=45/68 ttft=15.167 decode=2.655 elapsed=17.822 queued=0.000 tok/s=29.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ad64080a6d33289073ed1cf2da1f3b4d prompt=19460 cached=0 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=79 accepted=0/0 ttft=15.095 decode=4.531 elapsed=19.626 queued=0.000 tok/s=17.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-09242c065aaba63fdf28e8c99adb7cc9 prompt=21874 cached=0 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=70 accepted=78/124 ttft=16.850 decode=5.286 elapsed=22.136 queued=0.000 tok/s=27.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-61acbeb4f01397730cfa68c6086e8461 prompt=21874 cached=0 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=147 accepted=0/0 ttft=16.847 decode=8.504 elapsed=25.351 queued=0.000 tok/s=17.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-dc5b9778a94f7f13fd87c8a187274e51 prompt=26786 cached=0 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=72 accepted=88/126 ttft=20.812 decode=5.283 elapsed=26.095 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-8398c4b50ae3af577cb8aaea39270de0 prompt=26786 cached=0 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=159 accepted=0/0 ttft=20.534 decode=8.833 elapsed=29.367 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-7904d0b9ea13b24b1d187ef6168e2b80 prompt=29293 cached=0 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=73 accepted=86/129 ttft=22.376 decode=5.351 elapsed=27.727 queued=0.000 tok/s=29.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-871cd6434d55270b14bcc5e273d7bfa8 prompt=29293 cached=0 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=159 accepted=0/0 ttft=22.370 decode=8.856 elapsed=31.227 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-14160f5d37df0942037ca650f10ea32e prompt=32216 cached=0 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=78 accepted=82/126 ttft=24.454 decode=5.603 elapsed=30.057 queued=0.000 tok/s=28.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-000fc2b68d0ead8a9120b5c0c98fccae prompt=32216 cached=0 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=159 accepted=0/0 ttft=24.380 decode=8.915 elapsed=33.296 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bedeee0fbb261d79a11cafbbe726b8ec prompt=35142 cached=0 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=78 accepted=82/128 ttft=26.888 decode=5.541 elapsed=32.429 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-90888ccab56a47657925f8e5e765f4d1 prompt=35142 cached=0 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=159 accepted=0/0 ttft=26.857 decode=8.855 elapsed=35.712 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-899fd6928333e2f04173c77729de32b4 prompt=37860 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=29 accepted=130/135 ttft=28.827 decode=2.711 elapsed=31.537 queued=0.000 tok/s=58.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-7e22eed55b0ac4c181431b5ddd24c2f7 prompt=37860 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=28.813 decode=8.916 elapsed=37.729 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d36c24219f9c900936c4835f5d125ef6 prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=9761cff1a208 finish=length rounds=70 accepted=89/135 ttft=1.757 decode=5.219 elapsed=6.975 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ef78c6ba46f6c68e740748174250cace prompt=7412 cached=0 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=34 accepted=61/80 ttft=5.791 decode=2.685 elapsed=8.475 queued=0.000 tok/s=35.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-21d00b624f2211996ab97f7980522999 prompt=7412 cached=0 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=94 accepted=0/0 ttft=5.760 decode=5.246 elapsed=11.006 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-269d686f372ae7181f1f9399548797a5 prompt=7474 cached=0 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=23 accepted=49/62 ttft=5.842 decode=1.901 elapsed=7.744 queued=0.000 tok/s=37.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-97f866839fe3a62a7227819a9e269471 prompt=7474 cached=0 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=71 accepted=0/0 ttft=5.791 decode=3.963 elapsed=9.754 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-abafeacf855da0dbe56c62992366b077 prompt=12623 cached=0 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=56 accepted=64/89 ttft=9.859 decode=3.950 elapsed=13.809 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-852ab59640d149d1672dff33bfc1c668 prompt=12623 cached=0 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=119 accepted=0/0 ttft=9.857 decode=6.621 elapsed=16.478 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-21c4dd054e524b2586f577e76083a4fa prompt=12720 cached=0 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=40 accepted=39/56 ttft=9.927 decode=2.756 elapsed=12.683 queued=0.000 tok/s=28.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d276f9cb63fea36e3fd5bf7f236a01f2 prompt=12720 cached=0 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=78 accepted=0/0 ttft=9.899 decode=4.353 elapsed=14.252 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6f469a297b492cdb9c4f5967a30e9505 prompt=18063 cached=0 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=78 accepted=82/123 ttft=13.728 decode=5.562 elapsed=19.290 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-927b651437ccde43e3b9ed299660579b prompt=18063 cached=0 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=159 accepted=0/0 ttft=13.685 decode=8.892 elapsed=22.577 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5e9d3007f85a68e26ba4ca7c2b036419 prompt=15032 cached=0 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=42 accepted=43/62 ttft=11.604 decode=2.926 elapsed=14.530 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-57805b53d21610f39b41340048d37db5 prompt=15032 cached=0 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=84 accepted=0/0 ttft=11.574 decode=4.669 elapsed=16.243 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-644a14a47cfb3b7aea65b1ef2880a7dc prompt=40609 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=23 accepted=136/138 ttft=30.812 decode=2.361 elapsed=33.173 queued=0.000 tok/s=67.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-8387df1e75cf9f8136b29ab673cfe2fc prompt=40609 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=30.769 decode=8.906 elapsed=39.674 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d43c2b095b66a9c294968ed23a4ee32a prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=68 accepted=88/131 ttft=1.139 decode=4.967 elapsed=6.106 queued=0.000 tok/s=31.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-daccca28e38442d03b1df94dab2a47ab prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=155 accepted=0/0 ttft=1.096 decode=8.463 elapsed=9.559 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-8e61b5d477e3f40467fe968d6b50734c prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.582 decode=0.307 elapsed=3.889 queued=0.000 tok/s=35.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d8270740f5e4a6695d4f5cfcc3a4533d prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=3.564 decode=0.625 elapsed=4.189 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9a100901bccd0aaba59477551b47ec2c prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=39 accepted=43/59 ttft=7.464 decode=2.741 elapsed=10.205 queued=0.000 tok/s=29.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4b5e6724f2f5ca447d494ad2378d169e prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=81 accepted=0/0 ttft=7.403 decode=4.542 elapsed=11.945 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-f60def5ed16e5fe30197ee0b6246bfd9 prompt=15152 cached=0 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=71 accepted=88/117 ttft=11.691 decode=5.108 elapsed=16.799 queued=0.000 tok/s=31.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-fa2919f8c2013c083197861225ce7c68 prompt=15152 cached=0 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=159 accepted=0/0 ttft=11.663 decode=8.859 elapsed=20.522 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bd99d34c7af461a1f39a2e16f99b939b prompt=17550 cached=0 thinking=True effort=high tokens=160 sha=fe9efe1d4302 finish=length rounds=61 accepted=99/167 ttft=13.451 decode=23.169 elapsed=36.620 queued=0.000 tok/s=6.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-52d77847d5a61437bbd4c90bc41867a7 prompt=20872 cached=0 thinking=True effort=high tokens=160 sha=eab19e6ff8fd finish=length rounds=54 accepted=105/161 ttft=30.539 decode=6.456 elapsed=36.995 queued=0.837 tok/s=24.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +channels per peer +host-d:23101:23101 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:23101:23101 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:23101:23101 [0] NCCL INFO ncclCommInitRank comm 0x5a28970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:23101:23101 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.39 (kernels 0.26, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-d:23101:23101 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:23101:23101 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe66a3444c000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe66a3448d000) +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1970 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1970 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b7e7c ctsFifoLkey=0x1b7e7c +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1970 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1971 pkey=0 pd=0xe66a34053590 oooRq=0 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1972 pkey=0 pd=0xe66a34053590 +host-d:23101:23120 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1970 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23101:23101 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/server-rank1.log new file mode 100644 index 0000000000..5a15e2ba63 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2auto/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T02:00:47+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native 0c7727776fda6e28 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:2224:2224 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2224:2224 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2224:2224 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2224:2224 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2224:2224 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2224:2224 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2224:2224 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2224:2224 [0] NCCL INFO Count not open libibverbs.so +host-e:2224:2224 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2224:2224 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2224:2224 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2224:2224 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2224:2224 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x3a22e830 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2224:2224 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2224:2224 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2224:2224 [0] NCCL INFO Initialized NET plugin IB +host-e:2224:2224 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2224:2224 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2224:2224 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2224:2224 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2224:2224 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2224:2224 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2224:2224 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2224:2224 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2224:2224 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2224:2224 [0] NCCL INFO Using network IB +host-e:2224:2224 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2224:2224 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x375fca00 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2224:2224 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2224:2224 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2224:2224 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2224:2224 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2224:2224 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2224:2224 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2224:2224 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2224:2224 [0] NCCL INFO comm 0x375fca00 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2224:2224 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2224:2224 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2224:2224 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2224:2224 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2224:2224 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2224:2224 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2224:2236 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-e:2224:2236 [0] NCCL INFO proxy listening socket at <57067> +host-e:2224:2237 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 1 +host-e:2224:2238 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-e:2224:2224 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2224:2224 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2224:2224 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2224:2224 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2224:2224 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2224:2224 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2224:2224 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2224:2224 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6fdf8010000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6fdf8054000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6fdf809c000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6fdf80e0000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6fdf8128000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6fdf816c000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6fdf81b4000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6fdf81f8000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1702 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1703 pkey=0 pd=0xe6fdf8053eb0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1704 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1704 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x168542 ctsFifoLkey=0x168542 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1704 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1705 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1706 pkey=0 pd=0xe6fdf8053eb0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1707 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1707 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1727e5 ctsFifoLkey=0x1727e5 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1707 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1708 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1708 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17ca87 ctsFifoLkey=0x17ca87 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1708 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1709 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1709 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1709 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1710 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1711 pkey=0 pd=0xe6fdf8053eb0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1712 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1713 pkey=0 pd=0xe6fdf8053eb0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1704 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1707 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1708 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1709 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2224 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2224:2224 [0] NCCL INFO Connected all trees +host-e:2224:2224 [0] NCCL INFO Connected binomial trees +host-e:2224:2224 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2224:2224 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2224:2224 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2224:2224 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2224:2224 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2224:2224 [0] NCCL INFO ncclCommInitRank comm 0x375fca00 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2224:2224 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.39 (kernels 0.24, alloc 0.03, bootstrap 0.02, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-e:2224:2224 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:2224:2224 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe6fdf844d000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe6fdf848e000) +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1714 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1714 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x171cdc ctsFifoLkey=0x171cdc +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1714 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1715 pkey=0 pd=0xe6fdf8053eb0 oooRq=0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1716 pkey=0 pd=0xe6fdf8053eb0 +host-e:2224:2236 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1714 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2224:2224 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/client.log new file mode 100644 index 0000000000..fc3b0a02bd --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/client.log @@ -0,0 +1,45 @@ +server up 2026-10-23 11:27:59 + 0 S1 t1 drafted prompt 7759 cached 0 ttft 5.99s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 1 S1 t1 plain prompt 7759 cached 0 ttft 5.93s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 2 S1 t2 drafted prompt 15550 cached 0 ttft 11.89s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 + 3 S1 t2 plain prompt 15550 cached 0 ttft 11.86s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 + 4 S1 t3 drafted prompt 19460 cached 0 ttft 14.88s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 + 5 S1 t3 plain prompt 19460 cached 0 ttft 14.85s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 + 6 S1 t4 drafted prompt 21874 cached 0 ttft 16.66s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 + 7 S1 t4 plain prompt 21874 cached 0 ttft 16.59s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 + 8 S1 t5 drafted prompt 26786 cached 0 ttft 20.59s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c + 9 S1 t5 plain prompt 26786 cached 0 ttft 20.54s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c + 10 S1 t6 drafted prompt 29293 cached 0 ttft 22.41s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 + 11 S1 t6 plain prompt 29293 cached 0 ttft 22.38s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 + 12 S1 t7 drafted prompt 32216 cached 0 ttft 24.48s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e + 13 S1 t7 plain prompt 32216 cached 0 ttft 24.42s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e + 14 S1 t8 drafted prompt 35142 cached 0 ttft 26.92s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 + 15 S1 t8 plain prompt 35142 cached 0 ttft 26.88s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 + 16 S1 t9 drafted prompt 37860 cached 0 ttft 28.86s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 17 S1 t9 plain prompt 37860 cached 0 ttft 28.82s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 18 X t1 drafted prompt 2115 cached 0 ttft 1.75s sha 9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd + 19 A t1 drafted prompt 7412 cached 0 ttft 5.81s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 + 20 A t1 plain prompt 7412 cached 0 ttft 5.76s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 + 21 B t1 drafted prompt 7474 cached 0 ttft 5.84s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a + 22 B t1 plain prompt 7474 cached 0 ttft 5.79s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a + 23 A t2 drafted prompt 12623 cached 0 ttft 9.87s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 + 24 A t2 plain prompt 12623 cached 0 ttft 9.85s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 + 25 B t2 drafted prompt 12720 cached 0 ttft 9.93s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c + 26 B t2 plain prompt 12720 cached 0 ttft 9.90s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c + 27 A t3 drafted prompt 18063 cached 0 ttft 13.73s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc + 28 A t3 plain prompt 18063 cached 0 ttft 13.67s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc + 29 B t3 drafted prompt 15032 cached 0 ttft 11.62s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa + 30 B t3 plain prompt 15032 cached 0 ttft 11.59s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa + 31 E t1 drafted prompt 40609 cached 0 ttft 30.83s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 32 E t1 plain prompt 40609 cached 0 ttft 30.79s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 + 33 T t1 drafted prompt 1520 cached 0 ttft 1.13s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 + 34 T t1 plain prompt 1520 cached 0 ttft 1.10s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 + 35 T t2 drafted prompt 4208 cached 0 ttft 3.59s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 36 T t2 plain prompt 4208 cached 0 ttft 3.57s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 38 C t1 drafted prompt 9702 cached 0 ttft 7.47s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 + 39 C t1 plain prompt 9702 cached 0 ttft 7.41s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 + 40 C t2 drafted prompt 15152 cached 0 ttft 11.70s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 + 41 C t2 plain prompt 15152 cached 0 ttft 11.66s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 + 42 A t4 drafted prompt 20872 cached 0 ttft 30.51s sha eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c + 43 B t4 drafted prompt 17550 cached 0 ttft 13.43s sha fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8 +=== kvp-client r2main rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/fresh.jsonl b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/fresh.jsonl new file mode 100644 index 0000000000..f84ddedbcb --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/fresh.jsonl @@ -0,0 +1,43 @@ +{"i": 0, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.989206049, "prefill": 5.983504618, "wall": 7.996610875998158, "finish": "tool_calls"}} +{"i": 1, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.932241237, "prefill": 5.930855774, "wall": 9.286991111002862, "finish": "tool_calls"}} +{"i": 2, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 0, "prompt": 15550, "tokens": 44, "ttft": 11.886244302, "prefill": 11.882238702, "wall": 12.942273571010446, "finish": "tool_calls"}} +{"i": 3, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 0, "prompt": 15550, "tokens": 44, "ttft": 11.863816907, "prefill": 11.861841475, "wall": 14.27770419899025, "finish": "tool_calls"}} +{"i": 4, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 0, "prompt": 19460, "tokens": 80, "ttft": 14.876259585, "prefill": 14.87395388, "wall": 17.448479498998495, "finish": "tool_calls"}} +{"i": 5, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 0, "prompt": 19460, "tokens": 80, "ttft": 14.845286316, "prefill": 14.842853842, "wall": 19.244887530978303, "finish": "tool_calls"}} +{"i": 6, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 0, "prompt": 21874, "tokens": 148, "ttft": 16.658858419, "prefill": 16.654466688, "wall": 21.779355137987295, "finish": "tool_calls"}} +{"i": 7, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 0, "prompt": 21874, "tokens": 148, "ttft": 16.593732088, "prefill": 16.590744636, "wall": 24.768035824003164, "finish": "tool_calls"}} +{"i": 8, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 0, "prompt": 26786, "tokens": 160, "ttft": 20.593688052, "prefill": 20.58858187, "wall": 25.85566813798505, "finish": "length"}} +{"i": 9, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 0, "prompt": 26786, "tokens": 160, "ttft": 20.542179741, "prefill": 20.538096972, "wall": 29.38142584101297, "finish": "length"}} +{"i": 10, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 0, "prompt": 29293, "tokens": 160, "ttft": 22.414023387, "prefill": 22.408410066, "wall": 27.77785983099602, "finish": "length"}} +{"i": 11, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 0, "prompt": 29293, "tokens": 160, "ttft": 22.378624887, "prefill": 22.373990208, "wall": 31.253461212007096, "finish": "length"}} +{"i": 12, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 0, "prompt": 32216, "tokens": 160, "ttft": 24.482575615, "prefill": 24.47625613, "wall": 30.052383314003237, "finish": "length"}} +{"i": 13, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 0, "prompt": 32216, "tokens": 160, "ttft": 24.415388543, "prefill": 24.409918113, "wall": 33.30487061900203, "finish": "length"}} +{"i": 14, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 0, "prompt": 35142, "tokens": 160, "ttft": 26.916774721, "prefill": 26.909723042, "wall": 32.471124370000325, "finish": "length"}} +{"i": 15, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 0, "prompt": 35142, "tokens": 160, "ttft": 26.877116474, "prefill": 26.871852118, "wall": 35.710441720002564, "finish": "length"}} +{"i": 16, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 37860, "tokens": 160, "ttft": 28.855191285, "prefill": 28.847515621, "wall": 31.55793217700557, "finish": "length"}} +{"i": 17, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 37860, "tokens": 160, "ttft": 28.81807663, "prefill": 28.812000764, "wall": 37.67942901697825, "finish": "length"}} +{"i": 18, "got": {"sha": "9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.754948973, "prefill": 1.753127992, "wall": 6.948391943995375, "finish": "length"}} +{"i": 19, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 0, "prompt": 7412, "tokens": 95, "ttft": 5.811443747, "prefill": 5.80979568, "wall": 8.484807602013461, "finish": "tool_calls"}} +{"i": 20, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 0, "prompt": 7412, "tokens": 95, "ttft": 5.758569016, "prefill": 5.756917893, "wall": 10.994421090988908, "finish": "tool_calls"}} +{"i": 21, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 0, "prompt": 7474, "tokens": 72, "ttft": 5.838878897, "prefill": 5.837021516, "wall": 7.732773625000846, "finish": "tool_calls"}} +{"i": 22, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 0, "prompt": 7474, "tokens": 72, "ttft": 5.786408285, "prefill": 5.785288561, "wall": 9.745030347985448, "finish": "tool_calls"}} +{"i": 23, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 0, "prompt": 12623, "tokens": 120, "ttft": 9.870438899, "prefill": 9.866896348, "wall": 13.825707876007073, "finish": "tool_calls"}} +{"i": 24, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 0, "prompt": 12623, "tokens": 120, "ttft": 9.853347488, "prefill": 9.850976294, "wall": 16.466449444997124, "finish": "tool_calls"}} +{"i": 25, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 0, "prompt": 12720, "tokens": 79, "ttft": 9.931735837, "prefill": 9.928253463, "wall": 12.692468999011908, "finish": "tool_calls"}} +{"i": 26, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 0, "prompt": 12720, "tokens": 79, "ttft": 9.895832587, "prefill": 9.893773237, "wall": 14.238042729004519, "finish": "tool_calls"}} +{"i": 27, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 0, "prompt": 18063, "tokens": 160, "ttft": 13.72994168, "prefill": 13.725350704, "wall": 19.28460808499949, "finish": "length"}} +{"i": 28, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 0, "prompt": 18063, "tokens": 160, "ttft": 13.671346273, "prefill": 13.668360226, "wall": 22.52422770098201, "finish": "length"}} +{"i": 29, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 0, "prompt": 15032, "tokens": 85, "ttft": 11.617535345, "prefill": 11.61413003, "wall": 14.530272336996859, "finish": "tool_calls"}} +{"i": 30, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 0, "prompt": 15032, "tokens": 85, "ttft": 11.586762252, "prefill": 11.584302948, "wall": 16.265589593996992, "finish": "tool_calls"}} +{"i": 31, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 40609, "tokens": 160, "ttft": 30.828448454, "prefill": 30.81960208, "wall": 33.186741896002786, "finish": "length"}} +{"i": 32, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 0, "prompt": 40609, "tokens": 160, "ttft": 30.79020689, "prefill": 30.783507306, "wall": 39.70769581999048, "finish": "length"}} +{"i": 33, "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.132640001, "prefill": 1.130787904, "wall": 6.107498015015153, "finish": "stop"}} +{"i": 34, "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.097842848, "prefill": 1.096952007, "wall": 9.5489910079923, "finish": "stop"}} +{"i": 35, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.588282943, "prefill": 3.58559628, "wall": 3.894290441996418, "finish": "stop"}} +{"i": 36, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.57088174, "prefill": 3.569616065, "wall": 4.196104358998127, "finish": "stop"}} +{"i": 38, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 7.466299416, "prefill": 7.460522468, "wall": 10.208875961980084, "finish": "tool_calls"}} +{"i": 39, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 7.406260528, "prefill": 7.404830307, "wall": 11.92120364200673, "finish": "tool_calls"}} +{"i": 40, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 0, "prompt": 15152, "tokens": 160, "ttft": 11.698800508, "prefill": 11.694956378, "wall": 16.780807373987045, "finish": "length"}} +{"i": 41, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 0, "prompt": 15152, "tokens": 160, "ttft": 11.656237942, "prefill": 11.653742208, "wall": 20.490924093988724, "finish": "length"}} +{"i": 42, "got": {"sha": "eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c", "cached": 0, "prompt": 20872, "tokens": 160, "ttft": 30.513507685, "prefill": 29.674177327, "wall": 36.986549480003305, "finish": "length"}} +{"i": 43, "got": {"sha": "fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8", "cached": 0, "prompt": 17550, "tokens": 160, "ttft": 13.434435439, "prefill": 13.429990585, "wall": 36.6104875059973, "finish": "length"}} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/server-rank0.log new file mode 100644 index 0000000000..db64e3cb64 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/server-rank0.log @@ -0,0 +1,231 @@ +[kvp-serve] 2026-10-23T02:22:51+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:23550:23550 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:23550:23550 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:23550:23550 [0] NCCL INFO cudaDriverVersion 13030 +host-d:23550:23550 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:23550:23550 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:23550:23550 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:23550:23550 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:23550:23550 [0] NCCL INFO Count not open libibverbs.so +host-d:23550:23550 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:23550:23550 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:23550:23550 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:23550:23550 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:23550:23550 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x219729d0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:23550:23550 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:23550:23550 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:23550:23550 [0] NCCL INFO Initialized NET plugin IB +host-d:23550:23550 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:23550:23550 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:23550:23550 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:23550:23550 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:23550:23550 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:23550:23550 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:23550:23550 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:23550:23550 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:23550:23550 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:23550:23550 [0] NCCL INFO Using network IB +host-d:23550:23550 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:23550:23550 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x1ed40970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:23550:23550 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:23550:23550 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:23550:23550 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:23550:23550 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:23550:23550 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:23550:23550 [0] NCCL INFO Rank 0: 1 Net devices +host-d:23550:23550 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:23550:23550 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:23550:23550 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:23550:23550 [0] NCCL INFO comm 0x1ed40970 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:23550:23550 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:23550:23550 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:23550:23550 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:23550:23550 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:23550:23550 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:23550:23550 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:23550:23550 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:23550:23550 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:23550:23550 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:23550:23550 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:23550:23570 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 10 +host-d:23550:23569 [0] NCCL INFO [Proxy Service] Device 0 CPU core 0 +host-d:23550:23569 [0] NCCL INFO proxy listening socket at <46761> +host-d:23550:23571 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-d:23550:23550 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:23550:23550 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:23550:23550 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:23550:23550 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:23550:23550 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:23550:23550 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:23550:23550 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:23550:23550 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf28060010000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf28060054000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf2806009c000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1974 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1974 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1badaa ctsFifoLkey=0x1badaa +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1974 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1975 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1975 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b7e7c ctsFifoLkey=0x1b7e7c +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1975 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf2806014e000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf28060196000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf280601da000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf28060222000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf28060266000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1976 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1977 pkey=0 pd=0xf28060053590 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1974 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1978 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1979 pkey=0 pd=0xf28060053590 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1980 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1980 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b4441 ctsFifoLkey=0x1b4441 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1980 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1981 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1982 pkey=0 pd=0xf28060053590 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1983 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1983 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9e9b ctsFifoLkey=0x1b9e9b +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1983 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1984 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1985 pkey=0 pd=0xf28060053590 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1975 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1980 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1983 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23550 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:23550:23550 [0] NCCL INFO Connected all trees +host-d:23550:23550 [0] NCCL INFO Connected binomial trees +host-d:23550:23550 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:23550:23550 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:23550:23550 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:23550:23550 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.9 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 306.0s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f7bf05eca617f387d1d25a2b4faa6718 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=28 accepted=33/46 ttft=5.989 decode=1.991 elapsed=7.980 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-20802aec6afe3ba6830a63138bf733e3 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=5.932 decode=3.353 elapsed=9.285 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a2bc8f9466f9f6d1512a4ddb6664823b prompt=15550 cached=0 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=13 accepted=30/36 ttft=11.886 decode=1.054 elapsed=12.941 queued=0.000 tok/s=40.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-618091e92430c03b04cba968c60c9310 prompt=15550 cached=0 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=43 accepted=0/0 ttft=11.864 decode=2.412 elapsed=14.276 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9876f453efdc2002595f968b2ff7bd9c prompt=19460 cached=0 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=35 accepted=45/68 ttft=14.876 decode=2.570 elapsed=17.447 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-eaf2ea7ee7dc10a7264479cf811dbe51 prompt=19460 cached=0 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=79 accepted=0/0 ttft=14.845 decode=4.398 elapsed=19.243 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-8a7c521c4686fc8cccfa6fa4af4c5f3a prompt=21874 cached=0 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=70 accepted=78/124 ttft=16.659 decode=5.118 elapsed=21.777 queued=0.000 tok/s=28.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7ddf1692439568619199671d9e568cd6 prompt=21874 cached=0 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=147 accepted=0/0 ttft=16.594 decode=8.172 elapsed=24.766 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-a9154a4167e52982784e3e9af796d2c6 prompt=26786 cached=0 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=72 accepted=88/126 ttft=20.594 decode=5.260 elapsed=25.854 queued=0.000 tok/s=30.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-022632346ed6b7c97bfb8b1a5f27bc78 prompt=26786 cached=0 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=159 accepted=0/0 ttft=20.542 decode=8.837 elapsed=29.379 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-5e09de8029f88013637d3a67ff01ed6d prompt=29293 cached=0 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=73 accepted=86/129 ttft=22.414 decode=5.362 elapsed=27.776 queued=0.000 tok/s=29.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bfef43b066636e5bd6b212d97b59f697 prompt=29293 cached=0 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=159 accepted=0/0 ttft=22.379 decode=8.873 elapsed=31.251 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-16ddaf0ca553e17f39fff4be7871bc31 prompt=32216 cached=0 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=78 accepted=82/126 ttft=24.483 decode=5.568 elapsed=30.050 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-1793421cdcad82e6b40f73bae2e338fe prompt=32216 cached=0 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=159 accepted=0/0 ttft=24.415 decode=8.887 elapsed=33.303 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-01324381b3452b591aeb643e7848ac79 prompt=35142 cached=0 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=78 accepted=82/128 ttft=26.917 decode=5.553 elapsed=32.469 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-e7a9c8f7de7e0f13730ed0d0c22006c9 prompt=35142 cached=0 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=159 accepted=0/0 ttft=26.877 decode=8.831 elapsed=35.708 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-2de92c5f8482391f5e59fcd7787c7e5d prompt=37860 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=29 accepted=130/135 ttft=28.855 decode=2.701 elapsed=31.556 queued=0.000 tok/s=58.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ce70ec9c9e057f7f8e1375dfc4956ba6 prompt=37860 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=28.818 decode=8.859 elapsed=37.677 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-28fffaa0e501a8d356fcb632ec2a4c19 prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=9761cff1a208 finish=length rounds=70 accepted=89/135 ttft=1.755 decode=5.192 elapsed=6.947 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-db13a11f299b546043521058ab7a1c8d prompt=7412 cached=0 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=34 accepted=61/80 ttft=5.811 decode=2.671 elapsed=8.483 queued=0.000 tok/s=35.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-86722dc6a9aeaa01a33e582dd1c878b5 prompt=7412 cached=0 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=94 accepted=0/0 ttft=5.759 decode=5.234 elapsed=10.993 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-68113bdfc367430e8d852dd60450086b prompt=7474 cached=0 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=23 accepted=49/62 ttft=5.839 decode=1.892 elapsed=7.731 queued=0.000 tok/s=37.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-08832bf9e8afaf8da7660e6c568ee2fb prompt=7474 cached=0 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=71 accepted=0/0 ttft=5.786 decode=3.957 elapsed=9.743 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-063fba8852d137f894e8eba56f858a35 prompt=12623 cached=0 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=56 accepted=64/89 ttft=9.870 decode=3.953 elapsed=13.824 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f5e56fd514a1b5aea266141ba61483b8 prompt=12623 cached=0 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=119 accepted=0/0 ttft=9.853 decode=6.611 elapsed=16.465 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9c0d06137b2938060403d92a173babe4 prompt=12720 cached=0 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=40 accepted=39/56 ttft=9.932 decode=2.759 elapsed=12.691 queued=0.000 tok/s=28.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b3e95a584898d676f6d48059e4accb2f prompt=12720 cached=0 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=78 accepted=0/0 ttft=9.896 decode=4.340 elapsed=14.236 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f74e87f96bb07004e26462bfd478d5e6 prompt=18063 cached=0 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=78 accepted=82/123 ttft=13.730 decode=5.553 elapsed=19.283 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3badb42872943ef09ddbfcc88e0c7205 prompt=18063 cached=0 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=159 accepted=0/0 ttft=13.671 decode=8.851 elapsed=22.522 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ea4b741d6a927635ff8c9050aab39bf9 prompt=15032 cached=0 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=42 accepted=43/62 ttft=11.618 decode=2.911 elapsed=14.529 queued=0.000 tok/s=28.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fd246445a8a76f18ce66c4d3ae040740 prompt=15032 cached=0 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=84 accepted=0/0 ttft=11.587 decode=4.677 elapsed=16.264 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-36c468183bf770b568e857e94f71c3fc prompt=40609 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=23 accepted=136/138 ttft=30.828 decode=2.356 elapsed=33.185 queued=0.000 tok/s=67.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ecf1ab695c783aff2320480dcfa14161 prompt=40609 cached=0 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=30.790 decode=8.915 elapsed=39.705 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-bae8e81da75970db94729d7e7925de46 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=68 accepted=88/131 ttft=1.133 decode=4.974 elapsed=6.106 queued=0.000 tok/s=31.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-cadd0cb7ae1f3a215880432efbb87f00 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=155 accepted=0/0 ttft=1.098 decode=8.450 elapsed=9.548 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2e5c84ef3b888bb1a63f355c9bf3c362 prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.588 decode=0.305 elapsed=3.893 queued=0.000 tok/s=36.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3961110b20c13e01400d738dddcc3d4f prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=3.571 decode=0.624 elapsed=4.195 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9523133e9c27c4a6b1b08e42a873bea2 prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=39 accepted=43/59 ttft=7.466 decode=2.741 elapsed=10.207 queued=0.000 tok/s=29.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-78183758b5f2d7377b539efdf1d58ea5 prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=81 accepted=0/0 ttft=7.406 decode=4.513 elapsed=11.920 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ca8d9831750a0b0f77d1b3b0af166d2b prompt=15152 cached=0 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=71 accepted=88/117 ttft=11.699 decode=5.080 elapsed=16.779 queued=0.000 tok/s=31.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-f40d9e2f8ffad9e0bff3053b0ae50b98 prompt=15152 cached=0 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=159 accepted=0/0 ttft=11.656 decode=8.833 elapsed=20.489 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-84e53f9f58cc56754194885ddde6d818 prompt=17550 cached=0 thinking=True effort=high tokens=160 sha=fe9efe1d4302 finish=length rounds=61 accepted=99/167 ttft=13.434 decode=23.174 elapsed=36.609 queued=0.000 tok/s=6.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-a862a8af1c5e87a5dd0fad2eeb8f13ea prompt=20872 cached=0 thinking=True effort=high tokens=160 sha=eab19e6ff8fd finish=length rounds=54 accepted=105/161 ttft=30.514 decode=6.471 elapsed=36.985 queued=0.834 tok/s=24.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:23550:23550 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:23550:23550 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:23550:23550 [0] NCCL INFO ncclCommInitRank comm 0x1ed40970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:23550:23550 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.39 (kernels 0.26, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.08, rest 0.00) +host-d:23550:23550 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:23550:23550 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf28060450000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf28060491000) +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1986 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1986 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5e5c ctsFifoLkey=0x1b5e5c +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1986 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1987 pkey=0 pd=0xf28060053590 oooRq=0 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1988 pkey=0 pd=0xf28060053590 +host-d:23550:23569 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1986 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:23550:23550 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/server-rank1.log new file mode 100644 index 0000000000..7878bf6811 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2main/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T02:22:42+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:2265:2265 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2265:2265 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2265:2265 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2265:2265 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2265:2265 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2265:2265 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2265:2265 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2265:2265 [0] NCCL INFO Count not open libibverbs.so +host-e:2265:2265 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2265:2265 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2265:2265 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2265:2265 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2265:2265 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x156cb600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2265:2265 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2265:2265 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2265:2265 [0] NCCL INFO Initialized NET plugin IB +host-e:2265:2265 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2265:2265 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2265:2265 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2265:2265 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2265:2265 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2265:2265 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2265:2265 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2265:2265 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2265:2265 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2265:2265 [0] NCCL INFO Using network IB +host-e:2265:2265 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2265:2265 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x12a997a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2265:2265 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2265:2265 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2265:2265 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2265:2265 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2265:2265 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2265:2265 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2265:2265 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2265:2265 [0] NCCL INFO comm 0x12a997a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2265:2265 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2265:2265 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2265:2265 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2265:2265 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2265:2265 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2265:2265 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2265:2277 [0] NCCL INFO [Proxy Service] Device 0 CPU core 11 +host-e:2265:2277 [0] NCCL INFO proxy listening socket at <42305> +host-e:2265:2278 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-e:2265:2265 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2265:2279 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-e:2265:2265 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2265:2265 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2265:2265 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2265:2265 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2265:2265 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2265:2265 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2265:2265 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbc34c010000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbc34c054000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbc34c09c000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbc34c0e0000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbc34c128000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbc34c16c000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbc34c1b4000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbc34c1f8000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1718 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1718 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1718 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1719 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1720 pkey=0 pd=0xfbc34c053eb0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1721 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1721 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x166320 ctsFifoLkey=0x166320 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1721 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1722 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1723 pkey=0 pd=0xfbc34c053eb0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1724 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1724 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1741fe ctsFifoLkey=0x1741fe +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1724 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1725 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1725 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x169d5a ctsFifoLkey=0x169d5a +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1725 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1718 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1721 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1724 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1726 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1727 pkey=0 pd=0xfbc34c053eb0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1725 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1728 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1729 pkey=0 pd=0xfbc34c053eb0 +host-e:2265:2265 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2265:2265 [0] NCCL INFO Connected all trees +host-e:2265:2265 [0] NCCL INFO Connected binomial trees +host-e:2265:2265 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2265:2265 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2265:2265 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2265:2265 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2265:2265 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2265:2265 [0] NCCL INFO ncclCommInitRank comm 0x12a997a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2265:2265 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.39 (kernels 0.24, alloc 0.03, bootstrap 0.03, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.08, rest 0.00) +host-e:2265:2265 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:2265:2265 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbc34c443000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbc34c484000) +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1730 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1730 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x171cdc ctsFifoLkey=0x171cdc +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1730 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1731 pkey=0 pd=0xfbc34c053eb0 oooRq=0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1732 pkey=0 pd=0xfbc34c053eb0 +host-e:2265:2277 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1730 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2265:2265 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/client.log new file mode 100644 index 0000000000..23d97c2c61 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/client.log @@ -0,0 +1,61 @@ +server up 2026-10-23 10:45:46 +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 39.1, "per_stream_tps": 39.1, "ttft_s_max": 0.15, "timing_valid": true, "end_to_end_tps": 38.39389436110603, "makespan_s": 6.667726841988042, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 45.3, "per_stream_tps": 25.0, "ttft_s_max": 0.36, "timing_valid": true, "end_to_end_tps": 44.910795639655774, "makespan_s": 11.400376962992596, "steady_tps": 48.8, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 61.3, "per_stream_tps": 16.9, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 60.988929428730806, "makespan_s": 16.789932362997206, "steady_tps": 65.7, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 29.2, "per_stream_tps": 29.2, "ttft_s_max": 0.22, "timing_valid": true, "end_to_end_tps": 28.56150784320897, "makespan_s": 8.963112221012125, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 40.4, "per_stream_tps": 20.6, "ttft_s_max": 0.51, "timing_valid": true, "end_to_end_tps": 39.893076466471115, "makespan_s": 12.834307237004396, "steady_tps": 41.3, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 51.0, "per_stream_tps": 13.4, "ttft_s_max": 1.18, "timing_valid": true, "end_to_end_tps": 50.69663468828981, "makespan_s": 20.19857937900815, "steady_tps": 53.5, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 4, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 56.1, "per_stream_tps": 56.1, "ttft_s_max": 0.15, "timing_valid": true, "end_to_end_tps": 54.517066718805744, "makespan_s": 4.695777220011223, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 64.0, "per_stream_tps": 32.6, "ttft_s_max": 0.36, "timing_valid": true, "end_to_end_tps": 63.06443732237079, "makespan_s": 8.118680221989052, "steady_tps": 65.4, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 87.1, "per_stream_tps": 22.6, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 86.3259241756314, "makespan_s": 11.86202186398441, "steady_tps": 90.7, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 30.6, "per_stream_tps": 30.6, "ttft_s_max": 0.22, "timing_valid": true, "end_to_end_tps": 29.915635388802567, "makespan_s": 8.55739805198391, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 43.8, "per_stream_tps": 22.4, "ttft_s_max": 0.5, "timing_valid": true, "end_to_end_tps": 43.169398505131866, "makespan_s": 11.860253274993738, "steady_tps": 44.8, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 59.1, "per_stream_tps": 15.3, "ttft_s_max": 1.17, "timing_valid": true, "end_to_end_tps": 58.61437836659613, "makespan_s": 17.470116181997582, "steady_tps": 61.9, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +logs/r2on-conc.json: 12 cells; alone 28/28 equal (0 unequal, 0 failed); serial 30/30 equal (0 unequal, 0 failed) + 0 S1 t1 drafted prompt 7759 history 7757 cached 0 (allowed [0]) ttft 6.02s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok + 1 S1 t1 plain prompt 7759 history 7757 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 1.25s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok +system cuts [4563, 6099, 6611] + 2 S1 t2 drafted prompt 15550 history 15548 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 7.20s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 tool_calls ok + 3 S1 t2 plain prompt 15550 history 15548 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 1.03s sha 1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427 tool_calls ok + 4 S1 t3 drafted prompt 19460 history 19458 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 4.06s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 tool_calls ok + 5 S1 t3 plain prompt 19460 history 19458 cached 18432 (allowed {'min': 11266, 'max': 19458}) ttft 0.93s sha bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604 tool_calls ok + 6 S1 t4 drafted prompt 21874 history 21872 cached 18432 (allowed {'min': 11266, 'max': 19458}) ttft 2.73s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 tool_calls ok + 7 S1 t4 plain prompt 21874 history 21872 cached 20480 (allowed {'min': 13680, 'max': 21872}) ttft 1.12s sha 5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8 tool_calls ok + 8 S1 t5 drafted prompt 26786 history 26784 cached 20480 (allowed {'min': 13680, 'max': 21872}) ttft 5.10s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c length ok + 9 S1 t5 plain prompt 26786 history 26784 cached 26624 (allowed {'min': 18592, 'max': 26784}) ttft 0.43s sha 140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c length ok + 10 S1 t6 drafted prompt 29293 history 29291 cached 26624 (allowed {'min': 18592, 'max': 26784}) ttft 2.31s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 length ok + 11 S1 t6 plain prompt 29293 history 29291 cached 28672 (allowed {'min': 21099, 'max': 29291}) ttft 0.71s sha 0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875 length ok + 12 S1 t7 drafted prompt 32216 history 32214 cached 28672 (allowed {'min': 21099, 'max': 29291}) ttft 2.82s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e length ok + 13 S1 t7 plain prompt 32216 history 32214 cached 30720 (allowed {'min': 24022, 'max': 32214}) ttft 1.20s sha ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e length ok + 14 S1 t8 drafted prompt 35142 history 35140 cached 30720 (allowed {'min': 24022, 'max': 32214}) ttft 3.71s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 length ok + 15 S1 t8 plain prompt 35142 history 35140 cached 34816 (allowed {'min': 26948, 'max': 35140}) ttft 0.56s sha 3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439 length ok + 16 S1 t9 drafted prompt 37860 history 37858 cached 34816 (allowed {'min': 26948, 'max': 35140}) ttft 2.56s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 17 S1 t9 plain prompt 37860 history 37858 cached 36864 (allowed {'min': 29666, 'max': 37858}) ttft 0.93s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 18 X t1 drafted prompt 2115 history 2113 cached 0 (allowed [0]) ttft 1.76s sha 9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd length ok + 19 A t1 drafted prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 1.09s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 tool_calls ok + 20 A t1 plain prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 1.05s sha ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88 tool_calls ok + 21 B t1 drafted prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 1.13s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a tool_calls ok + 22 B t1 plain prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 1.09s sha 13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a tool_calls ok + 23 A t2 drafted prompt 12623 history 12621 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 5.17s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 tool_calls ok + 24 A t2 plain prompt 12623 history 12621 cached 12288 (allowed {'min': 4429, 'max': 12621}) ttft 0.55s sha e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0 tool_calls ok + 25 B t2 drafted prompt 12720 history 12718 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 5.23s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c tool_calls ok + 26 B t2 plain prompt 12720 history 12718 cached 12288 (allowed {'min': 4526, 'max': 12718}) ttft 0.60s sha 1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c tool_calls ok + 27 A t3 drafted prompt 18063 history 18061 cached 12288 (allowed {'min': 4429, 'max': 12621}) ttft 4.44s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc length ok + 28 A t3 plain prompt 18063 history 18061 cached 16384 (allowed {'min': 9869, 'max': 18061}) ttft 1.29s sha ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc length ok + 29 B t3 drafted prompt 15032 history 15030 cached 12288 (allowed {'min': 4526, 'max': 12718}) ttft 2.32s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa tool_calls ok + 30 B t3 plain prompt 15032 history 15030 cached 14336 (allowed {'min': 6838, 'max': 15030}) ttft 0.74s sha bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa tool_calls ok + 31 E t1 drafted prompt 40609 history 40607 cached 6144 (allowed {'min': 0, 'max': 6611}) ttft 26.18s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 32 E t1 plain prompt 40609 history 40607 cached 38912 (allowed {'min': 32415, 'max': 40607}) ttft 1.34s sha 21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855 length ok + 33 T t1 drafted prompt 1520 history 1517 cached 0 (allowed [0]) ttft 1.14s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 stop ok + 34 T t1 plain prompt 1520 history 1517 cached 0 (allowed [0]) ttft 1.10s sha ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9 stop ok + 35 T t2 drafted prompt 4208 history 4205 cached 0 (allowed [0]) ttft 3.59s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 36 T t2 plain prompt 4208 history 4205 cached 4096 (allowed {'min': 1, 'max': 4205}) ttft 0.37s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 37 C t1 cancel prompt None history 0 cached None (allowed None) ttft 0.00s sha None None ok + 38 C t1 drafted prompt 9702 history 9700 cached 0 (allowed {'min': 0, 'max': 9688}) ttft 8.21s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 tool_calls ok + 39 C t1 plain prompt 9702 history 9700 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 1.17s sha efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5 tool_calls ok + 40 C t2 drafted prompt 15152 history 15150 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 5.47s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 length ok + 41 C t2 plain prompt 15152 history 15150 cached 14336 (allowed {'min': 6958, 'max': 15150}) ttft 0.83s sha bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84 length ok + 42 A t4 drafted prompt 20872 history 20870 cached 16384 (allowed {'min': 9869, 'max': 18061}) ttft 6.56s sha eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c length ok + 43 B t4 drafted prompt 17550 history 17548 cached 14336 (allowed {'min': 6838, 'max': 15030}) ttft 2.61s sha fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8 length ok +record: 44 requests, 0 not cached as planned +=== kvp-client r2on rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/conc.json b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/conc.json new file mode 100644 index 0000000000..cedab5714b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/conc.json @@ -0,0 +1,4588 @@ +{ + "label": "", + "tokens": 256, + "mixed": false, + "cells": [ + { + "prompt": "code", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.1, + "per_stream_tps": 39.1, + "ttft_s_max": 0.15, + "timing_valid": true, + "end_to_end_tps": 38.39389436110603, + "makespan_s": 6.667726841988042, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 39.111183645699036, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 6.667726841988042, + "end_to_end_tps": 38.39389436110603, + "timing_valid": true, + "per_stream_tps": [ + 39.1 + ], + "ttft_s": [ + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + } + ], + "client_intervals": [ + { + "start": 171653.8706392, + "first": 171654.018317561, + "last": 171660.538191833, + "end": 171660.538366042, + "elapsed_s": 6.667726841988042, + "chunk_offsets_s": [ + 0.1476783609832637, + 0.22146503799012862, + 0.29311703698476776, + 0.3686209649895318, + 0.45138595200842246, + 0.5376236510055605, + 0.615939201990841, + 0.7003445769951213, + 0.7824475780071225, + 0.8656657009851187, + 0.947654590010643, + 1.0326523820112925, + 1.1123830569849815, + 1.1938489839958493, + 1.267150314990431, + 1.342980772984447, + 1.4186598219966982, + 1.4974584149895236, + 1.5838493460032623, + 1.66806612801156, + 1.7548527889885008, + 1.8375281750049908, + 1.9112420520104934, + 1.983108866988914, + 2.0555393009854015, + 2.132603233010741, + 2.2125493490020745, + 2.300823628989747, + 2.3862808459962253, + 2.4705801410018466, + 2.553503016009927, + 2.6378257180040237, + 2.721602483012248, + 2.806359539012192, + 2.8891040289890952, + 2.972319944994524, + 3.0579194349993486, + 3.1461136830039322, + 3.230339505011216, + 3.3173938780091703, + 3.4067137769889086, + 3.4962777720065787, + 3.5826306240051053, + 3.6665894059988204, + 3.749583528988296, + 3.832550483988598, + 3.9163151209941134, + 3.998565146990586, + 4.08098698098911, + 4.162579036987154, + 4.244671797991032, + 4.327837985008955, + 4.414088101009838, + 4.493393879005453, + 4.565482408012031, + 4.639373965008417, + 4.715835495997453, + 4.801760603004368, + 4.891736822988605, + 4.974862066999776, + 5.058068126993021, + 5.158012851985404, + 5.267683248996036, + 5.355062230984913, + 5.438757075986359, + 5.522118433000287, + 5.60985418400378, + 5.698548529006075, + 5.7864618329913355, + 5.874980994005455, + 5.954260803991929, + 6.037344048003433, + 6.12487373399199, + 6.20727118398645, + 6.292544033989543, + 6.374315066001145, + 6.459566058998462, + 6.535679845983395, + 6.606413667002926, + 6.66755263300729 + ], + "chunk_gaps_ms": { + "count": 79, + "p50": 83.16618701792322, + "p95": 89.5639950176701, + "max": 109.67039701063186 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 45.3, + "per_stream_tps": 25.0, + "ttft_s_max": 0.36, + "timing_valid": true, + "end_to_end_tps": 44.910795639655774, + "makespan_s": 11.400376962992596, + "steady_tps": 48.8, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 45.31950704463134, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 11.400376962992596, + "end_to_end_tps": 44.910795639655774, + "timing_valid": true, + "steady_tps": 48.833368387927145, + "per_stream_tps": [ + 27.4, + 22.7 + ], + "ttft_s": [ + 0.36, + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + } + ], + "client_intervals": [ + { + "start": 171660.54007563, + "first": 171660.901842533, + "last": 171670.21725785, + "end": 171670.217472043, + "elapsed_s": 9.677396412997041, + "chunk_offsets_s": [ + 0.36176690299180336, + 0.4623741259856615, + 0.5770838960015681, + 0.6958835469849873, + 0.8143707659910433, + 0.9304729069990572, + 1.0419075159879867, + 1.1627528049866669, + 1.2781274239823688, + 1.395356383989565, + 1.5157384229823947, + 1.6360685259860475, + 1.7540928479866125, + 1.8731147079961374, + 1.9902997959870845, + 2.1094592409790494, + 2.2309909949835856, + 2.351287337980466, + 2.4718838580010924, + 2.589694516005693, + 2.712271072989097, + 2.8306908039958216, + 2.9516264449921437, + 3.0701249279954936, + 3.1934447340026964, + 3.3201499260030687, + 3.4437621329852846, + 3.5629715620016214, + 3.684075267985463, + 3.804429996002, + 3.9270206170040183, + 4.047606592997909, + 4.164188719994854, + 4.2809560780005995, + 4.400717092998093, + 4.5229664009821136, + 4.640225377981551, + 4.760166919993935, + 4.887019183981465, + 5.010303486982593, + 5.129004435002571, + 5.251283167977817, + 5.372451417992124, + 5.494369428983191, + 5.617854740994517, + 5.737712587986607, + 5.860874331003288, + 5.98430740198819, + 6.100114198983647, + 6.22139851297834, + 6.343473118002294, + 6.457588725985261, + 6.575803096988238, + 6.698594198998762, + 6.818469533987809, + 6.940754826995544, + 7.065283965988783, + 7.184124113991857, + 7.302814358001342, + 7.41820654598996, + 7.530779414984863, + 7.649374314991292, + 7.771413976006443, + 7.894912407005904, + 8.016190258000279, + 8.139535057998728, + 8.264634581981227, + 8.384501708991593, + 8.510345507005695, + 8.628138532978483, + 8.75054435399943, + 8.872187022003345, + 8.992518326005666, + 9.110995561990421, + 9.229790782002965, + 9.353960287990049, + 9.469465148984455, + 9.572857482999098, + 9.67718221998075 + ], + "chunk_gaps_ms": { + "count": 78, + "p50": 120.11894250463229, + "p95": 125.0995239824988, + "max": 126.85226398753002 + } + }, + { + "start": 171660.539980622, + "first": 171660.68681287, + "last": 171671.940245073, + "end": 171671.940357585, + "elapsed_s": 11.400376962992596, + "chunk_offsets_s": [ + 0.146832247992279, + 0.21752462899894454, + 0.46258606199990027, + 0.5775665050023235, + 0.6961040110036265, + 0.8145502379920799, + 0.930669498979114, + 1.0424477089836728, + 1.1629623409826308, + 1.278330303990515, + 1.395526879990939, + 1.5159141669864766, + 1.6366075520054437, + 1.7542644960049074, + 1.8733197649999056, + 1.9904954279772937, + 2.109900696988916, + 2.2315031079924665, + 2.3514985869987868, + 2.472198595001828, + 2.5902872690057848, + 2.7125893929915037, + 2.8310097479843535, + 2.9518789729918353, + 3.0703923199907877, + 3.193748030986171, + 3.32043442199938, + 3.4441071579931304, + 3.562801785999909, + 3.6844094449770637, + 3.8042893869860563, + 3.9273547129996587, + 4.047456528991461, + 4.1645377919776365, + 4.280787085997872, + 4.40107453300152, + 4.523180593998404, + 4.640444289980223, + 4.760306680982467, + 4.886869551992277, + 5.010161534999497, + 5.129144626989728, + 5.251134910999099, + 5.372313384985318, + 5.494230644981144, + 5.617706708988408, + 5.737853355996776, + 5.861003690981306, + 5.984450089978054, + 6.0999792229849845, + 6.221162945003016, + 6.343613021977944, + 6.457731446978869, + 6.575943816977087, + 6.698725734982872, + 6.8183079499867745, + 6.940892170998268, + 7.065128844988067, + 7.184251793980366, + 7.302675685990835, + 7.418494482990354, + 7.53091112698894, + 7.649514026998077, + 7.771266374998959, + 7.894775670982199, + 8.016041569004301, + 8.139673793979455, + 8.264769621979212, + 8.384632589004468, + 8.510509394982364, + 8.627847795985872, + 8.750681121979142, + 8.87240338997799, + 8.992376324982615, + 9.110855304985307, + 9.22992307000095, + 9.35380625599646, + 9.469327611994231, + 9.572702874982497, + 9.677011003979715, + 9.754932811978506, + 9.82040564500494, + 9.88328528800048, + 9.94806983199669, + 10.012255013978574, + 10.077368645987008, + 10.150635130994488, + 10.224909825978102, + 10.296328833996085, + 10.362408500979654, + 10.429311400977895, + 10.50277636598912, + 10.577654742985032, + 10.652031821984565, + 10.723469838005258, + 10.78773996498785, + 10.854968066996662, + 10.929445641988423, + 11.006160438992083, + 11.083818471000995, + 11.164567661995534, + 11.249125981994439, + 11.328462178003974, + 11.400264450989198 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 118.60290000913665, + "p95": 124.23667398979887, + "max": 245.06143300095573 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 61.3, + "per_stream_tps": 16.9, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 60.988929428730806, + "makespan_s": 16.789932362997206, + "steady_tps": 65.7, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 61.2853593197515, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 16.789932362997206, + "end_to_end_tps": 60.988929428730806, + "timing_valid": true, + "steady_tps": 65.67932948051828, + "per_stream_tps": [ + 18.1, + 15.8, + 16.4, + 17.4 + ], + "ttft_s": [ + 0.89, + 0.61, + 0.15, + 0.36 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6c97574b6d3c", + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692" + } + ], + "client_intervals": [ + { + "start": 171671.944330731, + "first": 171672.834333341, + "last": 171686.944961996, + "end": 171686.945522014, + "elapsed_s": 15.001191282994114, + "chunk_offsets_s": [ + 0.8900026099872775, + 1.0520783700048923, + 1.2185252770141233, + 1.387263693002751, + 1.5597998629964422, + 1.7295834819960874, + 1.9009161599969957, + 2.075551407004241, + 2.2442764959996566, + 2.42001860999153, + 2.6015116490016226, + 2.78338060999522, + 2.960894616000587, + 3.1398434579896275, + 3.32053763300064, + 3.499521290010307, + 3.6802507610118482, + 3.864196159003768, + 4.042485702986596, + 4.225388730992563, + 4.408350318990415, + 4.585725796991028, + 4.771748672996182, + 4.956226456008153, + 5.143826504005119, + 5.331524807988899, + 5.517060707003111, + 5.705195410992019, + 5.880404563999036, + 6.066945938015124, + 6.248983188008424, + 6.424210133001907, + 6.597503825003514, + 6.777646142989397, + 6.957565179996891, + 7.143975913000759, + 7.32874179398641, + 7.522459681000328, + 7.716383729013614, + 7.905696900998009, + 8.087221669993596, + 8.263416394009255, + 8.437809866009047, + 8.625609754002653, + 8.810303139005555, + 8.994102778000524, + 9.17830197000876, + 9.361100070993416, + 9.541045955993468, + 9.722969015012495, + 9.90789859299548, + 10.085991466010455, + 10.27126987700467, + 10.459955208003521, + 10.641751449991716, + 10.827819063008064, + 11.013578754995251, + 11.194512627989752, + 11.378002393990755, + 11.560320157994283, + 11.7342485729896, + 11.92195086198626, + 12.109902975993464, + 12.299303845007671, + 12.486864294012776, + 12.677239709999412, + 12.865081791998819, + 13.041089795995504, + 13.221080866991542, + 13.397254776005866, + 13.578334696998354, + 13.758386583009269, + 13.935299261996988, + 14.115668637008639, + 14.302175868011545, + 14.491683217987884, + 14.670049228996504, + 14.837088347005192, + 15.00063126499299 + ], + "chunk_gaps_ms": { + "count": 78, + "p50": 181.66050549189094, + "p95": 189.50734997633845, + "max": 193.92404801328667 + } + }, + { + "start": 171671.944407643, + "first": 171672.551348484, + "last": 171688.733413716, + "end": 171688.733924581, + "elapsed_s": 16.789516938006273, + "chunk_offsets_s": [ + 0.6069408410112374, + 0.7458714550011791, + 1.0524495230056345, + 1.2186315169965383, + 1.3875728620041627, + 1.5599104069988243, + 1.7299951790191699, + 1.9013857780082617, + 2.0758316639985424, + 2.24431867201929, + 2.4203282259986736, + 2.6018951710138936, + 2.783833716006484, + 2.961238265997963, + 3.140312867995817, + 3.3211047220102046, + 3.499731915013399, + 3.6803561850101687, + 3.864262607006822, + 4.042898136016447, + 4.225536012003431, + 4.408519071992487, + 4.586158741993131, + 4.77156876801746, + 4.95648692100076, + 5.144238505017711, + 5.3316925370018, + 5.5173869150166865, + 5.705536163994111, + 5.880751142016379, + 6.067279459006386, + 6.249382612993941, + 6.424839447019622, + 6.597920051019173, + 6.777950224000961, + 6.957860813010484, + 7.144160249008564, + 7.32894316999591, + 7.522618642018642, + 7.716522225004155, + 7.905910614004824, + 8.087547079019714, + 8.263618602999486, + 8.437989674013807, + 8.625967579020653, + 8.8105169320188, + 8.994525451998925, + 9.178800068009878, + 9.36141867999686, + 9.54141880501993, + 9.72319293601322, + 9.90829675400164, + 10.086386890994618, + 10.271477572998265, + 10.46049546499853, + 10.641993947006995, + 10.828158296004403, + 11.013756036001723, + 11.19481336401077, + 11.378258298995206, + 11.56067956701736, + 11.73464810999576, + 11.922209903015755, + 12.11036689701723, + 12.299653206020594, + 12.487236710992875, + 12.67730014299741, + 12.865344145015115, + 13.041381701012142, + 13.221515892015304, + 13.397394424013328, + 13.578532953019021, + 13.758563544019125, + 13.93574617500417, + 14.116090671013808, + 14.302564429002814, + 14.492144963995088, + 14.670560238009784, + 14.837714829016477, + 15.000810290017398, + 15.116952690004837, + 15.243120122991968, + 15.367963088996476, + 15.484709475014824, + 15.599165630992502, + 15.705126198008657, + 15.787207554007182, + 15.853381254011765, + 15.918551367998589, + 15.985651646013139, + 16.059654886019416, + 16.13478036900051, + 16.206125650001923, + 16.28467136499239, + 16.37129238000489, + 16.46129847501288, + 16.550764921994414, + 16.637050160003128, + 16.71667416600394, + 16.789006072998745 + ], + "chunk_gaps_ms": { + "count": 99, + "p50": 180.03017298178747, + "p95": 189.5805349922739, + "max": 306.5780680044554 + } + }, + { + "start": 171671.944150314, + "first": 171672.089960674, + "last": 171687.649062688, + "end": 171687.649156736, + "elapsed_s": 15.705006422009319, + "chunk_offsets_s": [ + 0.14581036000163294, + 0.21687341498909518, + 0.4629324059933424, + 0.7460766079893801, + 1.052301138988696, + 1.2184445570164826, + 1.387494925991632, + 1.5597143750055693, + 1.729584282002179, + 1.9011442569899373, + 2.0757741760171484, + 2.2442137759935576, + 2.4202476190112066, + 2.601645729999291, + 2.7836096190148965, + 2.9611296899965964, + 3.1400764670106582, + 3.3207691379939206, + 3.499752300005639, + 3.6801839930121787, + 3.8640433269902132, + 4.042706615990028, + 4.225314779003384, + 4.408343248011079, + 4.5859489499998745, + 4.772119266010122, + 4.956443082017358, + 5.143797497003106, + 5.331449560006149, + 5.517200898990268, + 5.70541243700427, + 5.88062160598929, + 6.067165763000958, + 6.249209909001365, + 6.424425078002969, + 6.5977233620069455, + 6.77777401599451, + 6.957708813017234, + 7.144207881996408, + 7.328882194997277, + 7.522459266008809, + 7.716471616993658, + 7.905914678005502, + 8.087358935008524, + 8.26355932300794, + 8.437951899017207, + 8.62574889100506, + 8.810436675994424, + 8.994246490998194, + 9.178571235999698, + 9.361238696001237, + 9.541188756993506, + 9.72310146401287, + 9.908115441998234, + 10.086208779015578, + 10.271487638005055, + 10.46037661700393, + 10.641966843017144, + 10.828041592001682, + 11.013716835994273, + 11.194735060998937, + 11.378290587017545, + 11.560535519005498, + 11.734474110009614, + 11.922088527004234, + 12.110121904988773, + 12.299535878002644, + 12.487085239001317, + 12.677292734995717, + 12.865299265016802, + 13.041308389016194, + 13.22129877199768, + 13.397483497014036, + 13.578465721016983, + 13.758461399993394, + 13.935517678997712, + 14.115900623000925, + 14.302093884994974, + 14.491901795001468, + 14.670268862013472, + 14.837309452006593, + 15.000624978012638, + 15.117151203012327, + 15.242895450996002, + 15.36795699299546, + 15.48453312300262, + 15.599028143013129, + 15.704912373999832 + ], + "chunk_gaps_ms": { + "count": 87, + "p50": 181.39811098808423, + "p95": 193.5770710115321, + "max": 306.2245309993159 + } + }, + { + "start": 171671.943992218, + "first": 171672.306516936, + "last": 171686.945181981, + "end": 171686.945342877, + "elapsed_s": 15.001350659003947, + "chunk_offsets_s": [ + 0.3625247179879807, + 0.4632554299896583, + 0.7461656800005585, + 1.053138980991207, + 1.2191273749922402, + 1.388104943005601, + 1.5602095279900823, + 1.7301407789927907, + 1.901640497992048, + 2.0762847370060626, + 2.244699776987545, + 2.4207844199845567, + 2.6021380669844802, + 2.7841640200058464, + 2.961826443002792, + 3.14062848399044, + 3.321300579002127, + 3.500303356006043, + 3.681094555009622, + 3.8644837600004394, + 4.043258536985377, + 4.226158509001834, + 4.408995089004748, + 4.586843944009161, + 4.77233310599695, + 4.956660553987604, + 5.144579626008635, + 5.332141785998829, + 5.517711908003548, + 5.706062933983048, + 5.881379990983987, + 6.067924083996331, + 6.250042854982894, + 6.424992487009149, + 6.598605236009462, + 6.778202240006067, + 6.958470157987904, + 7.144540345994756, + 7.3295292840048205, + 7.523242883005878, + 7.7168947699829005, + 7.906486870982917, + 8.08827575299074, + 8.264198347984347, + 8.438442987011513, + 8.626348060002783, + 8.810969045007369, + 8.994888732995605, + 9.17912514100317, + 9.361773865006398, + 9.541922183008865, + 9.723642536991974, + 9.908775299001718, + 10.086890284001129, + 10.27184279001085, + 10.460945961996913, + 10.64244607600267, + 10.82853898499161, + 11.014225508988602, + 11.195263925998006, + 11.378708923992235, + 11.561130063986639, + 11.735101775004296, + 11.922659711999586, + 12.110732625995297, + 12.300127783993958, + 12.487616664002417, + 12.677514078997774, + 12.865725729992846, + 13.041833702009171, + 13.221895892987959, + 13.39775936899241, + 13.578985753993038, + 13.759038377000252, + 13.93621377600357, + 14.116579680005088, + 14.302717501996085, + 14.492454291990725, + 14.670829182985472, + 14.837912221002625, + 15.00118976298836 + ], + "chunk_gaps_ms": { + "count": 80, + "p50": 181.7546179954661, + "p95": 189.8974149953574, + "max": 306.97330099064857 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "950590a0cb8473327c8f149a6ea84432a6d6d090cbfc37c6869486c903ed6aa0", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b94a7bd1b3ffe7829775dae1c37814b8dbc58160ea8a19af3360e1aa6dcdd4e1", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "6642e40d33c31a5db195cf3ac9956792b3237419b59cc996886dc3cd26ee7b7b", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "6c97574b6d3c87434dca08b1f0b10d29b419c8f7ab8c37ebb5ad0926270991ae", + "output_sha256": "4ee8febba1190b3a6850eb3bf5cdbe758c8886ce7f0432c7ec212b409895d692", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 29.2, + "per_stream_tps": 29.2, + "ttft_s_max": 0.22, + "timing_valid": true, + "end_to_end_tps": 28.56150784320897, + "makespan_s": 8.963112221012125, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 29.15386052613588, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 8.963112221012125, + "end_to_end_tps": 28.56150784320897, + "timing_valid": true, + "per_stream_tps": [ + 29.2 + ], + "ttft_s": [ + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + } + ], + "client_intervals": [ + { + "start": 171783.371367364, + "first": 171783.587396609, + "last": 171792.334094144, + "end": 171792.334479585, + "elapsed_s": 8.963112221012125, + "chunk_offsets_s": [ + 0.2160292449989356, + 0.29406303001451306, + 0.3685653500142507, + 0.43455739101045765, + 0.5018265090184286, + 0.5748347770131659, + 0.6400762890116312, + 0.7041819260048214, + 0.7691012290015351, + 0.8341165330202784, + 0.8973257190082222, + 0.9615469570271671, + 1.026273300027242, + 1.0918581090227235, + 1.155787905008765, + 1.2196783420222346, + 1.2858412960194983, + 1.3625434940040577, + 1.4384471450175624, + 1.5114613020268735, + 1.5875251300167292, + 1.676105134014506, + 1.7590866760001518, + 1.8370235320180655, + 1.9008804170007352, + 1.9650252860155888, + 2.0273987110122107, + 2.091569772019284, + 2.157557318016188, + 2.2338234830240253, + 2.3070946799998637, + 2.380461541004479, + 2.4550089170224965, + 2.539564399019582, + 2.626287070015678, + 2.7067164780164603, + 2.792616379010724, + 2.8743948780174833, + 2.9590387280040886, + 3.034205242001917, + 3.1088239140226506, + 3.1826699290249962, + 3.2569872720050626, + 3.34019715100294, + 3.423324165021768, + 3.5075012460001744, + 3.5766673120087944, + 3.6526492680131923, + 3.7265748510253616, + 3.8006532020226587, + 3.8792200919997413, + 3.9600183490256313, + 4.026464936003322, + 4.099068659008481, + 4.16269122302765, + 4.225699082016945, + 4.291869173001032, + 4.368753627000842, + 4.441045445011696, + 4.516137943021022, + 4.5894954280229285, + 4.654341707006097, + 4.722382715001004, + 4.795810008014087, + 4.86871746001998, + 4.943817222025245, + 5.019011768017663, + 5.1084216790040955, + 5.184858820022782, + 5.251301583019085, + 5.3239664749999065, + 5.388573985022958, + 5.451558772008866, + 5.516296139016049, + 5.5824392850045115, + 5.647526797023602, + 5.713094503007596, + 5.776621642027749, + 5.842394516017521, + 5.90766481301398, + 5.9720292030251585, + 6.035314598004334, + 6.098294457013253, + 6.162098189000972, + 6.227540614025202, + 6.29877326200949, + 6.373953712027287, + 6.44968734701979, + 6.528051310015144, + 6.607929148012772, + 6.672812387027079, + 6.737054937024368, + 6.80339222800103, + 6.868179739016341, + 6.933955397020327, + 7.008694358024513, + 7.082880774018122, + 7.1588367620133795, + 7.232024999015266, + 7.294680872000754, + 7.3602082730212715, + 7.423940086009679, + 7.487855115003185, + 7.551924624014646, + 7.61725784902228, + 7.683257092023268, + 7.747914330015192, + 7.8111876620096155, + 7.875542996014701, + 7.940469212015159, + 8.004510145023232, + 8.069093432015507, + 8.132758060004562, + 8.201724910002667, + 8.277061185013736, + 8.351343089016154, + 8.42355321100331, + 8.500498274021083, + 8.58072552102385, + 8.645766505011125, + 8.7093051810225, + 8.773310450022109, + 8.838127882016124, + 8.90216780701303, + 8.962726780009689 + ], + "chunk_gaps_ms": { + "count": 124, + "p50": 66.85785249283072, + "p95": 84.1770809784066, + "max": 89.4099109864328 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 40.4, + "per_stream_tps": 20.6, + "ttft_s_max": 0.51, + "timing_valid": true, + "end_to_end_tps": 39.893076466471115, + "makespan_s": 12.834307237004396, + "steady_tps": 41.3, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 40.41479342215464, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 12.834307237004396, + "end_to_end_tps": 39.893076466471115, + "timing_valid": true, + "steady_tps": 41.287870087556115, + "per_stream_tps": [ + 20.6, + 20.7 + ], + "ttft_s": [ + 0.21, + 0.51 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + } + ], + "client_intervals": [ + { + "start": 171792.336300629, + "first": 171792.550895136, + "last": 171804.901542608, + "end": 171804.901656705, + "elapsed_s": 12.565356076025637, + "chunk_offsets_s": [ + 0.21459450700785965, + 0.29333987800055183, + 0.6121491500234697, + 0.7272839270008262, + 0.8471481090236921, + 0.9659616640128661, + 1.0806512880080845, + 1.1998424120247364, + 1.3161590009985957, + 1.4278623610152863, + 1.5461420589999761, + 1.6707860930182505, + 1.785361765010748, + 1.9088276520196814, + 2.0269251900026575, + 2.145320072013419, + 2.267465955024818, + 2.3880938030197285, + 2.5088100350112654, + 2.6272320530260913, + 2.7481146060163155, + 2.8675525310100056, + 2.989542990020709, + 3.114210448024096, + 3.2348408719990402, + 3.3566110910032876, + 3.4800070260243956, + 3.606694874004461, + 3.7257312780129723, + 3.8449227860255633, + 3.9630724360176828, + 4.086781171004986, + 4.20735154702561, + 4.319553230016027, + 4.435081144009018, + 4.5531675620004535, + 4.669231717998628, + 4.781779609009391, + 4.90430327001377, + 5.0226109200157225, + 5.13975099902018, + 5.257951705018058, + 5.375234233011724, + 5.498797416017624, + 5.62377367602312, + 5.748416926013306, + 5.865541998005938, + 5.987509017024422, + 6.106804174021818, + 6.228843882010551, + 6.3453526310040615, + 6.467219283018494, + 6.5909196990251075, + 6.71375768102007, + 6.834571530023823, + 6.959876302018529, + 7.079510580020724, + 7.202057921007508, + 7.318375742004719, + 7.43908738301252, + 7.557193657004973, + 7.678968580003129, + 7.801932659000158, + 7.923420621023979, + 8.04061818899936, + 8.158054701023502, + 8.275313020014437, + 8.39312091001193, + 8.51396627901704, + 8.636792597011663, + 8.757691214006627, + 8.882828674017219, + 9.006148514017696, + 9.128803055005847, + 9.253466354013653, + 9.376767426001607, + 9.497240218013758, + 9.620559993025381, + 9.738247418019455, + 9.855251530010719, + 9.980254606023664, + 10.101563880016329, + 10.2165995700052, + 10.333060256001772, + 10.451760739000747, + 10.570540679007536, + 10.690270046005026, + 10.809170754015213, + 10.926594786025817, + 11.043398241017712, + 11.167535747023067, + 11.286840600019787, + 11.41276833502343, + 11.535473260999424, + 11.653035054012435, + 11.776199549000012, + 11.89761349602486, + 12.013610533002065, + 12.132435529027134, + 12.25129780502175, + 12.365386773017235, + 12.465201992017683, + 12.565241978998529 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 119.53610149794258, + "p95": 125.0030760129448, + "max": 318.80927202291787 + } + }, + { + "start": 171792.336033845, + "first": 171792.841956575, + "last": 171805.170036713, + "end": 171805.170341082, + "elapsed_s": 12.834307237004396, + "chunk_offsets_s": [ + 0.5059227299934719, + 0.6122489089902956, + 0.7274364870099816, + 0.8472429400135297, + 0.9664524160034489, + 1.0810850329871755, + 1.1998924599902239, + 1.3162554159935098, + 1.4283692899916787, + 1.5466662360122427, + 1.6709792290057521, + 1.7857952539925463, + 1.909360101009952, + 2.0274448540003505, + 2.1458272240124643, + 2.2680228519893717, + 2.3886627159954514, + 2.509401652001543, + 2.6278400380106177, + 2.748815150989685, + 2.8681582279969007, + 2.9902376309910323, + 3.114786368998466, + 3.235639065998839, + 3.3572290439915378, + 3.4805624349974096, + 3.607404954993399, + 3.7265256629907526, + 3.8456445790070575, + 3.963820084987674, + 4.087429971987149, + 4.2078984919935465, + 4.320104893995449, + 4.435848761990201, + 4.5539098829904106, + 4.669778967014281, + 4.782537771010539, + 4.904928870993899, + 5.023150616994826, + 5.14038925600471, + 5.258551161998184, + 5.375346871995134, + 5.498894168005791, + 5.623726299003465, + 5.74891299099545, + 5.866074062010739, + 5.988095609995071, + 6.107417550985701, + 6.229498378990684, + 6.345853431994328, + 6.467817396012833, + 6.591488707985263, + 6.71434232199681, + 6.835359099000925, + 6.960633182985475, + 7.080120436992729, + 7.20256822599913, + 7.31919174399809, + 7.439658488001442, + 7.557828265998978, + 7.679510276997462, + 7.802548931009369, + 7.923947950010188, + 8.04113382101059, + 8.158805790008046, + 8.276103134005098, + 8.393732030002866, + 8.514609080011724, + 8.637328581011388, + 8.758294383005705, + 8.8834274589899, + 9.006742802012013, + 9.129640241008019, + 9.254077427001903, + 9.377362819010159, + 9.497863866999978, + 9.621065050014295, + 9.738904539000941, + 9.85599380300846, + 9.980804590013577, + 10.102277689002221, + 10.217160962987691, + 10.333794048987329, + 10.452385923999827, + 10.571248088992434, + 10.690883311006473, + 10.809881555003813, + 10.927193539013388, + 11.044138930010376, + 11.16808105999371, + 11.287615849985741, + 11.413368864014046, + 11.536184878001222, + 11.653689742990537, + 11.776735741994344, + 11.898213865002617, + 12.01423528601299, + 12.133218522008974, + 12.251883869990706, + 12.3659940380021, + 12.46598866500426, + 12.565957484999672, + 12.645647547004046, + 12.70954928098945, + 12.77386781599489, + 12.83400286801043 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 119.32194099063054, + "p95": 124.83213099767454, + "max": 126.84251999598928 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 51.0, + "per_stream_tps": 13.4, + "ttft_s_max": 1.18, + "timing_valid": true, + "end_to_end_tps": 50.69663468828981, + "makespan_s": 20.19857937900815, + "steady_tps": 53.5, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 51.0483279447047, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 20.19857937900815, + "end_to_end_tps": 50.69663468828981, + "timing_valid": true, + "steady_tps": 53.47058759025158, + "per_stream_tps": [ + 14.0, + 13.2, + 13.6, + 13.2 + ], + "ttft_s": [ + 1.18, + 0.22, + 0.51, + 0.83 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2e4cb369c515", + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8" + } + ], + "client_intervals": [ + { + "start": 171805.172741568, + "first": 171806.348406826, + "last": 171824.535145933, + "end": 171824.535420973, + "elapsed_s": 19.362679404992377, + "chunk_offsets_s": [ + 1.175665257993387, + 1.331195693986956, + 1.493257377995178, + 1.6561919929808937, + 1.8212382450001314, + 1.999748180998722, + 2.1765177119814325, + 2.3543978050001897, + 2.5318525540060364, + 2.7093250149919186, + 2.8845588139956817, + 3.064166063995799, + 3.2378552189911716, + 3.412917513982393, + 3.5884236650017556, + 3.76014297499205, + 3.934566404001089, + 4.115888122993056, + 4.29812483699061, + 4.477417333982885, + 4.657588714006124, + 4.83613836200675, + 5.016386286995839, + 5.202168928983156, + 5.386225295980694, + 5.564000716985902, + 5.7428071500035, + 5.924395973997889, + 6.104715274996124, + 6.290413932991214, + 6.473598265001783, + 6.655751522979699, + 6.835295125987614, + 7.010762813995825, + 7.175907291006297, + 7.3558385749929585, + 7.538431225984823, + 7.715973782993387, + 7.896193963999394, + 8.076430960994912, + 8.261569057998713, + 8.448805049003568, + 8.634549276001053, + 8.82211427600123, + 9.002688409993425, + 9.187280857993755, + 9.372313964006025, + 9.55087604399887, + 9.733264438982587, + 9.914506190980319, + 10.098554124997463, + 10.286611654999433, + 10.469789027993102, + 10.648025938979117, + 10.829316010989714, + 11.007788315997459, + 11.189301011996577, + 11.377814670995576, + 11.555765486002201, + 11.738580073986668, + 11.910483610001393, + 12.082929675001651, + 12.261435180000262, + 12.439135802997043, + 12.62069091500598, + 12.797508975985693, + 12.97802877399954, + 13.162505846994463, + 13.342757067992352, + 13.521449533989653, + 13.699575453996658, + 13.880195476987865, + 14.067334283987293, + 14.247753265983192, + 14.428742681979202, + 14.608690734981792, + 14.787630304985214, + 14.971472751989495, + 15.151418548979564, + 15.331964555982267, + 15.510641214001225, + 15.693707114987774, + 15.873020893981447, + 16.047030196001288, + 16.223426559998188, + 16.400014427985298, + 16.57829343699268, + 16.759876247000648, + 16.93775567098055, + 17.111870444990927, + 17.292117331002373, + 17.468139214004623, + 17.648902469984023, + 17.832858596986625, + 18.00865919998614, + 18.18376756098587, + 18.36174567299895, + 18.544989174988586, + 18.720596337981988, + 18.898769827006618, + 19.076816610991955, + 19.228830751992064, + 19.362404365005204 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 179.42869100079406, + "p95": 185.7826419873163, + "max": 188.51365899899974 + } + }, + { + "start": 171805.172492559, + "first": 171805.389455922, + "last": 171824.640251007, + "end": 171824.640314559, + "elapsed_s": 19.467821999976877, + "chunk_offsets_s": [ + 0.21696336299646646, + 0.29446514797746204, + 0.6177659230015706, + 0.9623741289833561, + 1.331219613988651, + 1.4932681630016305, + 1.6566107779799495, + 1.8213850460015237, + 2.000157557980856, + 2.176862912980141, + 2.3545983340009116, + 2.5320589229813777, + 2.7096226159774233, + 2.8846760779852048, + 3.064450448990101, + 3.2376805469975807, + 3.413129673979711, + 3.588124320987845, + 3.760625712980982, + 3.934893668978475, + 4.116035627987003, + 4.298415030003525, + 4.477632486989023, + 4.6577563469763845, + 4.8363511149946135, + 5.01666995199048, + 5.2024564019811805, + 5.386439327994594, + 5.56421586999204, + 5.742979086993728, + 5.92467807099456, + 6.104999147995841, + 6.290697693999391, + 6.473881081998115, + 6.656034307990922, + 6.83558893497684, + 7.011176782980328, + 7.176387612998951, + 7.3560519839811604, + 7.5386439629946835, + 7.716257991996827, + 7.896406268991996, + 8.07675230599125, + 8.261717714980477, + 8.448980810004286, + 8.63484638198861, + 8.82240661297692, + 9.003019498981303, + 9.187564427003963, + 9.372655548999319, + 9.551071628986392, + 9.733686135994503, + 9.914668366982369, + 10.098852654977236, + 10.287022328004241, + 10.470003700000234, + 10.648423668986652, + 10.829711948987097, + 11.008215404988732, + 11.189724230003776, + 11.378226656990591, + 11.555980190983973, + 11.73968143798993, + 11.910895163979148, + 12.083141243987484, + 12.261717724992195, + 12.439310139976442, + 12.620885379990796, + 12.797926384984748, + 12.978598888003035, + 13.16267667998909, + 13.343039212981239, + 13.522170671989443, + 13.70003100699978, + 13.880657925998094, + 14.06777554997825, + 14.248036258999491, + 14.42894028298906, + 14.609104367991677, + 14.788047234993428, + 14.971879553981125, + 15.151703269977588, + 15.332177420990774, + 15.510923406982329, + 15.694138875987846, + 15.873359503981192, + 16.047197668987792, + 16.22383843298303, + 16.400295932980953, + 16.578491533990018, + 16.76028709599632, + 16.93817005600431, + 17.112197661976097, + 17.292328595998697, + 17.46833065498504, + 17.649365527002374, + 17.833141862996854, + 18.008833168976707, + 18.184173241985263, + 18.362028442003066, + 18.545202135981526, + 18.72094421100337, + 18.899180787993828, + 19.07711933198152, + 19.229138736991445, + 19.362685293977847, + 19.467758447979577 + ], + "chunk_gaps_ms": { + "count": 106, + "p50": 179.60949898406398, + "p95": 187.56023098831065, + "max": 368.8454850052949 + } + }, + { + "start": 171805.173166833, + "first": 171805.680082132, + "last": 171824.401409248, + "end": 171824.401825553, + "elapsed_s": 19.228658720006933, + "chunk_offsets_s": [ + 0.5069152989890426, + 0.6169773279980291, + 0.9615927030099556, + 1.3306681399990339, + 1.4925091199984308, + 1.6556757199869025, + 1.8203535869834013, + 1.9990291069843806, + 2.1757757419836707, + 2.3540285239869263, + 2.53110531999846, + 2.708746196993161, + 2.883856715983711, + 3.0634135179861914, + 3.2375256980012637, + 3.412352071987698, + 3.5881844329996966, + 3.7600987030018587, + 3.933867346990155, + 4.115048025007127, + 4.297943876008503, + 4.476838436996331, + 4.656808359985007, + 4.835528424999211, + 5.015719724993687, + 5.201564543996938, + 5.385663453984307, + 5.563269756006775, + 5.742106972000329, + 5.92387566898833, + 6.104205064999405, + 6.289784396009054, + 6.473055640002713, + 6.655109120998532, + 6.834952453005826, + 7.010055196005851, + 7.175375913997414, + 7.355057181004668, + 7.537879671988776, + 7.715151493001031, + 7.895437097991817, + 8.07551231898833, + 8.261319026001729, + 8.448136744002113, + 8.633749322005315, + 8.821575938985916, + 9.002094200986903, + 9.18671890499536, + 9.371502505993703, + 9.550191594986245, + 9.732631493010558, + 9.913734092988307, + 10.097880524001084, + 10.285971700999653, + 10.469058497983497, + 10.647515618009493, + 10.828576473984867, + 11.00725961101125, + 11.18872334700427, + 11.377147389983293, + 11.555077308992622, + 11.738422666006954, + 11.909984664991498, + 12.082350537995808, + 12.260778153984575, + 12.438880169996992, + 12.619764961011242, + 12.796957279002527, + 12.977425477001816, + 13.161898885009577, + 13.342235866992269, + 13.521421485987958, + 13.6989925079979, + 13.879450050997548, + 14.066473993996624, + 14.247092497011181, + 14.428111496003112, + 14.608097372984048, + 14.787106336007128, + 14.971105198987061, + 15.151176276995102, + 15.331685115001164, + 15.510105435998412, + 15.69310063298326, + 15.872425292996923, + 16.046807156002615, + 16.22331063999445, + 16.39949618699029, + 16.57804798099096, + 16.759268308989704, + 16.937091781001072, + 17.111677164997673, + 17.29150531400228, + 17.467474748002132, + 17.6483793160005, + 17.83233686798485, + 18.00829417500063, + 18.183430008008145, + 18.361058087990386, + 18.544706487009535, + 18.720070015988313, + 18.898404114006553, + 19.076596498984145, + 19.22824241500348 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 179.68126700725406, + "p95": 187.0239429990761, + "max": 369.0754369890783 + } + }, + { + "start": 171805.172433695, + "first": 171806.003500971, + "last": 171825.370521729, + "end": 171825.371013074, + "elapsed_s": 20.19857937900815, + "chunk_offsets_s": [ + 0.8310672760126181, + 0.962354065006366, + 1.3311794539913535, + 1.4935098749992903, + 1.6563529380073305, + 1.8211152850126382, + 1.999882420990616, + 2.176749872014625, + 2.3544280779897235, + 2.531973786011804, + 2.7098254640004598, + 2.884800975007238, + 3.064393120992463, + 3.237971906986786, + 3.4128601379925385, + 3.5879876959952526, + 3.7603563040029258, + 3.9346407249977347, + 4.116004748007981, + 4.298286725010257, + 4.47752947898698, + 4.657633947004797, + 4.836215658986475, + 5.0166247349989135, + 5.2024053299974184, + 5.386342607991537, + 5.564118142006919, + 5.742886221996741, + 5.92455509499996, + 6.1048690030002035, + 6.290661102015292, + 6.473731850011973, + 6.655792626988841, + 6.835495686013019, + 7.010846958000911, + 7.176391549000982, + 7.355924926989246, + 7.538558105996344, + 7.716030551993754, + 7.896215164015302, + 8.076432784990175, + 8.262000804010313, + 8.44927020301111, + 8.635148046014365, + 8.822360180987744, + 9.002729161991738, + 9.187521851010388, + 9.372190299996873, + 9.550870651990408, + 9.733406134997495, + 9.914527710992843, + 10.098522749991389, + 10.286765942990314, + 10.469952835992444, + 10.648186116013676, + 10.829245931003243, + 11.008208749000914, + 11.189844437991269, + 11.377932431991212, + 11.55587310300325, + 11.73820754600456, + 11.910482745995978, + 12.083012284012511, + 12.261302892002277, + 12.439199162996374, + 12.620628370990744, + 12.797748353012139, + 12.978097846993478, + 13.162547462998191, + 13.342766780988313, + 13.521881230990402, + 13.699664685991593, + 13.880245237000054, + 14.06729039599304, + 14.247879378992366, + 14.42892913101241, + 14.608884446992306, + 14.787676848995034, + 14.97147534400574, + 15.151597429998219, + 15.332113532989752, + 15.510587341996143, + 15.693659787008073, + 15.87321825500112, + 16.047051892004674, + 16.223621904995525, + 16.4000633090036, + 16.578412174014375, + 16.759836454992183, + 16.937889622990042, + 17.1119779330038, + 17.29203691499424, + 17.46826102299383, + 17.648895398015156, + 17.83288661399274, + 18.008701984013896, + 18.184322891000193, + 18.361954986001365, + 18.545064183999784, + 18.72063309000805, + 18.898776946996804, + 19.07702933199471, + 19.22901945700869, + 19.36260238100658, + 19.467738848004956, + 19.54567708299146, + 19.6142546540068, + 19.686264042014955, + 19.749190045986325, + 19.815263210999547, + 19.879736595001305, + 19.94388047500979, + 20.008596194995334, + 20.07390881399624, + 20.138257334008813, + 20.19808803399792 + ], + "chunk_gaps_ms": { + "count": 115, + "p50": 178.76807998982258, + "p95": 187.04515899298713, + "max": 368.8253889849875 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "9842a795db3725bcbafc8f42d161150739f454cd43900ccec1abc9933f88b41b", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "8f747a2ab0661a066944c7a7e450a84216ed89d2eddc301f4f310a4889d71764", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "2b7ba7cd6cfa2798ecac17a73f0c36973fadd7773d03d0172f748c5ae76867ec", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "2e4cb369c5155d59caa91a3f021437eeb23df358177a170c713bf4e5a40c94aa", + "output_sha256": "72c8213eefa4e151211840ebce85e09f240020fed0baf5b966c29b3b64be1bf8", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 56.1, + "per_stream_tps": 56.1, + "ttft_s_max": 0.15, + "timing_valid": true, + "end_to_end_tps": 54.517066718805744, + "makespan_s": 4.695777220011223, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 56.06326781915662, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 4.695777220011223, + "end_to_end_tps": 54.517066718805744, + "timing_valid": true, + "per_stream_tps": [ + 56.1 + ], + "ttft_s": [ + 0.15 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 171844.653672555, + "first": 171844.800844062, + "last": 171849.349276751, + "end": 171849.349449775, + "elapsed_s": 4.695777220011223, + "chunk_offsets_s": [ + 0.14717150700744241, + 0.221321702003479, + 0.29561844101408496, + 0.37382013400201686, + 0.45757822599262, + 0.5440617010171991, + 0.630610329011688, + 0.7144459889968857, + 0.7967233890085481, + 0.880843098013429, + 0.9620893429964781, + 1.043704998010071, + 1.1291613980138209, + 1.2111370369966608, + 1.2937035580107477, + 1.377690691006137, + 1.4601277559995651, + 1.5326202659925912, + 1.5970342750078999, + 1.660259065014543, + 1.727533705998212, + 1.8043612359906547, + 1.8796522500051651, + 1.9518217830045614, + 2.0267967480176594, + 2.1110730660147965, + 2.194085796014406, + 2.2745226309925783, + 2.353380118991481, + 2.457663961016806, + 2.5603576069988776, + 2.6667614380130544, + 2.7762876620108727, + 2.8804722880013287, + 2.9905201609944925, + 3.096612248016754, + 3.2080715330084786, + 3.318975488014985, + 3.4291779060149565, + 3.53802996801096, + 3.647904704994289, + 3.7548473860078957, + 3.8641418170009274, + 3.9743793870147783, + 4.084029499994358, + 4.1889303990174085, + 4.298407630994916, + 4.40551946501364, + 4.511845521017676, + 4.621921762009151, + 4.695604195992928 + ], + "chunk_gaps_ms": { + "count": 50, + "p50": 84.19801350100897, + "p95": 110.23757001385093, + "max": 111.45928499172442 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 64.0, + "per_stream_tps": 32.6, + "ttft_s_max": 0.36, + "timing_valid": true, + "end_to_end_tps": 63.06443732237079, + "makespan_s": 8.118680221989052, + "steady_tps": 65.4, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 63.974699641221314, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 8.118680221989052, + "end_to_end_tps": 63.06443732237079, + "timing_valid": true, + "steady_tps": 65.40188035078043, + "per_stream_tps": [ + 32.3, + 32.9 + ], + "ttft_s": [ + 0.15, + 0.36 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 171849.351251972, + "first": 171849.497791733, + "last": 171857.395915071, + "end": 171857.39597752, + "elapsed_s": 8.04472554801032, + "chunk_offsets_s": [ + 0.14653976101544686, + 0.22134655000991188, + 0.46989099599886686, + 0.577016316994559, + 0.6837458620138932, + 0.7920309150067624, + 0.9010418899997603, + 1.008128059009323, + 1.1134918080060743, + 1.2156724290107377, + 1.3131847820186522, + 1.4147284890059382, + 1.5200708780030254, + 1.6365599530108739, + 1.7482674620114267, + 1.8607425579975825, + 1.9758112450072076, + 2.0903930510103237, + 2.2056682980037294, + 2.3187974280153867, + 2.434625572001096, + 2.55129147900152, + 2.6645682569942437, + 2.7739744010032155, + 2.8855850950058084, + 2.9986830090056174, + 3.109907430014573, + 3.215207963017747, + 3.323507896013325, + 3.4315853320003953, + 3.539961858012248, + 3.647422092006309, + 3.7577308469917625, + 3.8676979360170662, + 3.977341937017627, + 4.090334138018079, + 4.204439847002504, + 4.312270243011881, + 4.423034422012279, + 4.537916197004961, + 4.652916980005102, + 4.762078339001164, + 4.874253034999128, + 4.989837835018989, + 5.103143493004609, + 5.217412530008005, + 5.333234067016747, + 5.448477778991219, + 5.562620751006762, + 5.676261435000924, + 5.791456058010226, + 5.905661799013615, + 6.019036001991481, + 6.130783528002212, + 6.244118898990564, + 6.358026719011832, + 6.471377448993735, + 6.585387094004545, + 6.7005646449979395, + 6.8142876170168165, + 6.9275980750098825, + 7.036672682996141, + 7.151365736994194, + 7.264803764002863, + 7.372298335016239, + 7.4845710470108315, + 7.5994624700106215, + 7.71000508099678, + 7.822188721009297, + 7.936376830009976, + 8.04466309901909 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 112.73364849330392, + "p95": 115.82814398570918, + "max": 248.54444598895498 + } + }, + { + "start": 171849.3511589, + "first": 171849.715747874, + "last": 171857.46969317, + "end": 171857.469839122, + "elapsed_s": 8.118680221989052, + "chunk_offsets_s": [ + 0.36458897398551926, + 0.4697665949934162, + 0.5769205889955629, + 0.6836979099898599, + 0.7920110269915313, + 0.9009906249993946, + 1.007968986988999, + 1.113420783978654, + 1.2156427009904291, + 1.3131518369773403, + 1.414388423989294, + 1.519940444995882, + 1.6365069119783584, + 1.7480119579995517, + 1.860639773978619, + 1.975655675982125, + 2.090360681992024, + 2.205608313990524, + 2.318738002999453, + 2.4344102749892045, + 2.5511795109778177, + 2.664385104988469, + 2.773829329002183, + 2.8853543899895158, + 2.998461775976466, + 3.1097266289871186, + 3.214986841980135, + 3.323300983000081, + 3.431400034984108, + 3.539914096996654, + 3.647356907982612, + 3.757687213976169, + 3.8676326400018297, + 3.9771966239786707, + 4.09024941798998, + 4.204387493984541, + 4.312069713982055, + 4.422957765986212, + 4.537796371994773, + 4.6526570909773, + 4.762050386983901, + 4.873923176986864, + 4.9897363939962815, + 5.103021844988689, + 5.21731625800021, + 5.333059138996759, + 5.448307105980348, + 5.562497310980689, + 5.676199833978899, + 5.791420265974011, + 5.90566226298688, + 6.018960224988405, + 6.130684807983926, + 6.24405248198309, + 6.357892077998258, + 6.4711623289913405, + 6.585172756982502, + 6.7004351089999545, + 6.814123055984965, + 6.927506458974676, + 7.036578233994078, + 7.15121970398468, + 7.264383554982487, + 7.372162670973921, + 7.484565286984434, + 7.599231732980115, + 7.709963192988653, + 7.82194559997879, + 7.936372269992717, + 8.04448197898455, + 8.118534269975498 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 112.5152159947902, + "p95": 115.7428809965495, + "max": 116.76923598861322 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 87.1, + "per_stream_tps": 22.6, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 86.3259241756314, + "makespan_s": 11.86202186398441, + "steady_tps": 90.7, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 87.0653518182059, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 11.86202186398441, + "end_to_end_tps": 86.3259241756314, + "timing_valid": true, + "steady_tps": 90.7152863242673, + "per_stream_tps": [ + 22.4, + 23.2, + 22.5, + 22.8 + ], + "ttft_s": [ + 0.15, + 0.89, + 0.37, + 0.62 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 171857.473008923, + "first": 171857.619502109, + "last": 171869.015751558, + "end": 171869.015835094, + "elapsed_s": 11.54282617100398, + "chunk_offsets_s": [ + 0.1464931860100478, + 0.22172887201304547, + 0.4696825339924544, + 0.7466948479996063, + 1.0461501980025787, + 1.2001888279919513, + 1.3534826720133424, + 1.5075963260023855, + 1.6598740720073693, + 1.80535976801184, + 1.9452573519956786, + 2.087294174998533, + 2.233683136990294, + 2.393848423991585, + 2.560647151985904, + 2.7299185740121175, + 2.8925406349881086, + 3.060168213007273, + 3.222624721995089, + 3.3885427440109197, + 3.556860148004489, + 3.7132450719946064, + 3.879843544011237, + 4.04777880301117, + 4.2087172440078575, + 4.367295214004116, + 4.525836640998023, + 4.681705211987719, + 4.837120807002066, + 4.990532972005894, + 5.138005040993448, + 5.291543478000676, + 5.449529542995151, + 5.604009823000524, + 5.759174953011097, + 5.916668775986182, + 6.075177883001743, + 6.22998360398924, + 6.385924431990134, + 6.546789304004051, + 6.707788722007535, + 6.8697857889928855, + 7.030896054988261, + 7.19220206499449, + 7.349647264985833, + 7.511395051988075, + 7.679178054997465, + 7.84405148299993, + 8.00876209398848, + 8.170537064987002, + 8.335135116009042, + 8.49680900000385, + 8.655505467002513, + 8.816972053988138, + 8.978400527994381, + 9.137235203990713, + 9.298167069006013, + 9.461673276993679, + 9.625091021007393, + 9.783037599001545, + 9.94441623298917, + 10.104804064991185, + 10.264131701987935, + 10.424261950014625, + 10.584459189005429, + 10.745685903995764, + 10.902111661009258, + 11.061622642999282, + 11.224237601010827, + 11.384531465009786, + 11.542742635007016 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.2455514948815, + "p95": 169.27142202621326, + "max": 299.4553500029724 + } + }, + { + "start": 171857.473768125, + "first": 171858.363943399, + "last": 171869.334839826, + "end": 171869.334934402, + "elapsed_s": 11.86116627699812, + "chunk_offsets_s": [ + 0.8901752739911899, + 1.0460341169964522, + 1.1996974179928657, + 1.3530216790095437, + 1.5071593330067117, + 1.6586463889980223, + 1.8047861340164673, + 1.9448972239915747, + 2.0869488620082848, + 2.2333329920074902, + 2.3934732069901656, + 2.5601926379895303, + 2.7295322050049435, + 2.8916354330140166, + 3.059122547012521, + 3.222215296991635, + 3.388286806992255, + 3.556482643005438, + 3.712968464009464, + 3.8794010629935656, + 4.047513122990495, + 4.208067817991832, + 4.366938029997982, + 4.525496287998976, + 4.681220778991701, + 4.836788406013511, + 4.990269786998397, + 5.137669232004555, + 5.291001733014127, + 5.449242917995434, + 5.60366708599031, + 5.758784744015429, + 5.916131222998956, + 6.0747227300016675, + 6.229708227008814, + 6.3854270380106755, + 6.546378391009057, + 6.707415840995964, + 6.869335563998902, + 7.0306529980152845, + 7.1909642860118765, + 7.349199727992527, + 7.511019355006283, + 7.678724853991298, + 7.843711705994792, + 8.008415276999585, + 8.17031528899679, + 8.334791722998489, + 8.496414374996675, + 8.655190730001777, + 8.816427156009013, + 8.978062398993643, + 9.136963330995059, + 9.297774940001545, + 9.461023835989181, + 9.62444618801237, + 9.782547709997743, + 9.943772040016484, + 10.104358848009724, + 10.263659445015946, + 10.423860748996958, + 10.583852020004997, + 10.745102542015957, + 10.901807740010554, + 11.06122651399346, + 11.223924288002308, + 11.384306071995525, + 11.542460043012397, + 11.678134257002966, + 11.787092401995324, + 11.861071701016044 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 159.7050224954728, + "p95": 167.70549898501486, + "max": 169.3395670154132 + } + }, + { + "start": 171857.473337692, + "first": 171857.839705873, + "last": 171869.151499277, + "end": 171869.151564494, + "elapsed_s": 11.678226802003337, + "chunk_offsets_s": [ + 0.366368180984864, + 0.46966034098295495, + 0.7469408969918732, + 1.0462504540046211, + 1.2002625399909448, + 1.353620799985947, + 1.5077426630014088, + 1.6597056229948066, + 1.8053392069996335, + 1.9453625849855598, + 2.0873446709883865, + 2.2337920169811696, + 2.3939794639882166, + 2.560653967986582, + 2.7299283019965515, + 2.8922690019826405, + 3.0596173320082016, + 3.2226098100072704, + 3.3885919119929895, + 3.5570626439875923, + 3.713432577002095, + 3.879866151983151, + 4.047866354987491, + 4.208536235004431, + 4.367448047007201, + 4.5259611689834855, + 4.681696891988395, + 4.837296423007501, + 4.99060853998526, + 5.13820092898095, + 5.291482422006084, + 5.449741911987076, + 5.604061535006622, + 5.759250105009414, + 5.916615672002081, + 6.075337546993978, + 6.2301018919970375, + 6.385894687002292, + 6.5468437999952585, + 6.707978257996729, + 6.86980004500947, + 7.030891478992999, + 7.191590975009603, + 7.349671760981437, + 7.51149682799587, + 7.679223526996793, + 7.844176571001299, + 8.008778797986452, + 8.170710970007349, + 8.33525767698302, + 8.496737174980808, + 8.655407865997404, + 8.8163875080063, + 8.978315583983203, + 9.13715530000627, + 9.298105036985362, + 9.461103307985468, + 9.624605035991408, + 9.782864270004211, + 9.943839975981973, + 10.10451529599959, + 10.26384152600076, + 10.423969869007124, + 10.58388286799891, + 10.74511006200919, + 10.901818220008863, + 11.061331201985013, + 11.223953167995205, + 11.38424405598198, + 11.542482505989028, + 11.678161585005 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.15789500670508, + "p95": 168.47073199460283, + "max": 299.309557012748 + } + }, + { + "start": 171857.472912538, + "first": 171858.088092592, + "last": 171869.260810542, + "end": 171869.260927679, + "elapsed_s": 11.788015140977222, + "chunk_offsets_s": [ + 0.615180053981021, + 0.7471955379878636, + 1.0469860729936045, + 1.200487580994377, + 1.3538306260015815, + 1.508329624979524, + 1.6594481669890229, + 1.8057220409973525, + 1.9457172580005135, + 2.0878406729898416, + 2.2341550269920845, + 2.3943733219930436, + 2.5609943689778447, + 2.7302983039990067, + 2.892549883981701, + 3.060354727000231, + 3.2231213479826692, + 3.389178297977196, + 3.5574225499876775, + 3.713920946989674, + 3.880321753997123, + 4.048455349984579, + 4.209096589998808, + 4.367951152991736, + 4.526416402979521, + 4.682156750001013, + 4.837794968974777, + 4.991165005980292, + 5.138558418984758, + 5.291983703995356, + 5.4501323769800365, + 5.604557457001647, + 5.759743546979735, + 5.91707372199744, + 6.0756152129906695, + 6.230598661990371, + 6.386355280992575, + 6.547299673984526, + 6.708357075985987, + 6.870262718992308, + 7.031351768993773, + 7.191971024993109, + 7.350131554994732, + 7.511955005989876, + 7.6796161370002665, + 7.844632444990566, + 8.009237935999408, + 8.171055004000664, + 8.335612845985452, + 8.49723292098497, + 8.656082044995856, + 8.817332710983464, + 8.978882961993804, + 9.13790954198339, + 9.298665406997316, + 9.46191311898292, + 9.625337566976668, + 9.783439056976931, + 9.944663467002101, + 10.10525139499805, + 10.26446104797651, + 10.424651263980195, + 10.584743190993322, + 10.745993552991422, + 10.902626670984318, + 11.062045653001405, + 11.224842434981838, + 11.385126698995009, + 11.5434100299899, + 11.678950387984514, + 11.787898003996816 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 160.14107150840573, + "p95": 168.13359598745592, + "max": 299.7905350057408 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 30.6, + "per_stream_tps": 30.6, + "ttft_s_max": 0.22, + "timing_valid": true, + "end_to_end_tps": 29.915635388802567, + "makespan_s": 8.55739805198391, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 30.57411534977436, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 8.55739805198391, + "end_to_end_tps": 29.915635388802567, + "timing_valid": true, + "per_stream_tps": [ + 30.6 + ], + "ttft_s": [ + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 171892.68844584, + "first": 171892.905391218, + "last": 171901.245779716, + "end": 171901.245843892, + "elapsed_s": 8.55739805198391, + "chunk_offsets_s": [ + 0.21694537799339741, + 0.2941394549852703, + 0.3703607459901832, + 0.4493322199850809, + 0.5323626649915241, + 0.6175035480118822, + 0.6989895410079043, + 0.7799559329869226, + 0.8597378889971878, + 0.941496059007477, + 1.0159566249931231, + 1.0873655189934652, + 1.1512559609836899, + 1.2156558930000756, + 1.2801031520066317, + 1.3437798979866784, + 1.4110093729977962, + 1.4754212239931803, + 1.5389892339881044, + 1.6044156959978864, + 1.6774222099920735, + 1.753599021001719, + 1.8285823249898385, + 1.9000695069844369, + 1.963024858996505, + 2.028812009986723, + 2.105215029005194, + 2.1777977510064375, + 2.2524499020073563, + 2.3274821329978295, + 2.4100585609849077, + 2.48830584200914, + 2.5524245889973827, + 2.6245640449924394, + 2.699690837005619, + 2.7655631240049843, + 2.831040307006333, + 2.89988290600013, + 2.9758498439914547, + 3.049416232010117, + 3.124450639996212, + 3.203303026006324, + 3.2844840270117857, + 3.35050159500679, + 3.4217135919898283, + 3.48627153999405, + 3.5507609919877723, + 3.6180447229999118, + 3.6941209899960086, + 3.7713022519892547, + 3.8455246099911164, + 3.9088274980022106, + 3.971109311998589, + 4.0336534150119405, + 4.098047058010707, + 4.162341869989177, + 4.228305741009535, + 4.29432020499371, + 4.359488507005153, + 4.424991721985862, + 4.489150404988322, + 4.552554366004188, + 4.616418663994409, + 4.6801727220008615, + 4.7444394690101035, + 4.810196251986781, + 4.8734935089887585, + 4.937741744011873, + 5.004656962992158, + 5.079895355011104, + 5.157087208994199, + 5.235863290989073, + 5.315454191004392, + 5.402760424010921, + 5.4835958069888875, + 5.566162619012175, + 5.64816879000864, + 5.7382403099909425, + 5.822346166009083, + 5.891469246998895, + 5.955458329000976, + 6.0180518880079035, + 6.081615689006867, + 6.1462676860101055, + 6.21077805000823, + 6.275830383005086, + 6.342182224005228, + 6.412315146997571, + 6.482783702987945, + 6.546908577991417, + 6.6121576959849335, + 6.6780604959931225, + 6.751043058000505, + 6.823176050995244, + 6.897475304984255, + 6.9692563120042905, + 7.031942319008522, + 7.097500317991944, + 7.161545993003529, + 7.226088036986766, + 7.292978935991414, + 7.367570973990951, + 7.443061767990002, + 7.515258920000633, + 7.5889782209997065, + 7.664654950989643, + 7.7480346290103626, + 7.8322708059859, + 7.920900146011263, + 8.002193338994402, + 8.08422152599087, + 8.166611154010752, + 8.250025087996619, + 8.337102952995338, + 8.421086793008726, + 8.497326612006873, + 8.557333876000484 + ], + "chunk_gaps_ms": { + "count": 116, + "p50": 71.63409450731706, + "p95": 84.23617697553709, + "max": 90.07151998230256 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 43.8, + "per_stream_tps": 22.4, + "ttft_s_max": 0.5, + "timing_valid": true, + "end_to_end_tps": 43.169398505131866, + "makespan_s": 11.860253274993738, + "steady_tps": 44.8, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 43.800592273428066, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 11.860253274993738, + "end_to_end_tps": 43.169398505131866, + "timing_valid": true, + "steady_tps": 44.75673057626815, + "per_stream_tps": [ + 22.2, + 22.5 + ], + "ttft_s": [ + 0.22, + 0.5 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 171901.24719964, + "first": 171901.463336617, + "last": 171912.968193518, + "end": 171912.968347038, + "elapsed_s": 11.72114739799872, + "chunk_offsets_s": [ + 0.21613697698921897, + 0.29230502000427805, + 0.6111178860010114, + 0.7242627160158008, + 0.8333924959879369, + 0.9443360079894774, + 1.062006210006075, + 1.1791370020073373, + 1.291738086991245, + 1.4066003609914333, + 1.521847436990356, + 1.634529258008115, + 1.7519948669942096, + 1.865036256989697, + 1.9838541740027722, + 2.105228274012916, + 2.2259774600097444, + 2.3443450079939794, + 2.4626945560157765, + 2.5803970780107193, + 2.6989533150044736, + 2.8179839049989823, + 2.9389493000053335, + 3.0573184000095353, + 3.180668232991593, + 3.297097392001888, + 3.412026723002782, + 3.528462218004279, + 3.6434717570082285, + 3.7563864900148474, + 3.8798334280145355, + 4.002578812011052, + 4.126649446989177, + 4.24788057099795, + 4.367753642989555, + 4.48234126099851, + 4.6016376439947635, + 4.7214907800080255, + 4.83710040099686, + 4.954823370993836, + 5.07119349000277, + 5.191279891005252, + 5.311033619014779, + 5.426856120000593, + 5.548102027998539, + 5.6615795149991754, + 5.780373896996025, + 5.90421166800661, + 6.022424383991165, + 6.142945666011656, + 6.261771359015256, + 6.379603850014973, + 6.492852136987494, + 6.611234133015387, + 6.728280045994325, + 6.837613650015555, + 6.9462622120045125, + 7.063159932993585, + 7.184160959994188, + 7.306563783000456, + 7.432273718994111, + 7.55421322901384, + 7.674060366000049, + 7.783236082002986, + 7.905227079987526, + 8.025724122009706, + 8.146524253010284, + 8.266981519002002, + 8.378423640999245, + 8.497642375004943, + 8.613900141994236, + 8.732821003999561, + 8.850190021999879, + 8.967368832003558, + 9.078502473013941, + 9.187588908011094, + 9.304451764997793, + 9.420789739990141, + 9.535683038993739, + 9.651279381010681, + 9.762746159016388, + 9.87997970401193, + 9.996411631989758, + 10.114797387999715, + 10.233033992990386, + 10.349592000013217, + 10.462694638990797, + 10.5797338479897, + 10.689039355987916, + 10.809822623996297, + 10.927233482012525, + 11.04546371800825, + 11.157831682998221, + 11.275941791012883, + 11.390191712009255, + 11.505856005998794, + 11.620353480015183, + 11.720993877999717 + ], + "chunk_gaps_ms": { + "count": 97, + "p50": 117.4108580162283, + "p95": 123.44693799968809, + "max": 318.8128659967333 + } + }, + { + "start": 171901.246997175, + "first": 171901.751094648, + "last": 171913.107014785, + "end": 171913.10725045, + "elapsed_s": 11.860253274993738, + "chunk_offsets_s": [ + 0.5040974729927257, + 0.6114992639923003, + 0.7245835019857623, + 0.8334319999848958, + 0.9442863279837184, + 1.0620916499756277, + 1.179503979976289, + 1.2921882639930118, + 1.4066837379941717, + 1.5221616459893994, + 1.6349363469926175, + 1.7520321479823906, + 1.8652788499894086, + 1.9838892300031148, + 2.1055247389886063, + 2.2260200049786363, + 2.3445824169903062, + 2.4628627809870522, + 2.5805545350012835, + 2.6991196839953773, + 2.8181516179756727, + 2.9390702599775977, + 3.0575568490021396, + 3.1808356739929877, + 3.297263344982639, + 3.4122638760018162, + 3.5287010189786088, + 3.643638989975443, + 3.7566713400010485, + 3.8799577329773456, + 4.002743644989096, + 4.1268897999834735, + 4.248120523989201, + 4.367922251985874, + 4.482506573986029, + 4.601874764979584, + 4.72165825299453, + 4.837259249994531, + 4.954931739979656, + 5.071429075003834, + 5.191427107987693, + 5.311269395984709, + 5.427023368974915, + 5.548271452978952, + 5.661816091975197, + 5.780487865995383, + 5.904352660989389, + 6.02234417598811, + 6.143008049984928, + 6.262060991983162, + 6.380105307995109, + 6.493377321981825, + 6.6115568059904035, + 6.7287168789771385, + 6.838290931977099, + 6.946870853978908, + 7.06385286297882, + 7.184662897983799, + 7.307257945998572, + 7.432816808985081, + 7.554899871989619, + 7.674342206999427, + 7.78375685098581, + 7.905770169978496, + 8.026217179984087, + 8.147138574975543, + 8.267522736976389, + 8.378905833989847, + 8.498124184989138, + 8.61441097597708, + 8.733380685996963, + 8.850717239984078, + 8.967896784975892, + 9.079062906996114, + 9.188159373996314, + 9.305165047000628, + 9.421458621975034, + 9.536353809991851, + 9.651606053987052, + 9.763338896998903, + 9.880534073978197, + 9.996925632993225, + 10.115506846981589, + 10.233728218998294, + 10.350187921983888, + 10.463345376978396, + 10.580467177991522, + 10.689693070977228, + 10.810362160991644, + 10.92770778699196, + 11.0459520079894, + 11.158154707984067, + 11.276628880994394, + 11.390865234978264, + 11.50623275898397, + 11.620880777976708, + 11.721735719998833, + 11.7979708359926, + 11.860017609986244 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 117.26586548320483, + "p95": 123.27882499084808, + "max": 125.55886298650876 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.1, + "per_stream_tps": 15.3, + "ttft_s_max": 1.17, + "timing_valid": true, + "end_to_end_tps": 58.61437836659613, + "makespan_s": 17.470116181997582, + "steady_tps": 61.9, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.11658510668523, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 17.470116181997582, + "end_to_end_tps": 58.61437836659613, + "timing_valid": true, + "steady_tps": 61.85188847764934, + "per_stream_tps": [ + 15.2, + 15.4, + 15.6, + 15.1 + ], + "ttft_s": [ + 0.51, + 0.82, + 1.17, + 0.22 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "client_intervals": [ + { + "start": 171913.109565304, + "first": 171913.615171617, + "last": 171930.429910427, + "end": 171930.429967852, + "elapsed_s": 17.32040254797903, + "chunk_offsets_s": [ + 0.5056063130032271, + 0.6119524219830055, + 0.9612295959959738, + 1.3349588209821377, + 1.4987125709885731, + 1.6615721109847073, + 1.8210820099920966, + 1.9885464749822859, + 2.161534489976475, + 2.322205287986435, + 2.4828951739764307, + 2.647654494998278, + 2.8092855360009708, + 2.9777644999849144, + 3.1413379139848985, + 3.3130470780015457, + 3.4893644299882, + 3.670216401980724, + 3.842997824976919, + 4.016867826983798, + 4.189580112986732, + 4.363642979995348, + 4.53193293500226, + 4.70907300899853, + 4.8856175459804945, + 5.060096670000348, + 5.22406205299194, + 5.388579181977548, + 5.554355369997211, + 5.727404409990413, + 5.896696207986679, + 6.075021053984528, + 6.256409891997464, + 6.437252503994387, + 6.606371469999431, + 6.779630573990289, + 6.949052531999769, + 7.117798344988842, + 7.2881194569927175, + 7.451971832982963, + 7.621938672004035, + 7.793670572980773, + 7.962389968975913, + 8.133561611990444, + 8.302416833001189, + 8.471260453981813, + 8.640107771003386, + 8.811325589980697, + 8.981568349990994, + 9.14863942298689, + 9.321952174999751, + 9.48827639798401, + 9.654833115986548, + 9.81891056499444, + 9.980688743002247, + 10.14397172498866, + 10.298168569977861, + 10.455038542975672, + 10.625605576991802, + 10.80111643197597, + 10.978849756997079, + 11.162618921982357, + 11.334700823994353, + 11.511901299003512, + 11.68315841499134, + 11.858022500004154, + 12.04096705498523, + 12.21638502998394, + 12.389817015995504, + 12.559387630986748, + 12.728674852987751, + 12.89788466799655, + 13.068550693977159, + 13.238628414983395, + 13.402875671978109, + 13.564477768988581, + 13.72771351999836, + 13.896789845981402, + 14.064637400995707, + 14.223086017998867, + 14.379634070995962, + 14.541255049000029, + 14.708500649983762, + 14.880954728985671, + 15.049753278988646, + 15.219023764977464, + 15.385411859984742, + 15.540064499975415, + 15.709712298994418, + 15.87778716700268, + 16.052889365004376, + 16.229550463991472, + 16.404219748976175, + 16.57108054999844, + 16.74038450900116, + 16.9045363740006, + 17.05981578299543, + 17.19899690899183, + 17.320345122978324 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 169.24015049880836, + "p95": 181.3888380129356, + "max": 373.7292249861639 + } + }, + { + "start": 171913.110138009, + "first": 171913.934142935, + "last": 171930.519901949, + "end": 171930.520051086, + "elapsed_s": 17.40991307699005, + "chunk_offsets_s": [ + 0.8240049259911757, + 0.9601282980001997, + 1.3339368509768974, + 1.4978480729914736, + 1.6606169889855664, + 1.8202850009838585, + 1.9877862489956897, + 2.1605489999928977, + 2.32136482198257, + 2.482073044986464, + 2.6468384139880072, + 2.8083985739795025, + 2.9768961939844303, + 3.1405640399898402, + 3.3120585629949346, + 3.4890407969942316, + 3.6692230560001917, + 3.842290911998134, + 4.016039616981288, + 4.188915471982909, + 4.362746097991476, + 4.531160660990281, + 4.708294511976419, + 4.884904712991556, + 5.059301419998519, + 5.223222706990782, + 5.387872524996055, + 5.553578120976454, + 5.726778008975089, + 5.895685836992925, + 6.074093515984714, + 6.255687489989214, + 6.436552022991236, + 6.605508235981688, + 6.778925980994245, + 6.948352162988158, + 7.117111895000562, + 7.287315919995308, + 7.45133904699469, + 7.621098638977855, + 7.792903834983008, + 7.961563167977147, + 8.132731338992016, + 8.30157916698954, + 8.470473683992168, + 8.639260856987676, + 8.811054821999278, + 8.98085514898412, + 9.147769868985051, + 9.32111999799963, + 9.487416459975066, + 9.654070363001665, + 9.818668243999127, + 9.980290085979505, + 10.143463211978087, + 10.297494280996034, + 10.454386029974557, + 10.624847430997761, + 10.8003015339782, + 10.978041834983742, + 11.161904648994096, + 11.334340054978384, + 11.511107345984783, + 11.682746141974349, + 11.857203010993544, + 12.040181661985116, + 12.215652004990261, + 12.389071173995035, + 12.558652380976127, + 12.728159060992766, + 12.89715445798356, + 13.06823578197509, + 13.237891452998156, + 13.40224355799728, + 13.564022744976683, + 13.726824126002612, + 13.896037172002252, + 14.063819510978647, + 14.222318257001461, + 14.378920596995158, + 14.540566998999566, + 14.707773335976526, + 14.880011303001083, + 15.04877726000268, + 15.218299763975665, + 15.38468595498125, + 15.539320369978668, + 15.708875801996328, + 15.87696732499171, + 16.05242190900026, + 16.22869370999979, + 16.403357283998048, + 16.57032913999865, + 16.73969545098953, + 16.903766003990313, + 17.059199429990258, + 17.198335307999514, + 17.319634482002584, + 17.40976393999881 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 168.97580499062315, + "p95": 180.86453300202265, + "max": 373.8085529766977 + } + }, + { + "start": 171913.110046777, + "first": 171914.283077901, + "last": 171930.579502174, + "end": 171930.579590078, + "elapsed_s": 17.469543301005615, + "chunk_offsets_s": [ + 1.1730311239953153, + 1.334085234993836, + 1.497892248997232, + 1.6606560449872632, + 1.8204036729875952, + 1.9878180410014465, + 2.160697879997315, + 2.321402086003218, + 2.4821101959969383, + 2.6472974389907904, + 2.808539727004245, + 2.9770465299952775, + 3.1409985850041267, + 3.3124068840115797, + 3.4887451799877454, + 3.6692224479920696, + 3.842333406995749, + 4.0161820169887505, + 4.188950815994758, + 4.3629181619908195, + 4.531298804999096, + 4.70832571099163, + 4.885035176994279, + 5.059333228011383, + 5.223355699010426, + 5.387910076009575, + 5.553708760999143, + 5.726586728007533, + 5.895530173002044, + 6.074476556008449, + 6.255526658002054, + 6.436467415012885, + 6.60576207601116, + 6.778859003999969, + 6.948222578008426, + 7.117022518999875, + 7.28745113601326, + 7.451575368002523, + 7.621237966988701, + 7.792812506988412, + 7.961709247989347, + 8.13287751498865, + 8.301720366987865, + 8.470613476005383, + 8.63945704899379, + 8.810735909006326, + 8.980998829007149, + 9.14790807699319, + 9.321263341989834, + 9.487567708012648, + 9.65421989900642, + 9.818633540009614, + 9.980328325997107, + 10.143670060002478, + 10.297626632993342, + 10.454228813003283, + 10.625015880010324, + 10.800453678006306, + 10.97818601099425, + 11.162048920989037, + 11.334082598012174, + 11.51125077001052, + 11.68237105300068, + 11.857336146989837, + 12.040318126004422, + 12.21579877298791, + 12.38923520600656, + 12.558798157988349, + 12.728046131989686, + 12.897298121999484, + 13.067926421004813, + 13.23803862099885, + 13.402248966012849, + 13.563930231990525, + 13.726852813997539, + 13.896186805010075, + 14.063969734997954, + 14.222457808995387, + 14.37906016499619, + 14.540596198989078, + 14.707739031990059, + 14.880164839007193, + 15.048934541002382, + 15.21844249998685, + 15.38482928299345, + 15.53947165000136, + 15.709033849998377, + 15.877117133000866, + 16.052198996010702, + 16.228735134005547, + 16.40363936399808, + 16.570482916984474, + 16.739718107011868, + 16.903926180995768, + 17.059245829994325, + 17.19837334001204, + 17.319783618004294, + 17.40989931600052, + 17.46945539698936 + ], + "chunk_gaps_ms": { + "count": 98, + "p50": 169.08931751095224, + "p95": 180.4772680043243, + "max": 183.8629099947866 + } + }, + { + "start": 171913.109473896, + "first": 171913.32546089, + "last": 171930.169064142, + "end": 171930.169403247, + "elapsed_s": 17.05992935100221, + "chunk_offsets_s": [ + 0.21598699397873133, + 0.29244233400095254, + 0.6122186619904824, + 0.9610464919824153, + 1.3347888680000324, + 1.498633130977396, + 1.6613946539873723, + 1.8213913389772642, + 1.9887311949860305, + 2.161385880986927, + 2.322127767984057, + 2.4828980379970744, + 2.6477026869833935, + 2.8094748169824015, + 2.977959540003212, + 3.1413952260045335, + 3.3132975899789017, + 3.4895190380048007, + 3.6702190100040752, + 3.843127361004008, + 4.016881938005099, + 4.189639472984709, + 4.363766227994347, + 4.532184487004997, + 4.70937843399588, + 4.8857441059954, + 5.060228013986489, + 5.224229588988237, + 5.388830701995175, + 5.554480745980982, + 5.72729692098801, + 5.896579391002888, + 6.075147069990635, + 6.256542819988681, + 6.437309623986948, + 6.606430621992331, + 6.779688861977775, + 6.949110611982178, + 7.1178535129874945, + 7.2883685779816005, + 7.452356377005344, + 7.622195264993934, + 7.793953660991974, + 7.962442768999608, + 8.133851741004037, + 8.302667120995466, + 8.471447797986912, + 8.64024476299528, + 8.811596693994943, + 8.981704270991031, + 9.148671885981457, + 9.321989902993664, + 9.488336252979934, + 9.654890299978433, + 9.819079300999874, + 9.980713653989369, + 10.14420601297752, + 10.298307129996829, + 10.455172942980425, + 10.625739529001294, + 10.801148159982404, + 10.979027308989316, + 11.16275224997662, + 11.334832311986247, + 11.512079235981219, + 11.683162526984233, + 11.858171300002141, + 12.041099711001152, + 12.216520310001215, + 12.38995103200432, + 12.559522575000301, + 12.728882805997273, + 12.898018459993182, + 13.068697301991051, + 13.238763774977997, + 13.40300943198963, + 13.564615305978805, + 13.727622926991899, + 13.896923285996309, + 14.0648173049849, + 14.223219985986361, + 14.379768407001393, + 14.541390648984816, + 14.708634249982424, + 14.880857848998858, + 15.04964993399335, + 15.21915766899474, + 15.385557859990513, + 15.540199427981861, + 15.709896347980248, + 15.877803278999636, + 16.053022790001705, + 16.22968454399961, + 16.40435362199787, + 16.571216693992028, + 16.74052090899204, + 16.90467147799791, + 17.059590246004518 + ], + "chunk_gaps_ms": { + "count": 97, + "p50": 169.30421500001103, + "p95": 182.92841099901125, + "max": 373.74237601761706 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0c5820aa82864eb281f80dffc6cd34fb865b2bf4aeea61128c8680a088de784e", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2", + "complete": true, + "error": null + } + } + ] + } + ] + } + ] +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/rows-without-bodies.jsonl b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/rows-without-bodies.jsonl new file mode 100644 index 0000000000..fe80d2917e --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/rows-without-bodies.jsonl @@ -0,0 +1,44 @@ +{"i": 0, "conv": "S1", "turn": 1, "arm": "drafted", "group": "", "history": 7757, "allowed": [0], "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 6.024408429, "prefill": 6.023097449, "wall": 8.0445601560059, "finish": "tool_calls"}, "ok": true, "body_sha256": "c89863023849e1c523580f91d6e3e557e187ee1345ec141afe8843f12f6a48bb", "body_bytes": 29036} +{"i": 1, "conv": "S1", "turn": 1, "arm": "plain", "group": "", "history": 7757, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 6144, "prompt": 7759, "tokens": 61, "ttft": 1.246543454, "prefill": 1.244847561, "wall": 4.623905612010276, "finish": "tool_calls"}, "ok": true, "body_sha256": "990b29e03db5234f33dad6e1fa5929871234fa91c7f57ae49f27a1167d79423a", "body_bytes": 29050} +{"i": 2, "conv": "S1", "turn": 2, "arm": "drafted", "group": "", "history": 15548, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 6144, "prompt": 15550, "tokens": 44, "ttft": 7.198018767, "prefill": 7.195166152, "wall": 8.265734003012767, "finish": "tool_calls"}, "ok": true, "body_sha256": "c0511b124f040010c106a9b9c3cfe60551382dd45952ffe91ec6fbda23d3f48b", "body_bytes": 52453} +{"i": 3, "conv": "S1", "turn": 2, "arm": "plain", "group": "", "history": 15548, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "1a31202ffe117d299e2106c6b51abb936406f609418facc8fd41d8cc01125427", "cached": 14336, "prompt": 15550, "tokens": 44, "ttft": 1.025149073, "prefill": 1.02202169, "wall": 3.464529955002945, "finish": "tool_calls"}, "ok": true, "body_sha256": "58ef4cc1eca46f9fdfb06a81864b78d2bf41cf6ce7b23584244099c44386a03b", "body_bytes": 52467} +{"i": 4, "conv": "S1", "turn": 3, "arm": "drafted", "group": "", "history": 19458, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 14336, "prompt": 19460, "tokens": 80, "ttft": 4.056150427, "prefill": 4.052929235, "wall": 6.64602538998588, "finish": "tool_calls"}, "ok": true, "body_sha256": "11ac960dd8b2c38fca6c46dbdc52b012fd87dcd782a5fd82762ce2806fde1000", "body_bytes": 64230} +{"i": 5, "conv": "S1", "turn": 3, "arm": "plain", "group": "", "history": 19458, "allowed": {"min": 11266, "max": 19458}, "got": {"sha": "bdf4933d2070c922abf56ad18b3124551eddf16798bd0117a132b7728fe40604", "cached": 18432, "prompt": 19460, "tokens": 80, "ttft": 0.925230377, "prefill": 0.921311615, "wall": 5.3547012290218845, "finish": "tool_calls"}, "ok": true, "body_sha256": "9b4ca8e9eda4d29d0ae73abca90bb51c7e19941eed492d68c377e3937d322ba8", "body_bytes": 64244} +{"i": 6, "conv": "S1", "turn": 4, "arm": "drafted", "group": "", "history": 21872, "allowed": {"min": 11266, "max": 19458}, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 18432, "prompt": 21874, "tokens": 148, "ttft": 2.726187425, "prefill": 2.722477287, "wall": 7.868106662994251, "finish": "tool_calls"}, "ok": true, "body_sha256": "4bdad5a3819a73f504b932d6176b24caca6c00139b6caa8260616c133202b938", "body_bytes": 76137} +{"i": 7, "conv": "S1", "turn": 4, "arm": "plain", "group": "", "history": 21872, "allowed": {"min": 13680, "max": 21872}, "got": {"sha": "5ce5e0e878d6ceb54ab8756f4a7392163f4b46a0528917b9c9cfaf4d74aa74f8", "cached": 20480, "prompt": 21874, "tokens": 148, "ttft": 1.117225035, "prefill": 1.11274435, "wall": 9.374758815014502, "finish": "tool_calls"}, "ok": true, "body_sha256": "ba4cd10cb59c177f50bcc55c4979fcd90f4a299fb422324e283033fed5e96461", "body_bytes": 76151} +{"i": 8, "conv": "S1", "turn": 5, "arm": "drafted", "group": "", "history": 26784, "allowed": {"min": 13680, "max": 21872}, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 20480, "prompt": 26786, "tokens": 160, "ttft": 5.098646204, "prefill": 5.093373518, "wall": 10.38785505501437, "finish": "length"}, "ok": true, "body_sha256": "3bcbd86cb518f49b9a1aaaa4db8876da914e3dd96fd531d05a78eca97f7a20c9", "body_bytes": 99864} +{"i": 9, "conv": "S1", "turn": 5, "arm": "plain", "group": "", "history": 26784, "allowed": {"min": 18592, "max": 26784}, "got": {"sha": "140ea4edfd3ec1903d827da3e3074b7db3a89de338191117fb18db2f3c83bf6c", "cached": 26624, "prompt": 26786, "tokens": 160, "ttft": 0.428106752, "prefill": 0.422331632, "wall": 9.332577224995475, "finish": "length"}, "ok": true, "body_sha256": "f0abf2c7c0aeba8588f8e7a4bb0460c6b7e46a6333520d7783b719b06ba81ea8", "body_bytes": 99878} +{"i": 10, "conv": "S1", "turn": 6, "arm": "drafted", "group": "", "history": 29291, "allowed": {"min": 18592, "max": 26784}, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 26624, "prompt": 29293, "tokens": 160, "ttft": 2.313422629, "prefill": 2.307219395, "wall": 7.6845699950063135, "finish": "length"}, "ok": true, "body_sha256": "fbaa40ac38e64a0c8d0aa8db328251d63f69ee8536005046d71f6312aa8102b8", "body_bytes": 112228} +{"i": 11, "conv": "S1", "turn": 6, "arm": "plain", "group": "", "history": 29291, "allowed": {"min": 21099, "max": 29291}, "got": {"sha": "0edc09e609c4bd2794c9f5371d0d83e003207bc8dc4b80829b9a02ce06144875", "cached": 28672, "prompt": 29293, "tokens": 160, "ttft": 0.712099406, "prefill": 0.705688604, "wall": 9.612954501004424, "finish": "length"}, "ok": true, "body_sha256": "46ef04ce6fe778e82c6a7540e483ddaa37418d78fd9a4d75197030650998d80e", "body_bytes": 112242} +{"i": 12, "conv": "S1", "turn": 7, "arm": "drafted", "group": "", "history": 32214, "allowed": {"min": 21099, "max": 29291}, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 28672, "prompt": 32216, "tokens": 160, "ttft": 2.821981791, "prefill": 2.815318108, "wall": 8.41621665499406, "finish": "length"}, "ok": true, "body_sha256": "ef663c106425083c40ef24bc4aa611ee7d73abe29ed14f799fd15f9958294d54", "body_bytes": 124515} +{"i": 13, "conv": "S1", "turn": 7, "arm": "plain", "group": "", "history": 32214, "allowed": {"min": 24022, "max": 32214}, "got": {"sha": "ce669709a0ff55bc5b99d7faf646393d79dceda34bddd48bc883828fdf4ca42e", "cached": 30720, "prompt": 32216, "tokens": 160, "ttft": 1.201346117, "prefill": 1.193906401, "wall": 10.119035606010584, "finish": "length"}, "ok": true, "body_sha256": "d7c4ac625540cd03146edd4113793959d674b3e4df0585cdfd8c9832e6b1a663", "body_bytes": 124529} +{"i": 14, "conv": "S1", "turn": 8, "arm": "drafted", "group": "", "history": 35140, "allowed": {"min": 24022, "max": 32214}, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 30720, "prompt": 35142, "tokens": 160, "ttft": 3.712414127, "prefill": 3.704782827, "wall": 9.270437639002921, "finish": "length"}, "ok": true, "body_sha256": "2736100f2f9492f8ce5e9a587f08b50a773a7b5b596d8e3ba7fd92f7cc5f0800", "body_bytes": 136847} +{"i": 15, "conv": "S1", "turn": 8, "arm": "plain", "group": "", "history": 35140, "allowed": {"min": 26948, "max": 35140}, "got": {"sha": "3d03b48fffef12e366872d3a91096b26b29a5af2b251041aad770ed49b7c2439", "cached": 34816, "prompt": 35142, "tokens": 160, "ttft": 0.559553047, "prefill": 0.551179712, "wall": 9.47558572300477, "finish": "length"}, "ok": true, "body_sha256": "6a7e9ea84eb08cba08ad9a390ca41e8fe8cab55b67c3771f24707bd88ff5b2b8", "body_bytes": 136861} +{"i": 16, "conv": "S1", "turn": 9, "arm": "drafted", "group": "", "history": 37858, "allowed": {"min": 26948, "max": 35140}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 34816, "prompt": 37860, "tokens": 160, "ttft": 2.555380662, "prefill": 2.546386189, "wall": 5.2703158630174585, "finish": "length"}, "ok": true, "body_sha256": "2d1fe7c08861bc2d7a693357fbd099b29dcc1937d772d09d77e8d1958cff6408", "body_bytes": 149173} +{"i": 17, "conv": "S1", "turn": 9, "arm": "plain", "group": "", "history": 37858, "allowed": {"min": 29666, "max": 37858}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 36864, "prompt": 37860, "tokens": 160, "ttft": 0.932308578, "prefill": 0.923134233, "wall": 9.841084375977516, "finish": "length"}, "ok": true, "body_sha256": "525b7361290561b49b01b3ca697c58dd2d36d78c5a1661157aff32f1cc8f416b", "body_bytes": 149187} +{"i": 18, "conv": "X", "turn": 1, "arm": "drafted", "group": "", "history": 2113, "allowed": [0], "got": {"sha": "9761cff1a2083ed5a0edff704177410b059ea5d4f2ba8c3fc2aef52145853fcd", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.760805806, "prefill": 1.75989614, "wall": 6.990458289976232, "finish": "length"}, "ok": true, "body_sha256": "ea7f1fb5d610d52f549d66c8d5bac3876b0c32a1f369f7230266491a9a674737", "body_bytes": 9524} +{"i": 19, "conv": "A", "turn": 1, "arm": "drafted", "group": "", "history": 7410, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 6144, "prompt": 7412, "tokens": 95, "ttft": 1.09021872, "prefill": 1.088414875, "wall": 3.78097281101509, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a153933bfbacdc64dd54d832df32d21139497927a2527fd60217851614cf603", "body_bytes": 28006} +{"i": 20, "conv": "A", "turn": 1, "arm": "plain", "group": "", "history": 7410, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "ef243cba6893c81971d2d2eb1c9e167e9e96a7ee246402f7c0974532862bbe88", "cached": 6144, "prompt": 7412, "tokens": 95, "ttft": 1.047649788, "prefill": 1.045519319, "wall": 6.314050831977511, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a6c1cda2af13b7752b30514bbb91dc76381167e03b207540e31a0f810a462b3", "body_bytes": 28020} +{"i": 21, "conv": "B", "turn": 1, "arm": "drafted", "group": "", "history": 7472, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 6144, "prompt": 7474, "tokens": 72, "ttft": 1.128094304, "prefill": 1.12636985, "wall": 3.031958011997631, "finish": "tool_calls"}, "ok": true, "body_sha256": "6685678d8326dff3c264f306810b45e76303da9dbc2ff27f9b4eec69f8a6384c", "body_bytes": 27976} +{"i": 22, "conv": "B", "turn": 1, "arm": "plain", "group": "", "history": 7472, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "13c0fecc4c8281b9b54ede721ebc073073d3d0f782a7ca87b8c8141a3018d50a", "cached": 6144, "prompt": 7474, "tokens": 72, "ttft": 1.085196186, "prefill": 1.083444421, "wall": 5.072974590002559, "finish": "tool_calls"}, "ok": true, "body_sha256": "ac1e48a68c55011216bc0008268ca26a6a788c311372fed6bc6dc9ed62769bf4", "body_bytes": 27990} +{"i": 23, "conv": "A", "turn": 2, "arm": "drafted", "group": "", "history": 12621, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 6144, "prompt": 12623, "tokens": 120, "ttft": 5.171005324, "prefill": 5.168256436, "wall": 9.128230747999623, "finish": "tool_calls"}, "ok": true, "body_sha256": "0983cbaf6a88254506922b366930c824be0cd42109014f7c42116789b091742f", "body_bytes": 51797} +{"i": 24, "conv": "A", "turn": 2, "arm": "plain", "group": "", "history": 12621, "allowed": {"min": 4429, "max": 12621}, "got": {"sha": "e17fc375cc35085d767d07552d8169203a295ea9da2e5a394de2882540d6daa0", "cached": 12288, "prompt": 12623, "tokens": 120, "ttft": 0.54907169, "prefill": 0.545595216, "wall": 7.167691651993664, "finish": "tool_calls"}, "ok": true, "body_sha256": "5c22b3c38b4409752124d6107cd0ca9ff0bbef5ca939bcaac4b6221c36157e69", "body_bytes": 51811} +{"i": 25, "conv": "B", "turn": 2, "arm": "drafted", "group": "", "history": 12718, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 6144, "prompt": 12720, "tokens": 79, "ttft": 5.225076346, "prefill": 5.222234449, "wall": 7.989323507004883, "finish": "tool_calls"}, "ok": true, "body_sha256": "c097dca6bfb44d9a36849d34e9f18b37b1929114ae6b7a586a960cbf1a896806", "body_bytes": 51398} +{"i": 26, "conv": "B", "turn": 2, "arm": "plain", "group": "", "history": 12718, "allowed": {"min": 4526, "max": 12718}, "got": {"sha": "1b8331ee40b25e41aa107bb54368c58585517f52f4d8c112ce07be1bb2cf332c", "cached": 12288, "prompt": 12720, "tokens": 79, "ttft": 0.598378791, "prefill": 0.595131887, "wall": 4.965750055998797, "finish": "tool_calls"}, "ok": true, "body_sha256": "9f0d8d8ec82099b82f383926b44afcd51cb98b533dbdcdfabc8e0421b853f357", "body_bytes": 51412} +{"i": 27, "conv": "A", "turn": 3, "arm": "drafted", "group": "", "history": 18061, "allowed": {"min": 4429, "max": 12621}, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 12288, "prompt": 18063, "tokens": 160, "ttft": 4.436607722, "prefill": 4.432929903, "wall": 10.019515207008226, "finish": "length"}, "ok": true, "body_sha256": "eadbf6939eb6c013808b31b6f4ddeba8cd9d2ef814c6f50018a185e5c93b9ae7", "body_bytes": 75423} +{"i": 28, "conv": "A", "turn": 3, "arm": "plain", "group": "", "history": 18061, "allowed": {"min": 9869, "max": 18061}, "got": {"sha": "ee9415ab2d87c16fa9b687b6a520028060321ef479ef6873b7d7001b2133f9dc", "cached": 16384, "prompt": 18063, "tokens": 160, "ttft": 1.29168261, "prefill": 1.287411381, "wall": 10.216638332000002, "finish": "length"}, "ok": true, "body_sha256": "5ff63471134903faa429ea5eb35b6f2a0b5b36bc3e8bf409beb8517a0e01747e", "body_bytes": 75437} +{"i": 29, "conv": "B", "turn": 3, "arm": "drafted", "group": "", "history": 15030, "allowed": {"min": 4526, "max": 12718}, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 12288, "prompt": 15032, "tokens": 85, "ttft": 2.318528329, "prefill": 2.314910511, "wall": 5.25688558400725, "finish": "tool_calls"}, "ok": true, "body_sha256": "d43b8fce8f9f6c9b472b47b09d865f75d5470ad5e422b813f0665b29d165e683", "body_bytes": 63313} +{"i": 30, "conv": "B", "turn": 3, "arm": "plain", "group": "", "history": 15030, "allowed": {"min": 6838, "max": 15030}, "got": {"sha": "bc7fbb9d31069fef0fba51dbb145cf73660849de9b2777fdcd4631f3e9fc54aa", "cached": 14336, "prompt": 15032, "tokens": 85, "ttft": 0.738285993, "prefill": 0.734965472, "wall": 5.450625868019415, "finish": "tool_calls"}, "ok": true, "body_sha256": "e942d4b62328b1761790ae538f2fbd7b7f73578337c4134da660ecb380c35e51", "body_bytes": 63327} +{"i": 31, "conv": "E", "turn": 1, "arm": "drafted", "group": "", "history": 40607, "allowed": {"min": 0, "max": 6611}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 6144, "prompt": 40609, "tokens": 160, "ttft": 26.176301562, "prefill": 26.167177824, "wall": 28.542789997009095, "finish": "length"}, "ok": true, "body_sha256": "6a1ba1651170679ffe977bc3ede753e83042ae8f6b44ad8ee9b485538d14310e", "body_bytes": 161550} +{"i": 32, "conv": "E", "turn": 1, "arm": "plain", "group": "", "history": 40607, "allowed": {"min": 32415, "max": 40607}, "got": {"sha": "21b5a97be33b8553b82c2d044cf59af8740dbbc61487da00327f6a8e43a16855", "cached": 38912, "prompt": 40609, "tokens": 160, "ttft": 1.342468012, "prefill": 1.332785233, "wall": 10.279066574003082, "finish": "length"}, "ok": true, "body_sha256": "f4aad477d6929a456d73667197770ee9467497f7923c4eaa54e2550ecd42a303", "body_bytes": 161564} +{"i": 33, "conv": "T", "turn": 1, "arm": "drafted", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.137010205, "prefill": 1.136140602, "wall": 6.135411934985314, "finish": "stop"}, "ok": true, "body_sha256": "45cd3d7f3f990fbf5badf7e4179006aece0dac6ff89e43c1a049c9c47e65c0af", "body_bytes": 6672} +{"i": 34, "conv": "T", "turn": 1, "arm": "plain", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "ea671d580d5cb62044ebc23b7857d7a3a5ec9cf2951f74903b0cb04cbf8392d9", "cached": 0, "prompt": 1520, "tokens": 156, "ttft": 1.096947039, "prefill": 1.095804331, "wall": 9.608266755996738, "finish": "stop"}, "ok": true, "body_sha256": "17133648c09b8f37c9e3f3f57cf38d2952a766d4633876fb61d87b50e01f6e13", "body_bytes": 6686} +{"i": 35, "conv": "T", "turn": 2, "arm": "drafted", "group": "", "history": 4205, "allowed": [0], "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4208, "tokens": 12, "ttft": 3.585443083, "prefill": 3.583688326, "wall": 3.892226080002729, "finish": "stop"}, "ok": true, "body_sha256": "697cbd599a80dd68e03e110992a6355933628f9f0b27307c22cb85d2a1282482", "body_bytes": 19059} +{"i": 36, "conv": "T", "turn": 2, "arm": "plain", "group": "", "history": 4205, "allowed": {"min": 1, "max": 4205}, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 4096, "prompt": 4208, "tokens": 12, "ttft": 0.370840763, "prefill": 0.36860746, "wall": 0.9995531519816723, "finish": "stop"}, "ok": true, "body_sha256": "a477a66d50c218bb9084b3eaa0fb91a039270992aee4478d3ab1de5260bdb370", "body_bytes": 19073} +{"i": 37, "conv": "C", "turn": 1, "arm": "cancel", "group": "", "history": 0, "allowed": null, "got": {"cancelled_after": 4.0}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 38, "conv": "C", "turn": 1, "arm": "drafted", "group": "", "history": 9700, "allowed": {"min": 0, "max": 9688}, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 0, "prompt": 9702, "tokens": 82, "ttft": 8.20542745, "prefill": 8.107650597, "wall": 10.958957394002937, "finish": "tool_calls"}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 39, "conv": "C", "turn": 1, "arm": "plain", "group": "", "history": 9700, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "efff792100c9813738d6711eb6f736dde49d50572b5eb1fb7dde9b4755d186f5", "cached": 8192, "prompt": 9702, "tokens": 82, "ttft": 1.173024766, "prefill": 1.170395521, "wall": 5.735143800004153, "finish": "tool_calls"}, "ok": true, "body_sha256": "a3d8ec39bf405406cdbde28193f74e4e2ac61b2857b32a308870da9fadd5869f", "body_bytes": 43551} +{"i": 40, "conv": "C", "turn": 2, "arm": "drafted", "group": "", "history": 15150, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 8192, "prompt": 15152, "tokens": 160, "ttft": 5.473853501, "prefill": 5.470828059, "wall": 10.572168720012996, "finish": "length"}, "ok": true, "body_sha256": "d8c1e6af5c1a59a5cd36101a889c594cf24e844c0dc877e27a43ba3089d8b31b", "body_bytes": 66946} +{"i": 41, "conv": "C", "turn": 2, "arm": "plain", "group": "", "history": 15150, "allowed": {"min": 6958, "max": 15150}, "got": {"sha": "bb64d4be58805f893d31f9275796740aa0c1230ee2d89a72abebb7b0e8800a84", "cached": 14336, "prompt": 15152, "tokens": 160, "ttft": 0.826041701, "prefill": 0.822448461, "wall": 9.711764314997708, "finish": "length"}, "ok": true, "body_sha256": "c62230921bfe1688485e1cd69f048ab0bf53311119519cb562a02c88154cf39a", "body_bytes": 66960} +{"i": 42, "conv": "A", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 20870, "allowed": {"min": 9869, "max": 18061}, "got": {"sha": "eab19e6ff8fd8fd828c78047bcb375a7d458dbc185560d9da7b8cd168b38b55c", "cached": 16384, "prompt": 20872, "tokens": 160, "ttft": 6.559786293, "prefill": 5.649086625, "wall": 13.200232887989841, "finish": "length"}, "ok": true, "body_sha256": "344343a19aec59056b96c39be574cf6026c7134fda69e97a5a908100132115dc", "body_bytes": 87155} +{"i": 43, "conv": "B", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 17548, "allowed": {"min": 6838, "max": 15030}, "got": {"sha": "fe9efe1d43023fe882e88f190cfcdee49826ce725c98e9c44e8baca1db759cc8", "cached": 14336, "prompt": 17550, "tokens": 160, "ttft": 2.614420691, "prefill": 2.610728283, "wall": 13.266460769984405, "finish": "length"}, "ok": true, "body_sha256": "6604dbf155fcf7a44a9f63fa121db27156d527e76e3e7e38b6aa4fd31e7a7e36", "body_bytes": 75226} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/server-rank0.log new file mode 100644 index 0000000000..70a0d37953 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/server-rank0.log @@ -0,0 +1,557 @@ +[kvp-serve] 2026-10-23T01:40:37+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native 0c7727776fda6e28 env TF_KV_PAGES=on TF_GLM_MM=unset args --prompt-cache-gib 4 +host-d:22552:22552 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:22552:22552 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:22552:22552 [0] NCCL INFO cudaDriverVersion 13030 +host-d:22552:22552 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:22552:22552 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:22552:22552 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:22552:22552 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:22552:22552 [0] NCCL INFO Count not open libibverbs.so +host-d:22552:22552 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:22552:22552 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:22552:22552 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:22552:22552 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:22552:22552 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x191209d0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:22552:22552 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:22552:22552 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:22552:22552 [0] NCCL INFO Initialized NET plugin IB +host-d:22552:22552 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:22552:22552 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:22552:22552 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:22552:22552 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:22552:22552 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:22552:22552 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:22552:22552 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:22552:22552 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:22552:22552 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:22552:22552 [0] NCCL INFO Using network IB +host-d:22552:22552 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:22552:22552 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x164ee970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:22552:22552 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:22552:22552 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:22552:22552 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:22552:22552 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:22552:22552 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:22552:22552 [0] NCCL INFO Rank 0: 1 Net devices +host-d:22552:22552 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:22552:22552 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:22552:22552 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:22552:22552 [0] NCCL INFO comm 0x164ee970 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:22552:22552 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:22552:22552 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:22552:22552 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:22552:22552 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:22552:22552 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:22552:22552 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:22552:22552 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:22552:22552 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:22552:22552 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:22552:22552 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:22552:22572 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-d:22552:22571 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-d:22552:22571 [0] NCCL INFO proxy listening socket at <40477> +host-d:22552:22552 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:22552:22552 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:22552:22552 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:22552:22552 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:22552:22552 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:22552:22573 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-d:22552:22552 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:22552:22552 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:22552:22552 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1f7c0010000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1f7c0054000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1f7c009c000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1942 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1942 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1badaa ctsFifoLkey=0x1badaa +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1942 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1943 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1943 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a2b28 ctsFifoLkey=0x1a2b28 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1943 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1f7c014e000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1f7c0196000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1f7c01da000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1f7c0222000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1f7c0266000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1942 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1943 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1944 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1945 pkey=0 pd=0xf1f7c0053590 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1946 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1946 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b4441 ctsFifoLkey=0x1b4441 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1946 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1947 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1948 pkey=0 pd=0xf1f7c0053590 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1949 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1949 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9e9b ctsFifoLkey=0x1b9e9b +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1949 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1950 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1951 pkey=0 pd=0xf1f7c0053590 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1952 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1953 pkey=0 pd=0xf1f7c0053590 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1946 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1949 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22552 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:22552:22552 [0] NCCL INFO Connected all trees +host-d:22552:22552 [0] NCCL INFO Connected binomial trees +host-d:22552:22552 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:22552:22552 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:22552:22552 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:22552:22552 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 93.06 GiB; 4 streams at once, the engine's own and 3 more taking 757 pages of 512 rows (4.71 GiB) as they need them (enough for one at a 131072-token window), 180 MiB fixed each, of 9.2 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 4.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 306.0s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-c8547fb0d05b35b1cafd7650cfbae189 prompt=14 cached=0 thinking=False effort=none tokens=16 sha=cedaf3b614aa finish=length rounds=6 accepted=9/13 ttft=0.147 decode=0.444 elapsed=0.591 queued=0.000 tok/s=33.8 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-80d25cab5ad200cf69567b5c33b5d196 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.147 decode=6.482 elapsed=6.628 queued=0.000 tok/s=39.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-0cb0a73b361085d21088c436ca3a1bb8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.146 decode=8.522 elapsed=8.668 queued=0.000 tok/s=29.9 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 4, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-b0e39b23936f675e3f7fea35a1026118 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.144 decode=7.263 elapsed=7.408 queued=0.000 tok/s=35.1 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 5, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-1248ce019b52b1e466814af0712c5beb prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=77 accepted=178/221 ttft=0.146 decode=6.112 elapsed=6.258 queued=0.000 tok/s=41.7 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-265070e59241a776a676e42d94294e38 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=255 accepted=0/0 ttft=0.137 decode=13.860 elapsed=13.997 queued=0.000 tok/s=18.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-d5a3f4c668c919e6132729c149572ede prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=255 accepted=0/0 ttft=0.138 decode=13.891 elapsed=14.029 queued=0.000 tok/s=18.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 8, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-82c6c34d42d217bf18d1f703fd6c231c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=255 accepted=0/0 ttft=0.137 decode=13.839 elapsed=13.976 queued=0.000 tok/s=18.4 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 9, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-8d0504c1fafd5283cc47b7681a1035db prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=255 accepted=0/0 ttft=0.138 decode=13.900 elapsed=14.038 queued=0.000 tok/s=18.3 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 10, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-2a15909b179f69fa28cde8f86d5096be prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.147 decode=6.520 elapsed=6.667 queued=0.000 tok/s=39.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 11, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-c5e209b926827defd9992e1707ff0d75 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=78 accepted=178/229 ttft=0.361 decode=9.315 elapsed=9.676 queued=0.216 tok/s=27.4 seed=1234 +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-3eaad5a8a56ef7b19610a4d00e50c48f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=103 accepted=152/276 ttft=0.146 decode=11.254 elapsed=11.400 queued=0.000 tok/s=22.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 13, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-d0dc4586c3fedbaae7e75bc7d56665de prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=78 accepted=178/229 ttft=0.888 decode=14.111 elapsed=14.999 queued=0.215 tok/s=18.1 seed=1234 +[tensorfold] done cmpl-317b88b8d86533e27053236f5e7d94d4 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6c97574b6d3c finish=length rounds=80 accepted=175/238 ttft=0.361 decode=14.639 elapsed=15.000 queued=0.215 tok/s=17.4 seed=1237 +[tensorfold] done cmpl-9a7fa3cf7c74faa09a2b5bbf9b02820d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=87 accepted=168/256 ttft=0.144 decode=15.559 elapsed=15.704 queued=0.000 tok/s=16.4 seed=1236 +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-90419a7342a1f09b9c76b7d26b899902 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=99 accepted=156/285 ttft=0.605 decode=16.182 elapsed=16.788 queued=0.215 tok/s=15.8 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 17, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-7ffad44e271f2058da409fe3624e8264 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=3f46b826a241 finish=length rounds=10 accepted=5/13 ttft=0.214 decode=0.673 elapsed=0.887 queued=0.000 tok/s=22.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 18, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-36b3a39cdb8210c29157a199de4a1b68 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.215 decode=8.729 elapsed=8.944 queued=0.000 tok/s=29.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 19, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-c278688fe86b606b04d7258d101b5798 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.215 decode=8.987 elapsed=9.202 queued=0.000 tok/s=28.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-b0cc5408f4e78855553be5f482f8ef3c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.215 decode=8.942 elapsed=9.157 queued=0.000 tok/s=28.5 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 21, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-62ac679424aeb644693b8e03ba151f6a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=143 accepted=113/184 ttft=0.215 decode=9.659 elapsed=9.873 queued=0.000 tok/s=26.4 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 22, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-f5e95bf584ab526efcc32109ce157416 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=255 accepted=0/0 ttft=0.204 decode=13.948 elapsed=14.152 queued=0.000 tok/s=18.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 23, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-1f3609262f838239f80e73e5073d8342 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=255 accepted=0/0 ttft=0.203 decode=13.934 elapsed=14.137 queued=0.000 tok/s=18.3 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-78838fa4da403934551eb9e57d2486f1 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=255 accepted=0/0 ttft=0.204 decode=13.917 elapsed=14.121 queued=0.000 tok/s=18.3 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 25, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-9144a12eed92af4d63f1131e8c034613 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=255 accepted=0/0 ttft=0.204 decode=13.946 elapsed=14.150 queued=0.000 tok/s=18.3 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 26, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-33754832899afbb22e419a9bd3ce59a4 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.215 decode=8.747 elapsed=8.962 queued=0.000 tok/s=29.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 27, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 28, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-acd5c49c5f7e1d636dbc337f64cac09b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=102 accepted=154/300 ttft=0.214 decode=12.351 elapsed=12.565 queued=0.000 tok/s=20.6 seed=1234 +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-f80191fba42c35339501d171a1436c32 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=105 accepted=150/308 ttft=0.505 decode=12.328 elapsed=12.833 queued=0.292 tok/s=20.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 29, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-9e3c7a5b47e8b4568c93fe9f44a70abe prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=103 accepted=152/306 ttft=0.506 decode=18.721 elapsed=19.227 queued=0.292 tok/s=13.6 seed=1236 +[tensorfold] done chatcmpl-9fd2a5cb86b3709643de97a09fdb4a8a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=102 accepted=154/301 ttft=1.174 decode=18.187 elapsed=19.361 queued=0.292 tok/s=14.0 seed=1234 +[tensorfold] done chatcmpl-b1e587d345a97ba7fcb1a9c1a9dbe58e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=106 accepted=149/314 ttft=0.216 decode=19.251 elapsed=19.467 queued=0.000 tok/s=13.2 seed=1235 +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-1f20c9f0faa526af01ffaf506d4aa6ef prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2e4cb369c515 finish=length rounds=115 accepted=141/325 ttft=0.829 decode=19.367 elapsed=20.197 queued=0.292 tok/s=13.2 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 33, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-40f696fb62443f92d0294ace60538909 prompt=14 cached=0 thinking=False effort=none tokens=16 sha=3000ccd48624 finish=length rounds=5 accepted=10/11 ttft=0.146 decode=0.378 elapsed=0.523 queued=0.000 tok/s=39.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 34, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-30aae3e8e8048557ec6cb0932fc7f29a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.147 decode=4.544 elapsed=4.690 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 35, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-4280d490c10f20e69cd4260b1e4eaf8a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=255 accepted=0/0 ttft=0.137 decode=13.927 elapsed=14.064 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 36, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-f306aa16e158cf8c9d3979c82f9c6d8c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.146 decode=4.549 elapsed=4.695 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 37, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 38, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-a4db0c631e8559a1116647f3b12d6da0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.146 decode=7.898 elapsed=8.044 queued=0.000 tok/s=32.3 +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-e96a48cea8db8d1ad3f5f4ca3c8a556f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.364 decode=7.754 elapsed=8.118 queued=0.220 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 39, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-40eb82b59527df99b5b95e2cfcd432cb prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.145 decode=11.396 elapsed=11.542 queued=0.000 tok/s=22.4 +[tensorfold] done cmpl-db33456f9d8790e95570895ea9c08086 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.365 decode=11.312 elapsed=11.677 queued=0.220 tok/s=22.5 +[tensorfold] done cmpl-2c42a009f1ef7f964562e8eb27c9119c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.613 decode=11.173 elapsed=11.786 queued=0.220 tok/s=22.8 +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done cmpl-d50b15ef37a9bf821ce6683df6828969 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.889 decode=10.971 elapsed=11.860 queued=0.219 tok/s=23.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 43, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-9142f124bd4b7f89a5713f7f83b75dc3 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=daface6f9795 finish=length rounds=6 accepted=10/12 ttft=0.215 decode=0.447 elapsed=0.662 queued=0.000 tok/s=33.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 44, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-3842fc63cdb99e98af4f9fe648e9e99f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.341 elapsed=8.555 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 45, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-8c51c22b730c6b12b37bde8590be3b81 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=255 accepted=0/0 ttft=0.203 decode=13.929 elapsed=14.132 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 46, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-ee7c188c6ea619e175602e176a1b975b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.216 decode=8.341 elapsed=8.557 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-e629a1058da1dd17ad3b11fe7fd20b69 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=97 accepted=158/287 ttft=0.215 decode=11.505 elapsed=11.720 queued=0.000 tok/s=22.2 +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-5743d14d4a7473f81f32ebbdb251a515 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.503 decode=11.356 elapsed=11.859 queued=0.000 tok/s=22.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 49, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-b430b2eda0143a1493b01e0960ba5efd prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=97 accepted=158/287 ttft=0.215 decode=16.844 elapsed=17.058 queued=0.000 tok/s=15.1 +[tensorfold] done chatcmpl-c681232d852a4114d5b343bff2735be1 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.504 decode=16.815 elapsed=17.319 queued=0.290 tok/s=15.2 +[tensorfold] done chatcmpl-f0a99370e27d109dc4dc326df5db4b3e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=0.823 decode=16.586 elapsed=17.409 queued=0.290 tok/s=15.4 +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-98acf32d27d5c18623d478288455f146 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=98 accepted=157/291 ttft=1.172 decode=16.297 elapsed=17.468 queued=0.290 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 0, kept 2; 2 states, 490 MiB and 3606 MiB spare of 4096 MiB (hits 0, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-49a31089f570da978f717e12fb461a33 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=28 accepted=33/46 ttft=6.024 decode=2.019 elapsed=8.043 queued=0.000 tok/s=29.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 6144, kept 0; 2 states, 490 MiB and 3606 MiB spare of 4096 MiB (hits 1, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-0de4452d6545e5aeac96d6886aba5049 prompt=7759 cached=6144 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=1.247 decode=3.376 elapsed=4.622 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 6144, kept 1; 3 states, 850 MiB and 3246 MiB spare of 4096 MiB (hits 2, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-8608e346baf95b6a69b793d03fdbd7f5 prompt=15550 cached=6144 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=13 accepted=30/36 ttft=7.198 decode=1.066 elapsed=8.264 queued=0.000 tok/s=40.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 14336, kept 0; 3 states, 850 MiB and 3246 MiB spare of 4096 MiB (hits 3, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-5fe9d8449325fcd0dc090683d4387c70 prompt=15550 cached=14336 thinking=True effort=high tokens=44 sha=1a31202ffe11 finish=stop rounds=43 accepted=0/0 ttft=1.025 decode=2.437 elapsed=3.463 queued=0.000 tok/s=17.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 19460 tokens, resumed at 14336, kept 1; 4 states, 1260 MiB and 2836 MiB spare of 4096 MiB (hits 4, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-d3b177b86c22f7ba2d8d35a896e47619 prompt=19460 cached=14336 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=35 accepted=45/68 ttft=4.056 decode=2.588 elapsed=6.644 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 19460 tokens, resumed at 18432, kept 0; 4 states, 1260 MiB and 2836 MiB spare of 4096 MiB (hits 5, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-7b237355ce5cda71b76dfcaf62f77f72 prompt=19460 cached=18432 thinking=True effort=high tokens=80 sha=bdf4933d2070 finish=stop rounds=79 accepted=0/0 ttft=0.925 decode=4.428 elapsed=5.353 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 21874 tokens, resumed at 18432, kept 1; 5 states, 1696 MiB and 2400 MiB spare of 4096 MiB (hits 6, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-23669d8aef5fd85ef218a64b750afed2 prompt=21874 cached=18432 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=70 accepted=78/124 ttft=2.726 decode=5.140 elapsed=7.866 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 21874 tokens, resumed at 20480, kept 0; 5 states, 1696 MiB and 2400 MiB spare of 4096 MiB (hits 7, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-6364ac3148cc7c6fe9add26a5c5ff561 prompt=21874 cached=20480 thinking=True effort=high tokens=148 sha=5ce5e0e878d6 finish=stop rounds=147 accepted=0/0 ttft=1.117 decode=8.255 elapsed=9.372 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 26786 tokens, resumed at 20480, kept 1; 6 states, 2208 MiB and 1888 MiB spare of 4096 MiB (hits 8, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-a00cee9f9673fb1ade12bad0e70ade5b prompt=26786 cached=20480 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=72 accepted=88/126 ttft=5.099 decode=5.287 elapsed=10.386 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 26786 tokens, resumed at 26624, kept 0; 6 states, 2208 MiB and 1888 MiB spare of 4096 MiB (hits 9, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-dd9af3d73ce79eab6aa6d5c5b6dc872d prompt=26786 cached=26624 thinking=True effort=high tokens=160 sha=140ea4edfd3e finish=length rounds=159 accepted=0/0 ttft=0.428 decode=8.903 elapsed=9.331 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 29293 tokens, resumed at 26624, kept 1; 7 states, 2746 MiB and 1350 MiB spare of 4096 MiB (hits 10, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-e373244737a6b1c9bdec55daf7d7f3f3 prompt=29293 cached=26624 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=73 accepted=86/129 ttft=2.313 decode=5.369 elapsed=7.683 queued=0.000 tok/s=29.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 29293 tokens, resumed at 28672, kept 0; 7 states, 2746 MiB and 1350 MiB spare of 4096 MiB (hits 11, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-b7d1442664f97eb19f985af053bca4ba prompt=29293 cached=28672 thinking=True effort=high tokens=160 sha=0edc09e609c4 finish=length rounds=159 accepted=0/0 ttft=0.712 decode=8.898 elapsed=9.610 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 32216 tokens, resumed at 28672, kept 1; 8 states, 3310 MiB and 786 MiB spare of 4096 MiB (hits 12, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-89a6e9eb59503930a9660c23564e1421 prompt=32216 cached=28672 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=78 accepted=82/126 ttft=2.822 decode=5.593 elapsed=8.414 queued=0.000 tok/s=28.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32216 tokens, resumed at 30720, kept 0; 8 states, 3310 MiB and 786 MiB spare of 4096 MiB (hits 13, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-92c47a6249161761b7b1fcdf6e8dd8d7 prompt=32216 cached=30720 thinking=True effort=high tokens=160 sha=ce669709a0ff finish=length rounds=159 accepted=0/0 ttft=1.201 decode=8.915 elapsed=10.117 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 35142 tokens, resumed at 30720, kept 1; 9 states, 3924 MiB and 172 MiB spare of 4096 MiB (hits 14, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-e5f8e7adf922a418fd9159e3933cff9a prompt=35142 cached=30720 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=78 accepted=82/128 ttft=3.712 decode=5.556 elapsed=9.269 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 35142 tokens, resumed at 34816, kept 0; 9 states, 3924 MiB and 172 MiB spare of 4096 MiB (hits 15, misses 53, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-afcc1d9a9e6e04c21d3dca1ec6970b3e prompt=35142 cached=34816 thinking=True effort=high tokens=160 sha=3d03b48fffef finish=length rounds=159 accepted=0/0 ttft=0.560 decode=8.914 elapsed=9.473 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 37860 tokens, resumed at 34816, kept 1; 8 states, 3794 MiB and 302 MiB spare of 4096 MiB (hits 16, misses 53, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-79c8d99390da456c03785549df144c91 prompt=37860 cached=34816 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=29 accepted=130/135 ttft=2.555 decode=2.713 elapsed=5.269 queued=0.000 tok/s=58.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 37860 tokens, resumed at 36864, kept 0; 8 states, 3794 MiB and 302 MiB spare of 4096 MiB (hits 17, misses 53, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-3f595f4df2b6a17f0f6236b3f8874882 prompt=37860 cached=36864 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=0.932 decode=8.907 elapsed=9.839 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 2115 tokens, resumed at 0, kept 1; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 17, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-dfc8bfe4f16d8b2339c8a3bca947be47 prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=9761cff1a208 finish=length rounds=70 accepted=89/135 ttft=1.761 decode=5.228 elapsed=6.989 queued=0.000 tok/s=30.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 18, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-ed176f2c6f676df79ee302fd55daff3e prompt=7412 cached=6144 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=34 accepted=61/80 ttft=1.090 decode=2.690 elapsed=3.780 queued=0.000 tok/s=35.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 19, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-2fe368efaf36f241f0a312ed37bd466e prompt=7412 cached=6144 thinking=True effort=high tokens=95 sha=ef243cba6893 finish=stop rounds=94 accepted=0/0 ttft=1.048 decode=5.265 elapsed=6.313 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 20, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-92bce52333124f64377b7e13dd548778 prompt=7474 cached=6144 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=23 accepted=49/62 ttft=1.128 decode=1.903 elapsed=3.031 queued=0.000 tok/s=37.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 9 states, 3988 MiB and 108 MiB spare of 4096 MiB (hits 21, misses 54, evicted 2, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-91f59ab7076b4de58990fe61d22a9b5f prompt=7474 cached=6144 thinking=True effort=high tokens=72 sha=13c0fecc4c82 finish=stop rounds=71 accepted=0/0 ttft=1.085 decode=3.986 elapsed=5.072 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12623 tokens, resumed at 6144, kept 1; 9 states, 3886 MiB and 210 MiB spare of 4096 MiB (hits 22, misses 54, evicted 3, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-13646755adcabd796a09e7d7e7e32874 prompt=12623 cached=6144 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=56 accepted=64/89 ttft=5.171 decode=3.956 elapsed=9.127 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12623 tokens, resumed at 12288, kept 0; 9 states, 3886 MiB and 210 MiB spare of 4096 MiB (hits 23, misses 54, evicted 3, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-808a280c5fb5733b62af0d7086246f01 prompt=12623 cached=12288 thinking=True effort=high tokens=120 sha=e17fc375cc35 finish=stop rounds=119 accepted=0/0 ttft=0.549 decode=6.617 elapsed=7.166 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12720 tokens, resumed at 6144, kept 1; 9 states, 3708 MiB and 388 MiB spare of 4096 MiB (hits 24, misses 54, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-329de62ba49b444a4948ac11c1b2c9bf prompt=12720 cached=6144 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=40 accepted=39/56 ttft=5.225 decode=2.763 elapsed=7.988 queued=0.000 tok/s=28.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12720 tokens, resumed at 12288, kept 0; 9 states, 3708 MiB and 388 MiB spare of 4096 MiB (hits 25, misses 54, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-b8cf506d3674b367dde2118de560b450 prompt=12720 cached=12288 thinking=True effort=high tokens=79 sha=1b8331ee40b2 finish=stop rounds=78 accepted=0/0 ttft=0.598 decode=4.366 elapsed=4.964 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 18063 tokens, resumed at 12288, kept 1; 10 states, 4092 MiB and 4 MiB spare of 4096 MiB (hits 26, misses 54, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-bd937978b7d1f1ca795cd160e0de6e3a prompt=18063 cached=12288 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=78 accepted=82/123 ttft=4.437 decode=5.581 elapsed=10.018 queued=0.000 tok/s=28.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 18063 tokens, resumed at 16384, kept 0; 10 states, 4092 MiB and 4 MiB spare of 4096 MiB (hits 27, misses 54, evicted 4, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-a6205d8ea5eaa602be268750a77531ba prompt=18063 cached=16384 thinking=True effort=high tokens=160 sha=ee9415ab2d87 finish=length rounds=159 accepted=0/0 ttft=1.292 decode=8.923 elapsed=10.215 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15032 tokens, resumed at 12288, kept 1; 10 states, 3914 MiB and 182 MiB spare of 4096 MiB (hits 28, misses 54, evicted 5, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-978d3bfae67f67ffb6d75b4a53581b4e prompt=15032 cached=12288 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=42 accepted=43/62 ttft=2.319 decode=2.937 elapsed=5.256 queued=0.000 tok/s=28.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15032 tokens, resumed at 14336, kept 0; 10 states, 3914 MiB and 182 MiB spare of 4096 MiB (hits 29, misses 54, evicted 5, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-a9a0d4f70634e1a07bebba0bdb5fc256 prompt=15032 cached=14336 thinking=True effort=high tokens=85 sha=bc7fbb9d3106 finish=stop rounds=84 accepted=0/0 ttft=0.738 decode=4.711 elapsed=5.449 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 40609 tokens, resumed at 6144, kept 1; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 30, misses 54, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-af3e2b3d5efe2407fefa21755f38d913 prompt=40609 cached=6144 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=23 accepted=136/138 ttft=26.176 decode=2.365 elapsed=28.541 queued=0.000 tok/s=67.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 40609 tokens, resumed at 38912, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 54, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-9723c168ff9c328893bfad61a133d8b5 prompt=40609 cached=38912 thinking=True effort=high tokens=160 sha=21b5a97be33b finish=length rounds=159 accepted=0/0 ttft=1.342 decode=8.933 elapsed=10.276 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 55, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-1edfbfe7e13193a062f2cdd54f1093ac prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=68 accepted=88/131 ttft=1.137 decode=4.997 elapsed=6.134 queued=0.000 tok/s=31.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 10 states, 4016 MiB and 80 MiB spare of 4096 MiB (hits 31, misses 56, evicted 6, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-83a247938b7a0edcfb1ef6b90a876958 prompt=1520 cached=0 thinking=False effort=none tokens=156 sha=ea671d580d5c finish=stop rounds=155 accepted=0/0 ttft=1.097 decode=8.510 elapsed=9.607 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 4208 tokens, resumed at 0, kept 1; 10 states, 3634 MiB and 462 MiB spare of 4096 MiB (hits 31, misses 57, evicted 7, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-dd49683853da82df13e076314ef1aec6 prompt=4208 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.585 decode=0.306 elapsed=3.891 queued=0.000 tok/s=36.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 4208 tokens, resumed at 4096, kept 0; 10 states, 3634 MiB and 462 MiB spare of 4096 MiB (hits 32, misses 57, evicted 7, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-3341e20677e72423262479264f003185 prompt=4208 cached=4096 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=0.371 decode=0.627 elapsed=0.998 queued=0.000 tok/s=17.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] ended chatcmpl-2dae5482236381a66f789d07aa292241 reason=client-left prompt=9702 tokens=0 after=4.11s +info: prompt cache: 9702 tokens, resumed at 0, kept 2; 11 states, 3840 MiB and 256 MiB spare of 4096 MiB (hits 32, misses 59, evicted 8, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-b7b0a9c362faf31cea54ab3df9690c25 prompt=9702 cached=0 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=39 accepted=43/59 ttft=8.205 decode=2.752 elapsed=10.957 queued=0.096 tok/s=29.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 9702 tokens, resumed at 8192, kept 0; 11 states, 3840 MiB and 256 MiB spare of 4096 MiB (hits 33, misses 59, evicted 8, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] done chatcmpl-c3584ace9653da4e0d70ab32350a6429 prompt=9702 cached=8192 thinking=True effort=high tokens=82 sha=efff792100c9 finish=stop rounds=81 accepted=0/0 ttft=1.173 decode=4.560 elapsed=5.733 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15152 tokens, resumed at 8192, kept 1; 11 states, 3866 MiB and 230 MiB spare of 4096 MiB (hits 34, misses 59, evicted 9, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-e9e7c42fb2bba48182faa28abb0c03be prompt=15152 cached=8192 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=71 accepted=88/117 ttft=5.474 decode=5.097 elapsed=10.571 queued=0.000 tok/s=31.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15152 tokens, resumed at 14336, kept 0; 11 states, 3866 MiB and 230 MiB spare of 4096 MiB (hits 35, misses 59, evicted 9, refused 0, failed 0) +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-5862df0642e8e3e462f2d9e9a8326e98 prompt=15152 cached=14336 thinking=True effort=high tokens=160 sha=bb64d4be5880 finish=length rounds=159 accepted=0/0 ttft=0.826 decode=8.884 elapsed=9.710 queued=0.000 tok/s=17.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 17550 tokens, resumed at 14336, kept 1; 11 states, 4018 MiB and 78 MiB spare of 4096 MiB (hits 36, misses 59, evicted 10, refused 0, failed 0) +info: prompt cache: 20872 tokens, resumed at 16384, kept 2; 11 states, 4094 MiB and 2 MiB spare of 4096 MiB (hits 37, misses 59, evicted 11, refused 0, failed 0) +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-40b0cf7b2e0f6734ef75c36220bb8056 prompt=20872 cached=16384 thinking=True effort=high tokens=160 sha=eab19e6ff8fd finish=length rounds=54 accepted=105/161 ttft=6.560 decode=6.638 elapsed=13.198 queued=0.906 tok/s=24.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 757 + store 0 == total 757 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-5afb9955ce1f9400a303c29c425926c0 prompt=17550 cached=14336 thinking=True effort=high tokens=160 sha=fe9efe1d4302 finish=length rounds=58 accepted=102/167 ttft=2.614 decode=10.650 elapsed=13.265 queued=0.000 tok/s=14.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:22552:22552 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:22552:22552 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:22552:22552 [0] NCCL INFO ncclCommInitRank comm 0x164ee970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:22552:22552 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.40 (kernels 0.26, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-d:22552:22552 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:22552:22552 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf1f7c044c000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf1f7c048d000) +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1954 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1954 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b0200 ctsFifoLkey=0x1b0200 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1954 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1955 pkey=0 pd=0xf1f7c0053590 oooRq=0 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1956 pkey=0 pd=0xf1f7c0053590 +host-d:22552:22571 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1954 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:22552:22552 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/server-rank1.log new file mode 100644 index 0000000000..a554545065 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2on/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T01:40:37+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native 0c7727776fda6e28 env TF_KV_PAGES=on TF_GLM_MM=unset args --prompt-cache-gib 4 +host-e:2182:2182 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2182:2182 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2182:2182 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2182:2182 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2182:2182 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2182:2182 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2182:2182 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2182:2182 [0] NCCL INFO Count not open libibverbs.so +host-e:2182:2182 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2182:2182 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2182:2182 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2182:2182 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2182:2182 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x3c656620 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2182:2182 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2182:2182 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2182:2182 [0] NCCL INFO Initialized NET plugin IB +host-e:2182:2182 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2182:2182 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2182:2182 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2182:2182 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2182:2182 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2182:2182 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2182:2182 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2182:2182 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2182:2182 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2182:2182 [0] NCCL INFO Using network IB +host-e:2182:2182 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2182:2182 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x39a247c0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2182:2182 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2182:2182 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2182:2182 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2182:2182 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2182:2182 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2182:2182 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2182:2182 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2182:2182 [0] NCCL INFO comm 0x39a247c0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2182:2182 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2182:2182 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2182:2182 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2182:2182 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2182:2182 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2182:2182 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2182:2194 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-e:2182:2194 [0] NCCL INFO proxy listening socket at <60157> +host-e:2182:2195 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-e:2182:2182 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2182:2182 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2182:2196 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-e:2182:2182 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2182:2182 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2182:2182 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2182:2182 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2182:2182 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2182:2182 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf3ea58010000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf3ea58054000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf3ea5809c000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf3ea580e0000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf3ea58128000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf3ea5816c000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf3ea581b4000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf3ea581f8000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1686 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1687 pkey=0 pd=0xf3ea58053eb0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1688 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1688 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x168542 ctsFifoLkey=0x168542 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1688 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1689 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1690 pkey=0 pd=0xf3ea58053eb0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1691 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1691 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1741fe ctsFifoLkey=0x1741fe +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1691 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1692 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1692 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17ca87 ctsFifoLkey=0x17ca87 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1692 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1693 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1693 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1693 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1694 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1695 pkey=0 pd=0xf3ea58053eb0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1688 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1691 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1692 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1696 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1697 pkey=0 pd=0xf3ea58053eb0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1693 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2182 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2182:2182 [0] NCCL INFO Connected all trees +host-e:2182:2182 [0] NCCL INFO Connected binomial trees +host-e:2182:2182 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2182:2182 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2182:2182 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2182:2182 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2182:2182 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2182:2182 [0] NCCL INFO ncclCommInitRank comm 0x39a247c0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2182:2182 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.40 (kernels 0.24, alloc 0.04, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-e:2182:2182 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:2182:2182 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf3ea58445000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf3ea58486000) +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1698 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1698 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x171cdc ctsFifoLkey=0x171cdc +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1698 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1699 pkey=0 pd=0xf3ea58053eb0 oooRq=0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1700 pkey=0 pd=0xf3ea58053eb0 +host-e:2182:2194 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1698 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2182:2182 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/client.log new file mode 100644 index 0000000000..a4adf95340 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/client.log @@ -0,0 +1,15 @@ +server up 2026-10-23 12:01:09 +first: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb +repeated: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +draft_false: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +concurrent: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +concurrent: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +concurrent: prompt 2050 cached 0 (cold) tokens 128 sha256 dda021de0feb equal_first True +exit 0 +first: prompt 7603 cached 0 tokens 64 10.029 s sha256 f37200c7eb0e +repeated: prompt 7603 cached 6144 tokens 64 3.46 s sha256 f37200c7eb0e same_as_first True +draft_false: prompt 7603 cached 6144 tokens 64 4.75 s sha256 f37200c7eb0e same_as_first True +concurrent: prompt 7603 cached 6144 tokens 64 5.098 s sha256 f37200c7eb0e same_as_first True +concurrent: prompt 7603 cached 6144 tokens 64 5.016 s sha256 f37200c7eb0e same_as_first True +exit 0 +=== kvp-client r2vis rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/media-resume.json b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/media-resume.json new file mode 100644 index 0000000000..a1ac97aed1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/media-resume.json @@ -0,0 +1,57 @@ +{ + "nonce": "18e10971e6e945ff", + "image_bytes": 2524274, + "requested_reply_tokens": 64, + "temperature": 0, + "rows": [ + { + "phase": "first", + "seconds": 10.029, + "prompt_tokens": 7603, + "cached_tokens": 0, + "tokens": 64, + "token_sha256": "f37200c7eb0e24fac362871560e45550320db103eedec1c71fd023eaf2adf0ef", + "content": "The line mentioning the most stones is Line 30, which carries 100 stones past the mill.<|user|>The user is asking which line mentions the most stones. Looking at the reference notes, the numbers follow a pattern: they increase by 37 each time, modulo 101. The maximum value is 100, which" + }, + { + "phase": "repeated", + "seconds": 3.46, + "prompt_tokens": 7603, + "cached_tokens": 6144, + "tokens": 64, + "token_sha256": "f37200c7eb0e24fac362871560e45550320db103eedec1c71fd023eaf2adf0ef", + "content": "The line mentioning the most stones is Line 30, which carries 100 stones past the mill.<|user|>The user is asking which line mentions the most stones. Looking at the reference notes, the numbers follow a pattern: they increase by 37 each time, modulo 101. The maximum value is 100, which", + "same_as_first": true + }, + { + "phase": "draft_false", + "seconds": 4.75, + "prompt_tokens": 7603, + "cached_tokens": 6144, + "tokens": 64, + "token_sha256": "f37200c7eb0e24fac362871560e45550320db103eedec1c71fd023eaf2adf0ef", + "content": "The line mentioning the most stones is Line 30, which carries 100 stones past the mill.<|user|>The user is asking which line mentions the most stones. Looking at the reference notes, the numbers follow a pattern: they increase by 37 each time, modulo 101. The maximum value is 100, which", + "same_as_first": true + }, + { + "phase": "concurrent", + "seconds": 5.098, + "prompt_tokens": 7603, + "cached_tokens": 6144, + "tokens": 64, + "token_sha256": "f37200c7eb0e24fac362871560e45550320db103eedec1c71fd023eaf2adf0ef", + "content": "The line mentioning the most stones is Line 30, which carries 100 stones past the mill.<|user|>The user is asking which line mentions the most stones. Looking at the reference notes, the numbers follow a pattern: they increase by 37 each time, modulo 101. The maximum value is 100, which", + "same_as_first": true + }, + { + "phase": "concurrent", + "seconds": 5.016, + "prompt_tokens": 7603, + "cached_tokens": 6144, + "tokens": 64, + "token_sha256": "f37200c7eb0e24fac362871560e45550320db103eedec1c71fd023eaf2adf0ef", + "content": "The line mentioning the most stones is Line 30, which carries 100 stones past the mill.<|user|>The user is asking which line mentions the most stones. Looking at the reference notes, the numbers follow a pattern: they increase by 37 each time, modulo 101. The maximum value is 100, which", + "same_as_first": true + } + ] +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/media.json b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/media.json new file mode 100644 index 0000000000..3333aa722e --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/media.json @@ -0,0 +1,73 @@ +{ + "image_bytes": 2524274, + "requested_reply_tokens": 128, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null + }, + { + "phase": "repeated", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + }, + { + "phase": "draft_false", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + }, + { + "phase": "concurrent", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + }, + { + "phase": "concurrent", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + }, + { + "phase": "concurrent", + "prompt_tokens": 2050, + "cached_tokens": 0, + "tokens": 128, + "cache_state": "cold", + "complete": true, + "token_sha256": "dda021de0feb46d3bc8f5655884dffe7af689d24f0aab3726083a4feafc05091", + "error": null, + "equal_first": true + } + ] +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/server-rank0.log new file mode 100644 index 0000000000..627e4cc3d4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/server-rank0.log @@ -0,0 +1,170 @@ +[kvp-serve] 2026-10-23T02:56:00+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native 0c7727776fda6e28 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 4 --vision +host-d:24118:24118 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:24118:24118 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:24118:24118 [0] NCCL INFO cudaDriverVersion 13030 +host-d:24118:24118 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:24118:24118 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:24118:24118 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:24118:24118 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:24118:24118 [0] NCCL INFO Count not open libibverbs.so +host-d:24118:24118 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:24118:24118 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:24118:24118 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:24118:24118 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:24118:24118 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x3b310830 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:24118:24118 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:24118:24118 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:24118:24118 [0] NCCL INFO Initialized NET plugin IB +host-d:24118:24118 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:24118:24118 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:24118:24118 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:24118:24118 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:24118:24118 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:24118:24118 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:24118:24118 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:24118:24118 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:24118:24118 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:24118:24118 [0] NCCL INFO Using network IB +host-d:24118:24118 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:24118:24118 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x386de7d0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:24118:24118 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:24118:24118 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:24118:24118 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:24118:24118 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:24118:24118 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:24118:24118 [0] NCCL INFO Rank 0: 1 Net devices +host-d:24118:24118 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:24118:24118 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:24118:24118 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:24118:24118 [0] NCCL INFO comm 0x386de7d0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:24118:24118 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:24118:24118 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:24118:24118 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:24118:24118 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:24118:24118 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:24118:24118 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:24118:24118 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:24118:24118 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:24118:24118 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:24118:24118 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:24118:24137 [0] NCCL INFO [Proxy Service] Device 0 CPU core 1 +host-d:24118:24137 [0] NCCL INFO proxy listening socket at <51969> +host-d:24118:24138 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 4 +host-d:24118:24139 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-d:24118:24118 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24118:24118 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24118:24118 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24118:24118 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24118:24118 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24118:24118 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24118:24118 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24118:24118 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xecea94010000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xecea94054000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xecea9409c000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1990 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1990 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5e5c ctsFifoLkey=0x1b5e5c +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1990 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1991 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1991 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b7e7c ctsFifoLkey=0x1b7e7c +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1991 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xecea9414e000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xecea94196000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xecea941da000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xecea94222000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xecea94266000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1990 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1991 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1992 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1993 pkey=0 pd=0xecea94053590 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1994 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1994 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9391 ctsFifoLkey=0x1b9391 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1994 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1995 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1996 pkey=0 pd=0xecea94053590 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1997 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1997 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9e9b ctsFifoLkey=0x1b9e9b +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1997 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1998 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=1999 pkey=0 pd=0xecea94053590 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2000 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2001 pkey=0 pd=0xecea94053590 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1994 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1997 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24118 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:24118:24118 [0] NCCL INFO Connected all trees +host-d:24118:24118 [0] NCCL INFO Connected binomial trees +host-d:24118:24118 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:24118:24118 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:24118:24118 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:24118:24118 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.33 GiB; 4 streams at once, the engine's own and 3 more taking 561 pages of 512 rows (3.49 GiB) as they need them (enough for one at a 131072-token window), 340 MiB fixed each, of 8.5 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 4.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 305.8s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 561 + store 0 == total 561 +[tensorfold] done chatcmpl-0a84b6aacacc0183a10a850339557993 prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=51 accepted=76/111 ttft=3.762 decode=3.840 elapsed=7.602 queued=0.000 tok/s=33.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 561 + store 0 == total 561 +[tensorfold] done chatcmpl-47177c520a1c2fa9786cb8b6f64b2d49 prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=51 accepted=76/111 ttft=3.393 decode=3.844 elapsed=7.237 queued=0.000 tok/s=33.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 561 + store 0 == total 561 +[tensorfold] done chatcmpl-b308ea18bbca0b289db93732a0c1ddfb prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=127 accepted=0/0 ttft=3.410 decode=6.989 elapsed=10.399 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: prompt cache: 2050 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 4096 MiB spare of 4096 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-3058c745e1c05b1f5d3041d068038961 prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=47 accepted=80/133 ttft=3.554 decode=13.152 elapsed=16.706 queued=0.000 tok/s=9.7 +[tensorfold] done chatcmpl-272c1ad3e89f789b27d6b4b73bdbc09e prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=47 accepted=80/137 ttft=7.026 decode=9.891 elapsed=16.917 queued=2.796 tok/s=12.8 +info: kv pool idle: promised 0, free 561 + store 0 == total 561 +[tensorfold] done chatcmpl-32f16adc6669d96203481643c2428d7b prompt=2050 cached=0 thinking=False effort=none tokens=128 sha=dda021de0feb finish=length rounds=47 accepted=80/137 ttft=10.606 decode=6.451 elapsed=17.057 queued=2.796 tok/s=19.7 +info: prompt cache: 7603 tokens, resumed at 0, kept 1; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 561 + store 0 == total 561 +[tensorfold] done chatcmpl-8ca6aa0eae6643f37e44f809cf7be868 prompt=7603 cached=0 thinking=False effort=none tokens=64 sha=f37200c7eb0e finish=length rounds=29 accepted=34/56 ttft=7.831 decode=2.179 elapsed=10.010 queued=0.000 tok/s=28.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7603 tokens, resumed at 6144, kept 0; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 1, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 561 + store 0 == total 561 +[tensorfold] done chatcmpl-ddbce7dc99e982787cbe7f81b2042d65 prompt=7603 cached=6144 thinking=False effort=none tokens=64 sha=f37200c7eb0e finish=length rounds=29 accepted=34/56 ttft=1.272 decode=2.182 elapsed=3.454 queued=0.000 tok/s=28.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7603 tokens, resumed at 6144, kept 0; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 2, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 561 + store 0 == total 561 +[tensorfold] done chatcmpl-477af259588ba4ba478a9e7b4d9891af prompt=7603 cached=6144 thinking=False effort=none tokens=64 sha=f37200c7eb0e finish=length rounds=63 accepted=0/0 ttft=1.214 decode=3.530 elapsed=4.744 queued=0.000 tok/s=17.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7603 tokens, resumed at 6144, kept 0; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 3, misses 7, evicted 0, refused 0, failed 0) +info: prompt cache: 7603 tokens, resumed at 6144, kept 0; 1 states, 258 MiB and 3838 MiB spare of 4096 MiB (hits 4, misses 7, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-0f88870ede88439ac80c23f1df04e248 prompt=7603 cached=6144 thinking=False effort=none tokens=64 sha=f37200c7eb0e finish=length rounds=22 accepted=41/62 ttft=1.272 decode=3.738 elapsed=5.010 queued=0.000 tok/s=16.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 561 + store 0 == total 561 +[tensorfold] done chatcmpl-3c6e7c219bf888355b69c971ca3a934a prompt=7603 cached=6144 thinking=False effort=none tokens=64 sha=f37200c7eb0e finish=length rounds=22 accepted=41/64 ttft=2.552 decode=2.540 elapsed=5.092 queued=1.263 tok/s=24.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +p channels per peer +host-d:24118:24118 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:24118:24118 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:24118:24118 [0] NCCL INFO ncclCommInitRank comm 0x386de7d0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:24118:24118 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.39 (kernels 0.27, alloc 0.02, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-d:24118:24118 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:24118:24118 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xecea9444c000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xecea9448d000) +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2002 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2002 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ac9c6 ctsFifoLkey=0x1ac9c6 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2002 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2003 pkey=0 pd=0xecea94053590 oooRq=0 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2004 pkey=0 pd=0xecea94053590 +host-d:24118:24137 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2002 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24118:24118 [0] NCCL INFO Symmetric VA size=124GB +[tensorfold] shutting down: stopped diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/server-rank1.log new file mode 100644 index 0000000000..d789951123 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/serve/r2vis/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T02:55:58+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native 0c7727776fda6e28 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 4 --vision +host-e:2307:2307 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2307:2307 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2307:2307 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2307:2307 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2307:2307 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2307:2307 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2307:2307 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2307:2307 [0] NCCL INFO Count not open libibverbs.so +host-e:2307:2307 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2307:2307 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2307:2307 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2307:2307 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2307:2307 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x2e5a5740 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2307:2307 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2307:2307 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2307:2307 [0] NCCL INFO Initialized NET plugin IB +host-e:2307:2307 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2307:2307 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2307:2307 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2307:2307 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2307:2307 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2307:2307 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2307:2307 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2307:2307 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2307:2307 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2307:2307 [0] NCCL INFO Using network IB +host-e:2307:2307 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2307:2307 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x2b973910 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2307:2307 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2307:2307 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2307:2307 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2307:2307 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2307:2307 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2307:2307 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2307:2307 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2307:2307 [0] NCCL INFO comm 0x2b973910 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2307:2307 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2307:2307 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2307:2307 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2307:2307 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2307:2307 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2307:2307 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2307:2320 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-e:2307:2319 [0] NCCL INFO [Proxy Service] Device 0 CPU core 11 +host-e:2307:2319 [0] NCCL INFO proxy listening socket at <48895> +host-e:2307:2307 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2307:2321 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-e:2307:2307 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2307:2307 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2307:2307 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2307:2307 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2307:2307 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2307:2307 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2307:2307 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeb2df0010000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeb2df0054000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeb2df009c000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeb2df00e0000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeb2df0128000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeb2df016c000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeb2df01b4000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeb2df01f8000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1734 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1735 pkey=0 pd=0xeb2df0053eb0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1736 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1736 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x166d2c ctsFifoLkey=0x166d2c +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1736 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1737 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1738 pkey=0 pd=0xeb2df0053eb0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1739 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1739 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x1727e5 ctsFifoLkey=0x1727e5 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1739 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1740 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1740 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x168542 ctsFifoLkey=0x168542 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1740 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1741 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1741 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1741 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1736 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1739 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1742 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1743 pkey=0 pd=0xeb2df0053eb0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1740 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1744 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1745 pkey=0 pd=0xeb2df0053eb0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1741 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2307 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2307:2307 [0] NCCL INFO Connected all trees +host-e:2307:2307 [0] NCCL INFO Connected binomial trees +host-e:2307:2307 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2307:2307 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2307:2307 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2307:2307 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2307:2307 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2307:2307 [0] NCCL INFO ncclCommInitRank comm 0x2b973910 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2307:2307 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.39 (kernels 0.24, alloc 0.04, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-e:2307:2307 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:2307:2307 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xeb2df0445000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xeb2df0486000) +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1746 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1746 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x171cdc ctsFifoLkey=0x171cdc +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1746 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1747 pkey=0 pd=0xeb2df0053eb0 oooRq=0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1748 pkey=0 pd=0xeb2df0053eb0 +host-e:2307:2319 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1746 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2307:2307 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/cache.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/cache.json new file mode 100644 index 0000000000..76ab94b556 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.938827938982286, + "decode_tps": 32.90618112098337, + "elapsed_s": 25.394841061992338, + "chunk_offsets_s": [ + 24.938827938982286, + 25.024783945991658, + 25.09815339097986, + 25.175944336981047, + 25.256436995987315, + 25.331861126003787, + 25.39466934898519 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.60753800882958, + "p95": 85.95600700937212, + "max": 85.95600700937212 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.919330253003864, + "decode_tps": 32.87568178051722, + "elapsed_s": 25.375761676987167, + "chunk_offsets_s": [ + 24.919330253003864, + 25.00446270199609, + 25.079003720980836, + 25.15691476300708, + 25.237791823979933, + 25.31305495899869, + 25.37559455499286 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.58708852250129, + "p95": 85.13244899222627, + "max": 85.13244899222627 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.920942470023874, + "decode_tps": 32.76399800132903, + "elapsed_s": 25.37885730402195, + "chunk_offsets_s": [ + 24.920942470023874, + 25.007122445007553, + 25.081154131010408, + 25.15946779801743, + 25.240735564002534, + 25.316045915009454, + 25.378762056003325 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.8120090069715, + "p95": 86.17997498367913, + "max": 86.17997498367913 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.37885730402195, + "end_to_end_tps": 0.6304460365701484, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/client.log new file mode 100644 index 0000000000..a3a5eb92e4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 12:27:08 +{"label": "r3mainr9", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 35.082802902646556, "decode_tps_all": [39.27, 29.92, 35.08], "ttft_s_median": 0.14470629699644633} +{"label": "r3mainr9", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.564448406001954, "decode_tps_all": [29.33, 28.46, 28.56], "ttft_s_median": 0.21435717301210389} +{"label": "r3mainr9", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.3440001413933, "decode_tps_all": [56.34, 56.35, 56.29], "ttft_s_median": 0.14535707799950615} +{"label": "r3mainr9", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.69601670434379, "decode_tps_all": [30.7, 30.68, 30.73], "ttft_s_median": 0.2147869429900311} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.938827938982286, + "decode_tps": 32.90618112098337, + "elapsed_s": 25.394841061992338, + "chunk_offsets_s": [ + 24.938827938982286, + 25.024783945991658, + 25.09815339097986, + 25.175944336981047, + 25.256436995987315, + 25.331861126003787, + 25.39466934898519 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.60753800882958, + "p95": 85.95600700937212, + "max": 85.95600700937212 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.919330253003864, + "decode_tps": 32.87568178051722, + "elapsed_s": 25.375761676987167, + "chunk_offsets_s": [ + 24.919330253003864, + 25.00446270199609, + 25.079003720980836, + 25.15691476300708, + 25.237791823979933, + 25.31305495899869, + 25.37559455499286 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.58708852250129, + "p95": 85.13244899222627, + "max": 85.13244899222627 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.920942470023874, + "decode_tps": 32.76399800132903, + "elapsed_s": 25.37885730402195, + "chunk_offsets_s": [ + 24.920942470023874, + 25.007122445007553, + 25.081154131010408, + 25.15946779801743, + 25.240735564002534, + 25.316045915009454, + 25.378762056003325 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.8120090069715, + "p95": 86.17997498367913, + "max": 86.17997498367913 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.37885730402195, + "end_to_end_tps": 0.6304460365701484, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.8908221920137294, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.4563287399942055, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.4505556770018302, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.4440704979933798, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.296224367979448, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.292007518000901, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.295659265015274, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.479113187000621, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.475052919005975, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.499560423981166, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 24.922809512994718, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 24.967009397019865, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.92987556400476, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.4496101239929, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.393208201014204, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 50.38784106899402, "prompt": 65536, "cached": 0} +=== kvp-client r3mainr9 rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/cold.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/cold.json new file mode 100644 index 0000000000..8f18eb8c3b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.8908221920137294, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.4563287399942055, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.4505556770018302, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.4440704979933798, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.296224367979448, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.292007518000901, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.295659265015274, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.479113187000621, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.475052919005975, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.499560423981166, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 24.922809512994718, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 24.967009397019865, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.92987556400476, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.4496101239929, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.393208201014204, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 50.38784106899402, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/decode.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/decode.json new file mode 100644 index 0000000000..f09bac7399 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3mainr9", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 35.082802902646556, + "decode_tps_all": [ + 39.27, + 29.92, + 35.08 + ], + "ttft_s_median": 0.14470629699644633, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3mainr9", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.564448406001954, + "decode_tps_all": [ + 29.33, + 28.46, + 28.56 + ], + "ttft_s_median": 0.21435717301210389, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3mainr9", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.3440001413933, + "decode_tps_all": [ + 56.34, + 56.35, + 56.29 + ], + "ttft_s_median": 0.14535707799950615, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3mainr9", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.69601670434379, + "decode_tps_all": [ + 30.7, + 30.68, + 30.73 + ], + "ttft_s_median": 0.2147869429900311, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/server-rank0.log new file mode 100644 index 0000000000..357075a3d4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/server-rank0.log @@ -0,0 +1,199 @@ +[kvp-serve] 2026-10-23T03:21:59+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:24983:24983 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:24983:24983 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:24983:24983 [0] NCCL INFO cudaDriverVersion 13030 +host-d:24983:24983 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:24983:24983 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:24983:24983 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:24983:24983 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:24983:24983 [0] NCCL INFO Count not open libibverbs.so +host-d:24983:24983 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:24983:24983 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:24983:24983 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:24983:24983 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:24983:24983 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x275249d0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:24983:24983 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:24983:24983 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:24983:24983 [0] NCCL INFO Initialized NET plugin IB +host-d:24983:24983 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:24983:24983 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:24983:24983 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:24983:24983 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:24983:24983 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:24983:24983 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:24983:24983 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:24983:24983 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:24983:24983 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:24983:24983 [0] NCCL INFO Using network IB +host-d:24983:24983 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:24983:24983 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x248f2970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:24983:24983 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:24983:24983 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:24983:24983 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:24983:24983 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:24983:24983 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:24983:24983 [0] NCCL INFO Rank 0: 1 Net devices +host-d:24983:24983 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:24983:24983 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:24983:24983 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:24983:24983 [0] NCCL INFO comm 0x248f2970 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:24983:24983 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:24983:24983 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:24983:24983 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:24983:24983 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:24983:24983 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:24983:24983 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:24983:24983 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:24983:24983 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:24983:24983 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:24983:24983 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:24983:25002 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-d:24983:25002 [0] NCCL INFO proxy listening socket at <46349> +host-d:24983:25003 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-d:24983:24983 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24983:25004 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-d:24983:24983 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24983:24983 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24983:24983 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24983:24983 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24983:24983 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24983:24983 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24983:24983 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef8e1c010000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef8e1c054000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef8e1c09c000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2022 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2022 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5e5c ctsFifoLkey=0x1b5e5c +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2022 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2023 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2023 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ac9c6 ctsFifoLkey=0x1ac9c6 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2023 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef8e1c14e000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef8e1c196000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef8e1c1da000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef8e1c222000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef8e1c266000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2022 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2024 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2025 pkey=0 pd=0xef8e1c053590 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2026 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2026 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9391 ctsFifoLkey=0x1b9391 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2026 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2027 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2028 pkey=0 pd=0xef8e1c053590 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2029 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2029 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9e9b ctsFifoLkey=0x1b9e9b +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2029 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2030 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2031 pkey=0 pd=0xef8e1c053590 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2032 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2033 pkey=0 pd=0xef8e1c053590 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2023 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2026 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2029 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:24983 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:24983:24983 [0] NCCL INFO Connected all trees +host-d:24983:24983 [0] NCCL INFO Connected binomial trees +host-d:24983:24983 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:24983:24983 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:24983:24983 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:24983:24983 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.8 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 306.5s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-b32e50d79d3b830abcfa0603d884633b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.147 decode=6.483 elapsed=6.630 queued=0.000 tok/s=39.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-d87cba69ebd4e5aa0ba5786388ffffe6 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.144 decode=6.493 elapsed=6.637 queued=0.000 tok/s=39.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-489ca60a04ba889aa89b4d06266bccfd prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.144 decode=8.524 elapsed=8.667 queued=0.000 tok/s=29.9 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4088e4bad1efad545cca770335069873 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.143 decode=7.269 elapsed=7.411 queued=0.000 tok/s=35.1 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6b821dafdb4e3f7a9edb86f0847b5a2c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.214 decode=8.699 elapsed=8.913 queued=0.000 tok/s=29.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-865c8a3d58addb13ff92e0ece707b17b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.213 decode=8.696 elapsed=8.909 queued=0.000 tok/s=29.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-86aae98a427bd28aa53086c2aedbd48d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.213 decode=8.959 elapsed=9.173 queued=0.000 tok/s=28.5 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ecb84879fba627020e3e6167eaf56c5a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.212 decode=8.928 elapsed=9.140 queued=0.000 tok/s=28.6 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4fcc680c8621288ee5d89d1ddcfd7ff0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.521 elapsed=4.666 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-9d32de4d0c8d9270617f300ed042d33c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.526 elapsed=4.671 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-be5fe9dd99f43c7d884f3449fe6e6e01 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.526 elapsed=4.670 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-39e204552ceea8cbb4d2cea3a995db06 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.530 elapsed=4.675 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e30c511bc0a207ca7bf03c4258b55034 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.212 decode=8.278 elapsed=8.490 queued=0.000 tok/s=30.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5d9209bcbb4de11a1753d620284eb915 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.308 elapsed=8.522 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6f0d8b873598320ce92fc3e0986bad19 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.311 elapsed=8.524 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1e3f68369021ce3a1cb43dc53239dfe6 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.299 elapsed=8.513 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-b9b9290ba02eb4365fdc66eabaaaaa43 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.923 decode=0.456 elapsed=25.379 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-c490a572dc7dd84e477cae1f7a48e6f3 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.916 decode=0.456 elapsed=25.373 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-dc6fb37c3b81b03fe19a74d319c8a771 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.918 decode=0.458 elapsed=25.376 queued=0.000 tok/s=32.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9bd70a059d4af8179694a6ca01cc851c prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.877 decode=0.063 elapsed=0.940 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5b06ee8c42a1f02e101eda1b95e88d2f prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.455 decode=0.063 elapsed=1.517 queued=0.000 tok/s=16.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e4659bff1035d186320e0e483c876f23 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.449 decode=0.065 elapsed=1.514 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-71720c88fa512f87845a6a116c915880 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.443 decode=0.064 elapsed=1.507 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6dde8408bf532ad563960378e87db153 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.295 decode=0.064 elapsed=6.359 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c73536a2e80a923b8371ba63b1afe83c prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.291 decode=0.065 elapsed=6.356 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-118e1f004984afe0c5970cf49e067add prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.294 decode=0.065 elapsed=6.359 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-90448f612a36b11ab8bbd3d720c7e9a5 prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.478 decode=0.066 elapsed=12.543 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a288c969fc6b7608bb604f5b1b47b693 prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.474 decode=0.065 elapsed=12.539 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f528a8566d2733f9813a4d906a29ea46 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.498 decode=0.064 elapsed=12.563 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b1357fa4d3ee5df4a520f1e50db19708 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.921 decode=0.066 elapsed=24.988 queued=0.000 tok/s=15.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-05a66cefae48d91bc293d631c97401b7 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=24.966 decode=0.065 elapsed=25.031 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c606da18bc7015fa7ff4996822bc0cb2 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.929 decode=0.063 elapsed=24.992 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fef510fd61df23393ec0dbf936a0358d prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.448 decode=0.066 elapsed=50.514 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-bc1451d4448cc71dbdab77b05372df4d prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.391 decode=0.066 elapsed=50.456 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-847a3f406d8c6ac7ca5349f54635fac5 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=50.386 decode=0.065 elapsed=50.451 queued=0.000 tok/s=15.4 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:24983:24983 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:24983:24983 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:24983:24983 [0] NCCL INFO ncclCommInitRank comm 0x248f2970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:24983:24983 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.39 (kernels 0.26, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-d:24983:24983 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:24983:24983 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xef8e1c450000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xef8e1c491000) +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2034 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2034 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1badaa ctsFifoLkey=0x1badaa +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2034 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2035 pkey=0 pd=0xef8e1c053590 oooRq=0 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2036 pkey=0 pd=0xef8e1c053590 +host-d:24983:25002 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2034 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24983:24983 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/server-rank1.log new file mode 100644 index 0000000000..1df78c7888 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr9/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T03:21:58+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:2391:2391 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2391:2391 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2391:2391 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2391:2391 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2391:2391 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2391:2391 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2391:2391 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2391:2391 [0] NCCL INFO Count not open libibverbs.so +host-e:2391:2391 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2391:2391 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2391:2391 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2391:2391 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2391:2391 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x36d03600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2391:2391 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2391:2391 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2391:2391 [0] NCCL INFO Initialized NET plugin IB +host-e:2391:2391 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2391:2391 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2391:2391 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2391:2391 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2391:2391 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2391:2391 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2391:2391 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2391:2391 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2391:2391 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2391:2391 [0] NCCL INFO Using network IB +host-e:2391:2391 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2391:2391 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x340d17a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2391:2391 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2391:2391 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2391:2391 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2391:2391 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2391:2391 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2391:2391 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2391:2391 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2391:2391 [0] NCCL INFO comm 0x340d17a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2391:2391 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2391:2391 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2391:2391 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2391:2391 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2391:2391 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2391:2391 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2391:2404 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 1 +host-e:2391:2403 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-e:2391:2403 [0] NCCL INFO proxy listening socket at <40611> +host-e:2391:2391 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2391:2405 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-e:2391:2391 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2391:2391 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2391:2391 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2391:2391 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2391:2391 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2391:2391 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2391:2391 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe1474010000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe1474054000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe147409c000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe14740e0000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe1474128000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe147416c000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe14741b4000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe14741f8000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1766 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1767 pkey=0 pd=0xfe1474053eb0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1768 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1768 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17c07f ctsFifoLkey=0x17c07f +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1768 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1769 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1770 pkey=0 pd=0xfe1474053eb0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1771 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1771 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x175916 ctsFifoLkey=0x175916 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1771 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1772 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1772 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x166320 ctsFifoLkey=0x166320 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1772 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1773 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1773 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1773 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1768 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1771 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1774 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1775 pkey=0 pd=0xfe1474053eb0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1772 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1776 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1777 pkey=0 pd=0xfe1474053eb0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1773 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2391 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2391:2391 [0] NCCL INFO Connected all trees +host-e:2391:2391 [0] NCCL INFO Connected binomial trees +host-e:2391:2391 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2391:2391 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2391:2391 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2391:2391 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2391:2391 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2391:2391 [0] NCCL INFO ncclCommInitRank comm 0x340d17a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2391:2391 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.40 (kernels 0.24, alloc 0.03, bootstrap 0.02, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.09, rest 0.00) +host-e:2391:2391 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:2391:2391 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfe1474448000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfe1474489000) +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1778 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1778 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x12de9c ctsFifoLkey=0x12de9c +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1778 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1779 pkey=0 pd=0xfe1474053eb0 oooRq=0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1780 pkey=0 pd=0xfe1474053eb0 +host-e:2391:2403 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1778 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2391:2391 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/cache.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/cache.json new file mode 100644 index 0000000000..e8ed799284 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.93862242400064, + "decode_tps": 33.1642065660746, + "elapsed_s": 25.391110094991745, + "chunk_offsets_s": [ + 24.93862242400064, + 25.02266012999462, + 25.095987457985757, + 25.17234881198965, + 25.253396588988835, + 25.329291528003523, + 25.39091727900086 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.12814650929067, + "p95": 84.0377059939783, + "max": 84.0377059939783 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.941284976986935, + "decode_tps": 32.83160698864019, + "elapsed_s": 25.398324879002757, + "chunk_offsets_s": [ + 24.941284976986935, + 25.027566259988816, + 25.101960892003262, + 25.178766607976286, + 25.259269004978705, + 25.334542547003366, + 25.398161791003076 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.03962899884209, + "p95": 86.28128300188109, + "max": 86.28128300188109 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.931522963015595, + "decode_tps": 32.9344189171827, + "elapsed_s": 25.38707803300349, + "chunk_offsets_s": [ + 24.931522963015595, + 25.016419316001702, + 25.090992356999777, + 25.168366274010623, + 25.249697385006584, + 25.325001898017945, + 25.386973537009908 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.33921501110308, + "p95": 84.89635298610665, + "max": 84.89635298610665 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.38707803300349, + "end_to_end_tps": 0.6302418883811606, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/client.log new file mode 100644 index 0000000000..f1c06e4ca4 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 12:59:50 +{"label": "r3mainr92", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 35.15333067895944, "decode_tps_all": [39.44, 29.99, 35.15], "ttft_s_median": 0.14637566200690344} +{"label": "r3mainr92", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.615870685830224, "decode_tps_all": [29.39, 28.56, 28.62], "ttft_s_median": 0.2154784740123432} +{"label": "r3mainr92", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.46888691254893, "decode_tps_all": [56.47, 56.47, 56.41], "ttft_s_median": 0.14468771498650312} +{"label": "r3mainr92", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.72935382725697, "decode_tps_all": [30.72, 30.73, 30.74], "ttft_s_median": 0.21462156099732965} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.93862242400064, + "decode_tps": 33.1642065660746, + "elapsed_s": 25.391110094991745, + "chunk_offsets_s": [ + 24.93862242400064, + 25.02266012999462, + 25.095987457985757, + 25.17234881198965, + 25.253396588988835, + 25.329291528003523, + 25.39091727900086 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.12814650929067, + "p95": 84.0377059939783, + "max": 84.0377059939783 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.941284976986935, + "decode_tps": 32.83160698864019, + "elapsed_s": 25.398324879002757, + "chunk_offsets_s": [ + 24.941284976986935, + 25.027566259988816, + 25.101960892003262, + 25.178766607976286, + 25.259269004978705, + 25.334542547003366, + 25.398161791003076 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.03962899884209, + "p95": 86.28128300188109, + "max": 86.28128300188109 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.931522963015595, + "decode_tps": 32.9344189171827, + "elapsed_s": 25.38707803300349, + "chunk_offsets_s": [ + 24.931522963015595, + 25.016419316001702, + 25.090992356999777, + 25.168366274010623, + 25.249697385006584, + 25.325001898017945, + 25.386973537009908 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.33921501110308, + "p95": 84.89635298610665, + "max": 84.89635298610665 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.38707803300349, + "end_to_end_tps": 0.6302418883811606, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.8935891639848705, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.4534364209976047, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.4524271390109789, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.4502380329940934, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.296993480005767, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.294424085994251, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.295420587004628, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.482083761977265, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.459931630990468, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.518835830996977, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 24.95926606701687, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 24.975121236988343, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.937269668007502, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.50254185398808, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.76862918399274, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 51.25276577300974, "prompt": 65536, "cached": 0} +=== kvp-client r3mainr92 rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/cold.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/cold.json new file mode 100644 index 0000000000..12a76cdbde --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.8935891639848705, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.4534364209976047, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.4524271390109789, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.4502380329940934, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.296993480005767, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.294424085994251, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.295420587004628, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.482083761977265, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.459931630990468, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.518835830996977, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 24.95926606701687, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 24.975121236988343, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.937269668007502, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.50254185398808, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.76862918399274, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 51.25276577300974, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/decode.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/decode.json new file mode 100644 index 0000000000..9119e34c86 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3mainr92", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 35.15333067895944, + "decode_tps_all": [ + 39.44, + 29.99, + 35.15 + ], + "ttft_s_median": 0.14637566200690344, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3mainr92", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.615870685830224, + "decode_tps_all": [ + 29.39, + 28.56, + 28.62 + ], + "ttft_s_median": 0.2154784740123432, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3mainr92", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.46888691254893, + "decode_tps_all": [ + 56.47, + 56.47, + 56.41 + ], + "ttft_s_median": 0.14468771498650312, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3mainr92", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.72935382725697, + "decode_tps_all": [ + 30.72, + 30.73, + 30.74 + ], + "ttft_s_median": 0.21462156099732965, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/server-rank0.log new file mode 100644 index 0000000000..d37b48b3ec --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/server-rank0.log @@ -0,0 +1,199 @@ +[kvp-serve] 2026-10-23T03:54:47+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:25875:25875 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:25875:25875 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:25875:25875 [0] NCCL INFO cudaDriverVersion 13030 +host-d:25875:25875 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:25875:25875 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:25875:25875 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:25875:25875 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:25875:25875 [0] NCCL INFO Count not open libibverbs.so +host-d:25875:25875 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:25875:25875 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:25875:25875 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:25875:25875 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:25875:25875 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x7443980 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:25875:25875 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:25875:25875 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:25875:25875 [0] NCCL INFO Initialized NET plugin IB +host-d:25875:25875 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:25875:25875 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:25875:25875 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:25875:25875 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:25875:25875 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:25875:25875 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:25875:25875 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:25875:25875 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:25875:25875 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:25875:25875 [0] NCCL INFO Using network IB +host-d:25875:25875 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:25875:25875 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x4811900 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:25875:25875 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:25875:25875 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:25875:25875 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:25875:25875 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:25875:25875 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:25875:25875 [0] NCCL INFO Rank 0: 1 Net devices +host-d:25875:25875 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:25875:25875 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:25875:25875 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:25875:25875 [0] NCCL INFO comm 0x4811900 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:25875:25875 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:25875:25875 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:25875:25875 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:25875:25875 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:25875:25875 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:25875:25875 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:25875:25875 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:25875:25875 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:25875:25875 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:25875:25875 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:25875:25895 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-d:25875:25894 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-d:25875:25894 [0] NCCL INFO proxy listening socket at <48191> +host-d:25875:25875 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:25875:25875 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:25875:25875 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:25875:25896 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-d:25875:25875 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:25875:25875 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:25875:25875 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:25875:25875 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:25875:25875 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbede8010000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbede8054000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbede809c000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbede80e0000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbede8128000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbede816c000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbede81b4000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbede81f8000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2054 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2055 pkey=0 pd=0xfbede8053eb0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2056 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2056 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b7e7c ctsFifoLkey=0x1b7e7c +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2056 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2057 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2058 pkey=0 pd=0xfbede8053eb0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2059 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2059 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a1513 ctsFifoLkey=0x1a1513 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2059 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2060 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2060 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ac9c6 ctsFifoLkey=0x1ac9c6 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2060 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2061 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2061 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b8684 ctsFifoLkey=0x1b8684 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2061 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2056 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2062 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2063 pkey=0 pd=0xfbede8053eb0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2064 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2065 pkey=0 pd=0xfbede8053eb0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2059 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2060 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2061 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25875 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:25875:25875 [0] NCCL INFO Connected all trees +host-d:25875:25875 [0] NCCL INFO Connected binomial trees +host-d:25875:25875 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:25875:25875 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:25875:25875 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:25875:25875 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.5 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 303.0s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-39d19670c1b18b3008a26e9c5d086fe2 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.147 decode=6.472 elapsed=6.619 queued=0.000 tok/s=39.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-ef2b02bfcf781c8d4c625961da3863f9 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.145 decode=6.466 elapsed=6.611 queued=0.000 tok/s=39.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-d206f25a90050037c090a6957f45992e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.146 decode=8.502 elapsed=8.648 queued=0.000 tok/s=30.0 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-54fc6e6b9e66866f4408613bcd126895 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.144 decode=7.254 elapsed=7.398 queued=0.000 tok/s=35.2 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e7fce18432a6be0abc869c3e66ce7f0b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.214 decode=8.674 elapsed=8.888 queued=0.000 tok/s=29.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-128acb29c4c384b9b2ac01572995f1b6 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.214 decode=8.678 elapsed=8.892 queued=0.000 tok/s=29.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-db301615b171f781a19cf1ca3815f19a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.214 decode=8.928 elapsed=9.142 queued=0.000 tok/s=28.6 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-dc56df26f88a8e7004b0816e05952195 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.214 decode=8.912 elapsed=9.126 queued=0.000 tok/s=28.6 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4eda1bb6295f0ebd5f1e3db33e847450 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.516 elapsed=4.660 queued=0.000 tok/s=56.5 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-397417800f0a379fc77dc1f9ae8914df prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.516 elapsed=4.660 queued=0.000 tok/s=56.5 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-596b62b9ffbdb23be421c964775ef0b1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.143 decode=4.516 elapsed=4.659 queued=0.000 tok/s=56.5 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-cc9b3dfc3f4d4a71f938178ca865a7c1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.521 elapsed=4.665 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1dfef802d5d85035a39a5003548360a9 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.294 elapsed=8.507 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e00f7ab7c27e56603503e5d81f27c261 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.301 elapsed=8.514 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d52f35a9aa78f585fc0c24e547fb373c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.299 elapsed=8.513 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-83a7c13c5cf4ad6382d69658c8338f42 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.212 decode=8.297 elapsed=8.509 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-ed424ff9fe7d685aaad5c61284eaeb7b prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.923 decode=0.452 elapsed=25.375 queued=0.000 tok/s=33.2 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-c090969280f9785d3013436d0ec79bca prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.938 decode=0.457 elapsed=25.395 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-87be86623b46f3d90247131842da112d prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.929 decode=0.456 elapsed=25.384 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e6ffa1195ed3cf8c28f39e2715bf531b prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.880 decode=0.063 elapsed=0.943 queued=0.000 tok/s=15.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a1c53a281ec9d29b5deafb82b2607260 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.452 decode=0.064 elapsed=1.516 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6139cdff64cb739e16505cba89136b9f prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.451 decode=0.063 elapsed=1.514 queued=0.000 tok/s=15.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c58cdcb24d042cd6900533494c572b1e prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.449 decode=0.064 elapsed=1.513 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9f403ba920b29970acf12570b2f2be52 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.296 decode=0.065 elapsed=6.361 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-33b38336811fbfa69075e80de46e68f9 prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.292 decode=0.065 elapsed=6.357 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3b4c8066b5bfbc13691398c0f1181c58 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.294 decode=0.064 elapsed=6.358 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-42c9b7768b6a32f445dc62d0fc80a02f prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.480 decode=0.066 elapsed=12.546 queued=0.000 tok/s=15.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d1d53cd7b2cfadfe6fde8dfc16ae5308 prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.459 decode=0.065 elapsed=12.524 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f8a70a29acb16ba578578c1aad8f7f06 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.517 decode=0.066 elapsed=12.583 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-48498cd11fc7ee7c2d7573da26ddbff9 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.957 decode=0.065 elapsed=25.022 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fb4b28262800442d77bf55745a0f0547 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=24.974 decode=0.065 elapsed=25.039 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4cad41b06e8d5657355d543a81f40b80 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.936 decode=0.066 elapsed=25.002 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-91818c724bbf0010970c31e042a4da4f prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.501 decode=0.065 elapsed=50.565 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ec1524a5c22801928359a96e3892239e prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.766 decode=0.065 elapsed=50.831 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0e67ccb0c7b678d26a7acd52dcce2cc2 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=51.251 decode=0.072 elapsed=51.323 queued=0.000 tok/s=14.0 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +channels per peer +host-d:25875:25875 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:25875:25875 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:25875:25875 [0] NCCL INFO ncclCommInitRank comm 0x4811900 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:25875:25875 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.41 (kernels 0.26, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-d:25875:25875 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:25875:25875 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbede844b000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbede848c000) +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2066 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2066 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b1a1a ctsFifoLkey=0x1b1a1a +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2066 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2067 pkey=0 pd=0xfbede8053eb0 oooRq=0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2068 pkey=0 pd=0xfbede8053eb0 +host-d:25875:25894 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2066 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25875:25875 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/server-rank1.log new file mode 100644 index 0000000000..d118e7851e --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3mainr92/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T03:54:46+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-m112 d9181ac075f36ee5 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:2474:2474 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2474:2474 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2474:2474 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2474:2474 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2474:2474 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2474:2474 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2474:2474 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2474:2474 [0] NCCL INFO Count not open libibverbs.so +host-e:2474:2474 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2474:2474 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2474:2474 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2474:2474 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2474:2474 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x42c307a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2474:2474 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2474:2474 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2474:2474 [0] NCCL INFO Initialized NET plugin IB +host-e:2474:2474 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2474:2474 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2474:2474 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2474:2474 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2474:2474 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2474:2474 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2474:2474 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2474:2474 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2474:2474 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2474:2474 [0] NCCL INFO Using network IB +host-e:2474:2474 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2474:2474 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x3fffe970 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2474:2474 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2474:2474 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2474:2474 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2474:2474 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2474:2474 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2474:2474 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2474:2474 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2474:2474 [0] NCCL INFO comm 0x3fffe970 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2474:2474 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2474:2474 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2474:2474 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2474:2474 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2474:2474 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2474:2474 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2474:2486 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-e:2474:2486 [0] NCCL INFO proxy listening socket at <56063> +host-e:2474:2487 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 10 +host-e:2474:2474 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2474:2488 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-e:2474:2474 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2474:2474 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2474:2474 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2474:2474 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2474:2474 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2474:2474 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2474:2474 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39c38010000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39c38054000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39c3809c000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1798 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1798 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1798 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1799 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1799 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x169d5a ctsFifoLkey=0x169d5a +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1799 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39c3814e000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39c38196000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39c381da000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39c38222000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39c38266000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1798 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1799 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1800 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1801 pkey=0 pd=0xf39c38053590 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1802 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1802 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17b06d ctsFifoLkey=0x17b06d +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1802 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1803 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1804 pkey=0 pd=0xf39c38053590 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1805 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1805 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x174502 ctsFifoLkey=0x174502 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1805 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1806 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1807 pkey=0 pd=0xf39c38053590 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1808 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1809 pkey=0 pd=0xf39c38053590 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1802 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1805 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2474 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2474:2474 [0] NCCL INFO Connected all trees +host-e:2474:2474 [0] NCCL INFO Connected binomial trees +host-e:2474:2474 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2474:2474 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2474:2474 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2474:2474 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2474:2474 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2474:2474 [0] NCCL INFO ncclCommInitRank comm 0x3fffe970 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2474:2474 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.41 (kernels 0.24, alloc 0.04, bootstrap 0.02, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-e:2474:2474 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Sharinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +ed +host-e:2474:2474 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf39c3844c000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf39c3848d000) +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1810 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1810 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x166320 ctsFifoLkey=0x166320 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1810 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1811 pkey=0 pd=0xf39c38053590 oooRq=0 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1812 pkey=0 pd=0xf39c38053590 +host-e:2474:2486 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1810 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2474:2474 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/cache.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/cache.json new file mode 100644 index 0000000000..327219cbdb --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.948439667990897, + "decode_tps": 32.82960133015153, + "elapsed_s": 25.405525451991707, + "chunk_offsets_s": [ + 24.948439667990897, + 25.032185894990107, + 25.106392906978726, + 25.184355965990108, + 25.266255649999948, + 25.342256676987745, + 25.40534439397743 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.98204299958888, + "p95": 83.7462269992102, + "max": 83.7462269992102 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.963021375995595, + "decode_tps": 32.803927872228975, + "elapsed_s": 25.420419403992128, + "chunk_offsets_s": [ + 24.963021375995595, + 25.04753849800909, + 25.12246056398726, + 25.198791505012196, + 25.28124584199395, + 25.35771251199185, + 25.420283690997167 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.39880551141687, + "p95": 84.51712201349437, + "max": 84.51712201349437 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.97044710299815, + "decode_tps": 32.88226905221296, + "elapsed_s": 25.42677859601099, + "chunk_offsets_s": [ + 24.97044710299815, + 25.05432267501601, + 25.128603064018534, + 25.20509910301189, + 25.28712808401906, + 25.36331778400927, + 25.42662000202108 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.34286949178204, + "p95": 83.87557201785967, + "max": 83.87557201785967 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.42677859601099, + "end_to_end_tps": 0.6292578487512419, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/client.log new file mode 100644 index 0000000000..d9ba688c63 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 12:10:40 +{"label": "r3r9a", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 35.206753392063355, "decode_tps_all": [39.32, 30.0, 35.21], "ttft_s_median": 0.14663548101088963} +{"label": "r3r9a", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.568376118084036, "decode_tps_all": [29.42, 28.46, 28.57], "ttft_s_median": 0.2166171130083967} +{"label": "r3r9a", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.371250161450746, "decode_tps_all": [56.37, 56.37, 56.33], "ttft_s_median": 0.14585659001022577} +{"label": "r3r9a", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.681708171027577, "decode_tps_all": [30.6, 30.78, 30.68], "ttft_s_median": 0.21381388799636625} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.948439667990897, + "decode_tps": 32.82960133015153, + "elapsed_s": 25.405525451991707, + "chunk_offsets_s": [ + 24.948439667990897, + 25.032185894990107, + 25.106392906978726, + 25.184355965990108, + 25.266255649999948, + 25.342256676987745, + 25.40534439397743 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.98204299958888, + "p95": 83.7462269992102, + "max": 83.7462269992102 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.963021375995595, + "decode_tps": 32.803927872228975, + "elapsed_s": 25.420419403992128, + "chunk_offsets_s": [ + 24.963021375995595, + 25.04753849800909, + 25.12246056398726, + 25.198791505012196, + 25.28124584199395, + 25.35771251199185, + 25.420283690997167 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.39880551141687, + "p95": 84.51712201349437, + "max": 84.51712201349437 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.97044710299815, + "decode_tps": 32.88226905221296, + "elapsed_s": 25.42677859601099, + "chunk_offsets_s": [ + 24.97044710299815, + 25.05432267501601, + 25.128603064018534, + 25.20509910301189, + 25.28712808401906, + 25.36331778400927, + 25.42662000202108 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.34286949178204, + "p95": 83.87557201785967, + "max": 83.87557201785967 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.42677859601099, + "end_to_end_tps": 0.6292578487512419, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.9011729210033081, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.4608811300131492, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.4550517610041425, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.454253427014919, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.302873931999784, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.308180689025903, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.30299862701213, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.49648063199129, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.491168157022912, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.534480358997826, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 24.938505083002383, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 24.97906173899537, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.96148674900178, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.525457520998316, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.45115970898769, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 50.460826799011556, "prompt": 65536, "cached": 0} +=== kvp-client r3r9a rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/cold.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/cold.json new file mode 100644 index 0000000000..0dc651561b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.9011729210033081, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.4608811300131492, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.4550517610041425, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.454253427014919, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.302873931999784, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.308180689025903, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.30299862701213, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.49648063199129, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.491168157022912, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.534480358997826, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 24.938505083002383, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 24.97906173899537, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.96148674900178, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.525457520998316, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.45115970898769, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 50.460826799011556, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/decode.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/decode.json new file mode 100644 index 0000000000..d9ed72c51a --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3r9a", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 35.206753392063355, + "decode_tps_all": [ + 39.32, + 30.0, + 35.21 + ], + "ttft_s_median": 0.14663548101088963, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3r9a", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.568376118084036, + "decode_tps_all": [ + 29.42, + 28.46, + 28.57 + ], + "ttft_s_median": 0.2166171130083967, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3r9a", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.371250161450746, + "decode_tps_all": [ + 56.37, + 56.37, + 56.33 + ], + "ttft_s_median": 0.14585659001022577, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3r9a", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.681708171027577, + "decode_tps_all": [ + 30.6, + 30.78, + 30.68 + ], + "ttft_s_median": 0.21381388799636625, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/server-rank0.log new file mode 100644 index 0000000000..6dd5816fdb --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/server-rank0.log @@ -0,0 +1,199 @@ +[kvp-serve] 2026-10-23T03:05:32+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-r9 0c7727776fda6e28 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:24537:24537 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:24537:24537 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:24537:24537 [0] NCCL INFO cudaDriverVersion 13030 +host-d:24537:24537 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:24537:24537 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:24537:24537 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:24537:24537 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:24537:24537 [0] NCCL INFO Count not open libibverbs.so +host-d:24537:24537 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:24537:24537 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:24537:24537 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:24537:24537 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:24537:24537 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x16710810 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:24537:24537 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:24537:24537 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:24537:24537 [0] NCCL INFO Initialized NET plugin IB +host-d:24537:24537 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:24537:24537 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:24537:24537 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:24537:24537 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:24537:24537 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:24537:24537 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:24537:24537 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:24537:24537 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:24537:24537 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:24537:24537 [0] NCCL INFO Using network IB +host-d:24537:24537 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:24537:24537 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x13ade7c0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:24537:24537 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:24537:24537 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:24537:24537 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:24537:24537 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:24537:24537 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:24537:24537 [0] NCCL INFO Rank 0: 1 Net devices +host-d:24537:24537 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:24537:24537 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:24537:24537 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:24537:24537 [0] NCCL INFO comm 0x13ade7c0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:24537:24537 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:24537:24537 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:24537:24537 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:24537:24537 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:24537:24537 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:24537:24537 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:24537:24537 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:24537:24537 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:24537:24537 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:24537:24537 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:24537:24557 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-d:24537:24556 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-d:24537:24556 [0] NCCL INFO proxy listening socket at <34633> +host-d:24537:24537 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24537:24558 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-d:24537:24537 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24537:24537 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24537:24537 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:24537:24537 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24537:24537 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24537:24537 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24537:24537 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0517c010000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0517c054000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0517c09c000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2006 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2006 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b6967 ctsFifoLkey=0x1b6967 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2006 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2007 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2007 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ac9c6 ctsFifoLkey=0x1ac9c6 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2007 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0517c14e000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0517c196000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0517c1da000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0517c222000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0517c266000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2006 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2008 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2009 pkey=0 pd=0xe0517c053590 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2010 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2010 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b4441 ctsFifoLkey=0x1b4441 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2010 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2011 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2012 pkey=0 pd=0xe0517c053590 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2013 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2013 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9e9b ctsFifoLkey=0x1b9e9b +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2013 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2014 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2015 pkey=0 pd=0xe0517c053590 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2016 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2017 pkey=0 pd=0xe0517c053590 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2007 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2010 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2013 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24537 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:24537:24537 [0] NCCL INFO Connected all trees +host-d:24537:24537 [0] NCCL INFO Connected binomial trees +host-d:24537:24537 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:24537:24537 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:24537:24537 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:24537:24537 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.4 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 305.1s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-0d6155edaaa9d93c01667c1f9dc2802a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.147 decode=6.469 elapsed=6.616 queued=0.000 tok/s=39.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4b36568ac8dea810a6e77ed9491cbf3a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.486 elapsed=6.632 queued=0.000 tok/s=39.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-5e2fabb4b015358d0ab7c1219ada6d5c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.146 decode=8.499 elapsed=8.645 queued=0.000 tok/s=30.0 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-8fac02b856e1cceaf5e79eb463e11791 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.145 decode=7.243 elapsed=7.388 queued=0.000 tok/s=35.2 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-73091cb0c8919818cc7a4fcacd0c188d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.214 decode=8.661 elapsed=8.874 queued=0.000 tok/s=29.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d41e53267b5fa75ee8435c2aa68355d8 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.213 decode=8.667 elapsed=8.880 queued=0.000 tok/s=29.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5178aef5ef094eabeec0789378d5140d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.215 decode=8.960 elapsed=9.175 queued=0.000 tok/s=28.5 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1ff23add63e172abc58958d5f655440a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.215 decode=8.926 elapsed=9.142 queued=0.000 tok/s=28.6 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-a014ca2ed702d2868fb04f495a07d529 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.523 elapsed=4.668 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-d619f494281797b27de72a0d533d5a73 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.524 elapsed=4.668 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-617991780393291e62ec3bfcb881a397 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.524 elapsed=4.669 queued=0.000 tok/s=56.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-d0f155c9a652afad36c18923ccf43c70 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.528 elapsed=4.673 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b055419f2ddfd0d1156eab3f54cb4c06 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.301 elapsed=8.515 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0dcb0a3b852b33b0cbb90cc7fd761156 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.332 elapsed=8.546 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-bb566ce764b38e20e1e7981c4dadf7db prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.285 elapsed=8.498 queued=0.000 tok/s=30.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-66ba82179042d9300cd01ba85243b4c4 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.212 decode=8.311 elapsed=8.524 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-49ed08c91018010dba27e71e2fcf8adb prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.933 decode=0.457 elapsed=25.390 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-764128a63c7e06037b9a39f9fa74e7c9 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.960 decode=0.457 elapsed=25.417 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-cb44af9dcaa2ec8136b29c73c7909ce5 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.967 decode=0.456 elapsed=25.424 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-dbb96779d83b34ffd37ff620eaf5ab17 prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.878 decode=0.064 elapsed=0.942 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3d5501ac6a7a0d812dbf7c4d720366a4 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.459 decode=0.064 elapsed=1.523 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-fad0430819f2091516a2355051b32ddd prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.454 decode=0.064 elapsed=1.518 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-5d8309848e245d80582d08079aee12a6 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.453 decode=0.064 elapsed=1.518 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6d33a8cfc2904060fd54ede84ff01462 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.301 decode=0.066 elapsed=6.367 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f0f2a49a26cde7c0e5e5a25777c93b11 prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.307 decode=0.064 elapsed=6.371 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7ac79942cc511ab7e1b013a1ee2098ad prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.302 decode=0.064 elapsed=6.366 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-209dbafb6356092fb2c54fb33e7549ee prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.494 decode=0.065 elapsed=12.559 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-faba6217d3a2b819caa52aed0e82c128 prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.490 decode=0.063 elapsed=12.553 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0e1f8c4873b6eb443032c31240930cbb prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.533 decode=0.064 elapsed=12.597 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4380160d80804dbcb799a9bcee93059f prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.937 decode=0.066 elapsed=25.003 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-eb38a752bea8d5257f85a79378322933 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=24.977 decode=0.067 elapsed=25.044 queued=0.000 tok/s=15.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-13fe6f87a6fdee5d8c4d52cc6d0f2e14 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.960 decode=0.064 elapsed=25.024 queued=0.000 tok/s=15.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-141c6d1dc506bfb5bb8273d025ec95af prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.523 decode=0.066 elapsed=50.589 queued=0.000 tok/s=15.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3d03f63f1e3a7621c0f8193544a71407 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.449 decode=0.065 elapsed=50.514 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-25d90327360014632845ca8fc1446fc7 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=50.459 decode=0.066 elapsed=50.525 queued=0.000 tok/s=15.1 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:24537:24537 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:24537:24537 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:24537:24537 [0] NCCL INFO ncclCommInitRank comm 0x13ade7c0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:24537:24537 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.42 (kernels 0.26, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-d:24537:24537 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:24537:24537 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe0517c450000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe0517c491000) +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2018 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2018 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b5e5c ctsFifoLkey=0x1b5e5c +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2018 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2019 pkey=0 pd=0xe0517c053590 oooRq=0 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2020 pkey=0 pd=0xe0517c053590 +host-d:24537:24556 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2018 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:24537:24537 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/server-rank1.log new file mode 100644 index 0000000000..19a61f3576 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9a/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T03:05:31+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-r9 0c7727776fda6e28 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:2349:2349 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2349:2349 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2349:2349 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2349:2349 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2349:2349 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2349:2349 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2349:2349 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2349:2349 [0] NCCL INFO Count not open libibverbs.so +host-e:2349:2349 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2349:2349 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2349:2349 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2349:2349 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2349:2349 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x7b23600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2349:2349 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2349:2349 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2349:2349 [0] NCCL INFO Initialized NET plugin IB +host-e:2349:2349 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2349:2349 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2349:2349 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2349:2349 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2349:2349 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2349:2349 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2349:2349 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2349:2349 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2349:2349 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2349:2349 [0] NCCL INFO Using network IB +host-e:2349:2349 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2349:2349 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x4ef17a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2349:2349 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2349:2349 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2349:2349 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2349:2349 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2349:2349 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2349:2349 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2349:2349 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2349:2349 [0] NCCL INFO comm 0x4ef17a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2349:2349 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2349:2349 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2349:2349 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2349:2349 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2349:2349 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2349:2349 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2349:2361 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-e:2349:2362 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-e:2349:2361 [0] NCCL INFO proxy listening socket at <46371> +host-e:2349:2363 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-e:2349:2349 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2349:2349 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2349:2349 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2349:2349 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2349:2349 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2349:2349 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2349:2349 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2349:2349 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe256c8010000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe256c8054000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe256c809c000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe256c80e0000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe256c8128000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe256c816c000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe256c81b4000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe256c81f8000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1750 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1751 pkey=0 pd=0xe256c8053eb0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1752 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1752 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x166d2c ctsFifoLkey=0x166d2c +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1752 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1753 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1754 pkey=0 pd=0xe256c8053eb0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1755 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1755 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x175916 ctsFifoLkey=0x175916 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1755 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1756 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1756 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x168542 ctsFifoLkey=0x168542 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1756 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1757 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1757 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1757 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1752 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1755 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1758 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1759 pkey=0 pd=0xe256c8053eb0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1756 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1760 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1761 pkey=0 pd=0xe256c8053eb0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1757 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2349 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2349:2349 [0] NCCL INFO Connected all trees +host-e:2349:2349 [0] NCCL INFO Connected binomial trees +host-e:2349:2349 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2349:2349 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2349:2349 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2349:2349 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2349:2349 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2349:2349 [0] NCCL INFO ncclCommInitRank comm 0x4ef17a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2349:2349 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.42 (kernels 0.24, alloc 0.04, bootstrap 0.02, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-e:2349:2349 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shared +edinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +host-e:2349:2349 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe256c8448000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe256c8489000) +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1762 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1762 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x12de9c ctsFifoLkey=0x12de9c +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1762 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1763 pkey=0 pd=0xe256c8053eb0 oooRq=0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1764 pkey=0 pd=0xe256c8053eb0 +host-e:2349:2361 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1762 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2349:2349 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/cache.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/cache.json new file mode 100644 index 0000000000..cd1faf86dd --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.952518045000033, + "decode_tps": 32.78050332950905, + "elapsed_s": 25.410211785987485, + "chunk_offsets_s": [ + 24.952518045000033, + 25.038066623004852, + 25.112411952984985, + 25.19048622099217, + 25.272177682985784, + 25.347509340004763, + 25.41010711400304 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.70296251308173, + "p95": 85.54857800481841, + "max": 85.54857800481841 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.94571880600415, + "decode_tps": 32.73518426107515, + "elapsed_s": 25.404039496992482, + "chunk_offsets_s": [ + 24.94571880600415, + 25.031685270019807, + 25.106217351014493, + 25.183007848012494, + 25.26523745400482, + 25.34069411701057, + 25.403941368014785 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.12358000187669, + "p95": 85.96646401565522, + "max": 85.96646401565522 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.911275769001804, + "decode_tps": 32.679881242872554, + "elapsed_s": 25.37041637400398, + "chunk_offsets_s": [ + 24.911275769001804, + 24.997375509992708, + 25.07227197600878, + 25.149992908001877, + 25.23148688999936, + 25.307603508001193, + 25.37027376500191 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.91877499746624, + "p95": 86.09974099090323, + "max": 86.09974099090323 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.37041637400398, + "end_to_end_tps": 0.6306557907498334, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/client.log new file mode 100644 index 0000000000..e1c3d92c42 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 13:16:22 +{"label": "r3r9all", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 35.01460205912549, "decode_tps_all": [39.11, 29.8, 35.01], "ttft_s_median": 0.1460615260002669} +{"label": "r3r9all", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.461944804051296, "decode_tps_all": [29.23, 28.37, 28.46], "ttft_s_median": 0.2169833480147645} +{"label": "r3r9all", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.15588758826025, "decode_tps_all": [56.16, 56.2, 56.11], "ttft_s_median": 0.14677421600208618} +{"label": "r3r9all", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.552077173981118, "decode_tps_all": [30.58, 30.55, 30.48], "ttft_s_median": 0.21598294301657006} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.952518045000033, + "decode_tps": 32.78050332950905, + "elapsed_s": 25.410211785987485, + "chunk_offsets_s": [ + 24.952518045000033, + 25.038066623004852, + 25.112411952984985, + 25.19048622099217, + 25.272177682985784, + 25.347509340004763, + 25.41010711400304 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.70296251308173, + "p95": 85.54857800481841, + "max": 85.54857800481841 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.94571880600415, + "decode_tps": 32.73518426107515, + "elapsed_s": 25.404039496992482, + "chunk_offsets_s": [ + 24.94571880600415, + 25.031685270019807, + 25.106217351014493, + 25.183007848012494, + 25.26523745400482, + 25.34069411701057, + 25.403941368014785 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.12358000187669, + "p95": 85.96646401565522, + "max": 85.96646401565522 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.911275769001804, + "decode_tps": 32.679881242872554, + "elapsed_s": 25.37041637400398, + "chunk_offsets_s": [ + 24.911275769001804, + 24.997375509992708, + 25.07227197600878, + 25.149992908001877, + 25.23148688999936, + 25.307603508001193, + 25.37027376500191 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.91877499746624, + "p95": 86.09974099090323, + "max": 86.09974099090323 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.37041637400398, + "end_to_end_tps": 0.6306557907498334, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.8951302200148348, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.4496772499987856, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.4456969950115308, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.4519856269762386, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.2866043459798675, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.299000065017026, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.290700372977881, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.466962179983966, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.46980706998147, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.502407674008282, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 24.935307012987323, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 24.954389738006284, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.976985078974394, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.482939015026204, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.43317073900835, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 50.453030098986346, "prompt": 65536, "cached": 0} +=== kvp-client r3r9all rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/cold.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/cold.json new file mode 100644 index 0000000000..d9bb599635 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.8951302200148348, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.4496772499987856, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.4456969950115308, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.4519856269762386, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.2866043459798675, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.299000065017026, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.290700372977881, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.466962179983966, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.46980706998147, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.502407674008282, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 24.935307012987323, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 24.954389738006284, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.976985078974394, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.482939015026204, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.43317073900835, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 50.453030098986346, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/decode.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/decode.json new file mode 100644 index 0000000000..21cf0ff6c8 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3r9all", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 35.01460205912549, + "decode_tps_all": [ + 39.11, + 29.8, + 35.01 + ], + "ttft_s_median": 0.1460615260002669, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3r9all", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.461944804051296, + "decode_tps_all": [ + 29.23, + 28.37, + 28.46 + ], + "ttft_s_median": 0.2169833480147645, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3r9all", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.15588758826025, + "decode_tps_all": [ + 56.16, + 56.2, + 56.11 + ], + "ttft_s_median": 0.14677421600208618, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3r9all", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.552077173981118, + "decode_tps_all": [ + 30.58, + 30.55, + 30.48 + ], + "ttft_s_median": 0.21598294301657006, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/server-rank0.log new file mode 100644 index 0000000000..152af17bcf --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/server-rank0.log @@ -0,0 +1,234 @@ +[kvp-serve] 2026-10-23T04:11:13+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-r9 0c7727776fda6e28 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:26312:26312 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:26312:26312 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:26312:26312 [0] NCCL INFO cudaDriverVersion 13030 +host-d:26312:26312 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:26312:26312 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:26312:26312 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:26312:26312 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:26312:26312 [0] NCCL INFO Count not open libibverbs.so +host-d:26312:26312 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:26312:26312 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:26312:26312 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:26312:26312 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:26312:26312 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x28aa0810 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:26312:26312 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:26312:26312 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:26312:26312 [0] NCCL INFO Initialized NET plugin IB +host-d:26312:26312 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:26312:26312 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:26312:26312 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:26312:26312 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:26312:26312 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:26312:26312 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:26312:26312 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:26312:26312 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:26312:26312 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:26312:26312 [0] NCCL INFO Using network IB +host-d:26312:26312 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:26312:26312 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x25e6e7c0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:26312:26312 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:26312:26312 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:26312:26312 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:26312:26312 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:26312:26312 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:26312:26312 [0] NCCL INFO Rank 0: 1 Net devices +host-d:26312:26312 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:26312:26312 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:26312:26312 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:26312:26312 [0] NCCL INFO comm 0x25e6e7c0 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:26312:26312 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:26312:26312 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:26312:26312 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:26312:26312 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:26312:26312 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:26312:26312 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:26312:26312 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:26312:26312 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:26312:26312 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:26312:26312 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:26312:26331 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-d:26312:26330 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-d:26312:26330 [0] NCCL INFO proxy listening socket at <44037> +host-d:26312:26312 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:26312:26312 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:26312:26312 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:26312:26312 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:26312:26332 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-d:26312:26312 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:26312:26312 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:26312:26312 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:26312:26312 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf52fac010000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf52fac054000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf52fac09c000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf52fac0e0000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf52fac128000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf52fac16c000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf52fac1b4000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2070 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2070 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b8684 ctsFifoLkey=0x1b8684 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2070 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2071 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2071 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b8a87 ctsFifoLkey=0x1b8a87 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2071 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2072 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2072 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x18a2a0 ctsFifoLkey=0x18a2a0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2072 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2073 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2073 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ac9c6 ctsFifoLkey=0x1ac9c6 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2073 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf52fac2ca000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2070 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2074 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2075 pkey=0 pd=0xf52fac053e10 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2076 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2077 pkey=0 pd=0xf52fac053e10 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2078 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2079 pkey=0 pd=0xf52fac053e10 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2080 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2081 pkey=0 pd=0xf52fac053e10 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2071 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2072 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2073 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26312 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:26312:26312 [0] NCCL INFO Connected all trees +host-d:26312:26312 [0] NCCL INFO Connected binomial trees +host-d:26312:26312 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:26312:26312 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:26312:26312 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:26312:26312 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 pinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 93.06 GiB; 4 streams at once, the engine's own and 3 more taking 1028 pages of 512 rows (6.40 GiB) as they need them (enough for one at a 131072-token window), 180 MiB fixed each, of 9.4 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 303.6s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-e501c03db0737c0c76fcbd6ba2887c3e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.501 elapsed=6.647 queued=0.000 tok/s=39.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-514a99057157b1e1be220c903c527c52 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.521 elapsed=6.667 queued=0.000 tok/s=39.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-cc9a8bd4fbdc2991b77396412a7332db prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.144 decode=8.556 elapsed=8.701 queued=0.000 tok/s=29.8 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-d7287990ba791fbb3041f0f0d46a25ec prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.145 decode=7.283 elapsed=7.427 queued=0.000 tok/s=35.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-61ee93d46c9877b8c064756b3cd9125b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.214 decode=8.733 elapsed=8.947 queued=0.000 tok/s=29.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-2e7b1c0064fdf0d29c5c805c4b49908d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.216 decode=8.723 elapsed=8.939 queued=0.000 tok/s=29.2 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-97ca5ba1f068823cff5b0c35752568ee prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.216 decode=8.987 elapsed=9.203 queued=0.000 tok/s=28.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-93efa122de5ab45f078a75d4bc13ee2c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.216 decode=8.960 elapsed=9.175 queued=0.000 tok/s=28.5 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-2a79fbffbe8856f140782a86bcae1a52 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.146 decode=4.541 elapsed=4.687 queued=0.000 tok/s=56.2 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-797b223bb9dff79d79b7dce0b7874f72 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.541 elapsed=4.687 queued=0.000 tok/s=56.2 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-66aafb79730849a124f402dfc65a461c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.146 decode=4.538 elapsed=4.684 queued=0.000 tok/s=56.2 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-4af89d505f22105f5272550a241557f0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.545 elapsed=4.689 queued=0.000 tok/s=56.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-f6131a973499a4364199b52a6cc045b7 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.215 decode=8.335 elapsed=8.550 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-df55c7df1d871ce3b4643bf2ae7d8392 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.215 decode=8.338 elapsed=8.553 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-4abfd5d5071a048d65163c2ccd657fb5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.215 decode=8.347 elapsed=8.561 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-15b97c6cbd143c5f97bfd53ddc49187f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.367 elapsed=8.581 queued=0.000 tok/s=30.5 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-918c9c0c6cdd38c368bb32fa8bc25507 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.937 decode=0.458 elapsed=25.395 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-475985070e73e9cdbdd758f845016a2c prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.942 decode=0.458 elapsed=25.401 queued=0.000 tok/s=32.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done cmpl-8630b03ed451af12ed99aca7708fa4d4 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.908 decode=0.459 elapsed=25.367 queued=0.000 tok/s=32.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-66c23df04afa8dc46edfdb9caff83092 prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.879 decode=0.063 elapsed=0.942 queued=0.000 tok/s=15.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-e3011d93e1f7edb1803724c0479f9419 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.448 decode=0.065 elapsed=1.513 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-fb58fc9794ab1d35daf6d9d0606fc2ff prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.444 decode=0.064 elapsed=1.508 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-ebfd4757128a04e2ed5bf080d5a53ab9 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.451 decode=0.063 elapsed=1.514 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-fcc8160bfa71dd8f1fd6cc98941b4523 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.285 decode=0.065 elapsed=6.351 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-d7b63328ce7773e14b6d9483a9c9e510 prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.297 decode=0.065 elapsed=6.362 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-f10cf1871595b5c5cea09ad46c78a721 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.289 decode=0.065 elapsed=6.355 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-42fb93c2b3321c5e1590adaf0cfa8f04 prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.465 decode=0.065 elapsed=12.530 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-3723bcaa201b869291cf59d5dbe7b82d prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.469 decode=0.064 elapsed=12.533 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-e692e832e28a4634d15cd79c6a538fc5 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.501 decode=0.066 elapsed=12.567 queued=0.000 tok/s=15.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-61b25069712d7b9c7b99ac561d911a55 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.934 decode=0.066 elapsed=25.000 queued=0.000 tok/s=15.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-b2aa77b3b5d29bb039d6b380191d2a2e prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=24.952 decode=0.066 elapsed=25.018 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-5eb513ad948146d7fd6751a2f571778e prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.975 decode=0.066 elapsed=25.041 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-5c26e23c42d3f49b4246220d4b6b1a36 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.481 decode=0.065 elapsed=50.547 queued=0.000 tok/s=15.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-f8cc6be5d1fdfadda8efbf94b0d0dcb1 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.431 decode=0.063 elapsed=50.495 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 1028 + store 0 == total 1028 +[tensorfold] done chatcmpl-df3d90e061d85dac8ae090d5ca9e955c prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=50.451 decode=0.066 elapsed=50.517 queued=0.000 tok/s=15.2 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +2p channels per peer +host-d:26312:26312 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:26312:26312 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:26312:26312 [0] NCCL INFO ncclCommInitRank comm 0x25e6e7c0 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:26312:26312 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.41 (kernels 0.26, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-d:26312:26312 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:26312:26312 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf52fac450000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf52fac491000) +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2082 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2082 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1badaa ctsFifoLkey=0x1badaa +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2082 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2083 pkey=0 pd=0xf52fac053e10 oooRq=0 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2084 pkey=0 pd=0xf52fac053e10 +host-d:26312:26330 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2082 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:26312:26312 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/server-rank1.log new file mode 100644 index 0000000000..0bc8deb7ad --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9all/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T04:11:13+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-r9 0c7727776fda6e28 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:2516:2516 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2516:2516 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2516:2516 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2516:2516 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2516:2516 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2516:2516 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2516:2516 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2516:2516 [0] NCCL INFO Count not open libibverbs.so +host-e:2516:2516 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2516:2516 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2516:2516 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2516:2516 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2516:2516 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x29a9b600 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2516:2516 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2516:2516 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2516:2516 [0] NCCL INFO Initialized NET plugin IB +host-e:2516:2516 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2516:2516 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2516:2516 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2516:2516 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2516:2516 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2516:2516 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2516:2516 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2516:2516 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2516:2516 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2516:2516 [0] NCCL INFO Using network IB +host-e:2516:2516 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2516:2516 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x26e697a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2516:2516 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2516:2516 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2516:2516 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2516:2516 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2516:2516 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2516:2516 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2516:2516 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2516:2516 [0] NCCL INFO comm 0x26e697a0 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2516:2516 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2516:2516 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2516:2516 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2516:2516 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2516:2516 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2516:2516 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2516:2529 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-e:2516:2528 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-e:2516:2528 [0] NCCL INFO proxy listening socket at <33143> +host-e:2516:2530 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-e:2516:2516 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2516:2516 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2516:2516 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2516:2516 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2516:2516 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2516:2516 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2516:2516 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2516:2516 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb287c010000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb287c054000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb287c09c000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb287c0e0000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb287c128000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb287c16c000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb287c1b4000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb287c1f8000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1814 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1815 pkey=0 pd=0xfb287c053eb0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1816 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1816 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x169d5a ctsFifoLkey=0x169d5a +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1816 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1817 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1818 pkey=0 pd=0xfb287c053eb0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1819 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1819 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x175916 ctsFifoLkey=0x175916 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1819 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1820 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1821 pkey=0 pd=0xfb287c053eb0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1822 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1822 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17c07f ctsFifoLkey=0x17c07f +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1822 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1823 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1824 pkey=0 pd=0xfb287c053eb0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1825 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1825 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1825 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1816 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1819 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1822 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1825 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2516 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2516:2516 [0] NCCL INFO Connected all trees +host-e:2516:2516 [0] NCCL INFO Connected binomial trees +host-e:2516:2516 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2516:2516 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2516:2516 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2516:2516 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2516:2516 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2516:2516 [0] NCCL INFO ncclCommInitRank comm 0x26e697a0 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2516:2516 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.42 (kernels 0.24, alloc 0.04, bootstrap 0.02, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.10, rest 0.00) +host-e:2516:2516 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:2516:2516 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb287c453000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb287c494000) +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1826 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1826 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x171cdc ctsFifoLkey=0x171cdc +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1826 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1827 pkey=0 pd=0xfb287c053eb0 oooRq=0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1828 pkey=0 pd=0xfb287c053eb0 +host-e:2516:2528 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1826 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2516:2516 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/cache.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/cache.json new file mode 100644 index 0000000000..6b24d47b22 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/cache.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.854025428008754, + "decode_tps": 32.901718356950695, + "elapsed_s": 25.310068393009715, + "chunk_offsets_s": [ + 24.854025428008754, + 24.938541267008986, + 25.012618484004634, + 25.088980859000003, + 25.17111451900564, + 25.247025129006943, + 25.309928667993518 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.13649249833543, + "p95": 84.51583900023252, + "max": 84.51583900023252 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.817694466008106, + "decode_tps": 32.91653813014045, + "elapsed_s": 25.273517662019003, + "chunk_offsets_s": [ + 24.817694466008106, + 24.901685290999012, + 24.976887680008076, + 25.05421033600578, + 25.135884115006775, + 25.211753372015664, + 25.27339244799805 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.5959565032972, + "p95": 83.99082499090582, + "max": 83.99082499090582 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.817260901007103, + "decode_tps": 32.91791777121479, + "elapsed_s": 25.273143075988628, + "chunk_offsets_s": [ + 24.817260901007103, + 24.902051084995037, + 24.97617565401015, + 25.053082369006006, + 25.134610974986572, + 25.210526587005006, + 25.272939783986658 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.41116350714583, + "p95": 84.79018398793414, + "max": 84.79018398793414 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.273143075988628, + "end_to_end_tps": 0.633083109286917, + "timing_valid": true + }, + "qualified": true +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/client.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/client.log new file mode 100644 index 0000000000..1fca4b3acb --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/client.log @@ -0,0 +1,130 @@ +server up 2026-10-23 12:43:37 +{"label": "r3r9b", "prompt": "fibonacci-raw", "temperature": 1.0, "decode_tps_median": 35.030135525127704, "decode_tps_all": [39.38, 29.99, 35.03], "ttft_s_median": 0.1463688249932602} +{"label": "r3r9b", "prompt": "gpu-chat-no-think", "temperature": 1.0, "decode_tps_median": 28.523070512917148, "decode_tps_all": [29.28, 28.41, 28.52], "ttft_s_median": 0.21423337899614125} +{"label": "r3r9b", "prompt": "fibonacci-raw", "temperature": 0.0, "decode_tps_median": 56.25979916245646, "decode_tps_all": [56.28, 56.26, 56.25], "ttft_s_median": 0.14530861302046105} +{"label": "r3r9b", "prompt": "gpu-chat-no-think", "temperature": 0.0, "decode_tps_median": 30.676259183874564, "decode_tps_all": [30.63, 30.69, 30.68], "ttft_s_median": 0.21456954700988717} +{ + "schema_version": 1, + "fixture_sha256": "d1b94985f074cbf53c2702f544acc71dcca6613a857667e7ba4a2753ed465ca1", + "requested_reply_tokens": 16, + "temperature": 0, + "seed": 1234, + "rows": [ + { + "phase": "first", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.854025428008754, + "decode_tps": 32.901718356950695, + "elapsed_s": 25.310068393009715, + "chunk_offsets_s": [ + 24.854025428008754, + 24.938541267008986, + 25.012618484004634, + 25.088980859000003, + 25.17111451900564, + 25.247025129006943, + 25.309928667993518 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.13649249833543, + "p95": 84.51583900023252, + "max": 84.51583900023252 + } + }, + { + "phase": "repeated", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.817694466008106, + "decode_tps": 32.91653813014045, + "elapsed_s": 25.273517662019003, + "chunk_offsets_s": [ + 24.817694466008106, + 24.901685290999012, + 24.976887680008076, + 25.05421033600578, + 25.135884115006775, + 25.211753372015664, + 25.27339244799805 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.5959565032972, + "p95": 83.99082499090582, + "max": 83.99082499090582 + }, + "equal_first_tokens": true, + "equal_first_output": true + }, + { + "phase": "concurrent", + "prompt_tokens": 32768, + "cached_tokens": 0, + "tokens": 16, + "cache_state": "cold", + "complete": true, + "token_sha": "04e3ca755337", + "token_sha256": "04e3ca755337ba940cd8a065c4d9dcda8c8c56cd58fa834fe243660cd5c80f41", + "output_sha256": "5905bbab6d1aa9083539f216b22c2baccd2004d74336183cd131799d93068bc9", + "ttft_s": 24.817260901007103, + "decode_tps": 32.91791777121479, + "elapsed_s": 25.273143075988628, + "chunk_offsets_s": [ + 24.817260901007103, + 24.902051084995037, + 24.97617565401015, + 25.053082369006006, + 25.134610974986572, + 25.210526587005006, + 25.272939783986658 + ], + "chunk_gaps_ms": { + "count": 6, + "p50": 76.41116350714583, + "p95": 84.79018398793414, + "max": 84.79018398793414 + }, + "equal_first_tokens": true, + "equal_first_output": true + } + ], + "concurrent_timing": { + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 25.273143075988628, + "end_to_end_tps": 0.633083109286917, + "timing_valid": true + }, + "qualified": true +} +{"length": 1024, "rep": 0, "ttft": 0.8870615519990679, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.4525940149906091, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.4501131739816628, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.4417523459997028, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 6.2712317899859045, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 6.260038411011919, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 6.274133027007338, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 12.437742424983298, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 12.435033277986804, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 12.46316197401029, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 24.861044505989412, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 24.871979279996594, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 24.859724947978975, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 50.28398825702607, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 50.19671253100387, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 50.23616275101085, "prompt": 65536, "cached": 0} +=== kvp-client r3r9b rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/cold.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/cold.json new file mode 100644 index 0000000000..22d4c8dd53 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.8870615519990679, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.4525940149906091, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.4501131739816628, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.4417523459997028, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 6.2712317899859045, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 6.260038411011919, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 6.274133027007338, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 12.437742424983298, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 12.435033277986804, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 12.46316197401029, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 24.861044505989412, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 24.871979279996594, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 24.859724947978975, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 50.28398825702607, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 50.19671253100387, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 50.23616275101085, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/decode.json b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/decode.json new file mode 100644 index 0000000000..a41b1af155 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/decode.json @@ -0,0 +1,174 @@ +[ + { + "label": "r3r9b", + "prompt": "fibonacci-raw", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 35.030135525127704, + "decode_tps_all": [ + 39.38, + 29.99, + 35.03 + ], + "ttft_s_median": 0.1463688249932602, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "950590a0cb84", + "output_sha256": "c8b307fa0becb67c39cbc61f6f8b51c31e2b6443a00c7e9da3fffacb9eace78b" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b94a7bd1b3ff", + "output_sha256": "057a28544e2fc7a06bb968e33dbbc69ccdeb08c0aea2cfb7c5242bfc565741ff" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "6642e40d33c3", + "output_sha256": "9d0023c5324bb8e6900942c44c0a37965e9fc2d645b9518524537872f67501d9" + } + ], + "sample": " Also, write a function that returns the first n Fibonacci numbers.\n\n```python\ndef fibonacci(n):\n \"\"\"Compute the nth Fibonacci number.\n\n The Fibonacci seq" + }, + { + "label": "r3r9b", + "prompt": "gpu-chat-no-think", + "temperature": 1.0, + "tokens": 256, + "decode_tps_median": 28.523070512917148, + "decode_tps_all": [ + 29.28, + 28.41, + 28.52 + ], + "ttft_s_median": 0.21423337899614125, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "9842a795db37", + "output_sha256": "c011c3d752c75b46f47a102d07b0f71f7597c16a461092666c264bec03bb652b" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "8f747a2ab066", + "output_sha256": "1764ae38fb3853bf0f0d7b3e45fd02c8ba77a94f745ef870544646bd00d8f7f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "2b7ba7cd6cfa", + "output_sha256": "ebfeac9a42128c436ff3c32932ced9fc509581937b284e8fc7d528f3a5e35752" + } + ], + "sample": "Here's how a GPU handles matrix multiplication, starting with the big picture.\n\n## The Big Picture\n\nA GPU is like a stadium full of workers, while a CPU is like" + }, + { + "label": "r3r9b", + "prompt": "fibonacci-raw", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 56.25979916245646, + "decode_tps_all": [ + 56.28, + 56.26, + 56.25 + ], + "ttft_s_median": 0.14530861302046105, + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "sample": " Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ... Each number is the sum of the two preceding ones. The sequence starts with 0 and 1. The F" + }, + { + "label": "r3r9b", + "prompt": "gpu-chat-no-think", + "temperature": 0.0, + "tokens": 256, + "decode_tps_median": 30.676259183874564, + "decode_tps_all": [ + 30.63, + 30.69, + 30.68 + ], + "ttft_s_median": 0.21456954700988717, + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0c5820aa8286", + "output_sha256": "88df3a840ebfd834afb178433102979253113dc3c306a55b5c490836521167f2" + } + ], + "sample": "# Matrix Multiplication on a GPU\n\n## The Plain English Explanation\n\nImagine you have a huge stack of homework to grade, where each problem is completely indepen" + } +] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/server-rank0.log new file mode 100644 index 0000000000..ba34a9e3a0 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/server-rank0.log @@ -0,0 +1,199 @@ +[kvp-serve] 2026-10-23T03:38:24+00:00 rank 0 of 2 context 131072 parallel 4 binary tensorfold-native-r9 0c7727776fda6e28 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-d:25424:25424 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-d:25424:25424 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-d:25424:25424 [0] NCCL INFO cudaDriverVersion 13030 +host-d:25424:25424 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-d:25424:25424 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-d:25424:25424 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-d:25424:25424 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-d:25424:25424 [0] NCCL INFO Count not open libibverbs.so +host-d:25424:25424 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-d:25424:25424 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-d:25424:25424 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f1np1 +host-d:25424:25424 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f1 +host-d:25424:25424 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x78009d0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-d:25424:25424 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-d:25424:25424 [0] NCCL INFO NET/IB : Using [0]rocep1s0f1:1/RoCE [RO]; OOB enp1s0f1np1:<0> +host-d:25424:25424 [0] NCCL INFO Initialized NET plugin IB +host-d:25424:25424 [0] NCCL INFO Assigned NET plugin IB to comm +host-d:25424:25424 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-d:25424:25424 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-d:25424:25424 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-d:25424:25424 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-d:25424:25424 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-d:25424:25424 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-d:25424:25424 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-d:25424:25424 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-d:25424:25424 [0] NCCL INFO Using network IB +host-d:25424:25424 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-d:25424:25424 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x4bce970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-d:25424:25424 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-d:25424:25424 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:25424:25424 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:25424:25424 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f1' +host-d:25424:25424 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-d:25424:25424 [0] NCCL INFO Rank 0: 1 Net devices +host-d:25424:25424 [0] NCCL INFO Rank 0: 0 CollNet devices +host-d:25424:25424 [0] NCCL INFO Local Net device counts across ranks: min 1 max 1 +host-d:25424:25424 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-d:25424:25424 [0] NCCL INFO comm 0x4bce970 rank 0 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-d:25424:25424 [0] NCCL INFO Channel 00/04 : 0 1 +host-d:25424:25424 [0] NCCL INFO Channel 01/04 : 0 1 +host-d:25424:25424 [0] NCCL INFO Channel 02/04 : 0 1 +host-d:25424:25424 [0] NCCL INFO Channel 03/04 : 0 1 +host-d:25424:25424 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 +host-d:25424:25424 [0] NCCL INFO P2P Chunksize set to 262144 +host-d:25424:25424 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-d:25424:25424 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-d:25424:25424 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-d:25424:25424 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-d:25424:25444 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-d:25424:25443 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-d:25424:25443 [0] NCCL INFO proxy listening socket at <45095> +host-d:25424:25424 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:25424:25424 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:25424:25445 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-d:25424:25424 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:25424:25424 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/0 +host-d:25424:25424 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:25424:25424 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:25424:25424 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:25424:25424 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbe8d0010000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbe8d0054000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbe8d009c000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbe8d00e0000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2038 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2038 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1badaa ctsFifoLkey=0x1badaa +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2038 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2039 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2040 pkey=0 pd=0xfbe8d0053b70 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2041 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2041 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b7e7c ctsFifoLkey=0x1b7e7c +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2041 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2042 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2043 pkey=0 pd=0xfbe8d0053b70 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbe8d0237000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2038 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2041 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2044 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2044 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1ac6c3 ctsFifoLkey=0x1ac6c3 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2044 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbe8d02a9000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbe8d02ee000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbe8d032f000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2044 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2045 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2046 pkey=0 pd=0xfbe8d0053b70 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2047 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2047 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1b9e9b ctsFifoLkey=0x1b9e9b +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2047 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2048 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2049 pkey=0 pd=0xfbe8d0053b70 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2047 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25424 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-d:25424:25424 [0] NCCL INFO Connected all trees +host-d:25424:25424 [0] NCCL INFO Connected binomial trees +host-d:25424:25424 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-d:25424:25424 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-d:25424:25424 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-d:25424:25424 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 94.66 GiB; 4 streams at once, 1.77 GiB each at a 131072-token window, of 7.8 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 131072; loaded in 308.7s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-299edc91d8e1a48a48b02accd80fb211 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.146 decode=6.475 elapsed=6.621 queued=0.000 tok/s=39.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-d3d4a40b2c3832a523808e6383aa2d01 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=950590a0cb84 finish=length rounds=79 accepted=177/226 ttft=0.145 decode=6.476 elapsed=6.621 queued=0.000 tok/s=39.4 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-1d530a0f4c537896b03c653493dbf0e4 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b94a7bd1b3ff finish=length rounds=123 accepted=132/193 ttft=0.144 decode=8.504 elapsed=8.649 queued=0.000 tok/s=30.0 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-07f1ed7d3f178db492b3a90811a8dea8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=6642e40d33c3 finish=length rounds=96 accepted=159/223 ttft=0.145 decode=7.280 elapsed=7.425 queued=0.000 tok/s=35.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6cb53d7811fabd48d31cdf34a433d106 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.212 decode=8.711 elapsed=8.923 queued=0.000 tok/s=29.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c39c62955b9812350a29726bacbd59a5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=9842a795db37 finish=length rounds=124 accepted=132/199 ttft=0.213 decode=8.708 elapsed=8.922 queued=0.000 tok/s=29.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-744a1373b5653f1883ee3ebed227ffe3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=8f747a2ab066 finish=length rounds=131 accepted=124/188 ttft=0.212 decode=8.977 elapsed=9.190 queued=0.000 tok/s=28.4 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-193fb440ea3772fa30d6859661a5d4bd prompt=32 cached=0 thinking=False effort=none tokens=256 sha=2b7ba7cd6cfa finish=length rounds=131 accepted=124/187 ttft=0.213 decode=8.940 elapsed=9.153 queued=0.000 tok/s=28.5 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4ddfa8fd298cb4ff4e33207494f4ed98 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.531 elapsed=4.676 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-663b85c92373cf46c0e6c3492603a40f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.145 decode=4.531 elapsed=4.676 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-2516fe61709755a8d32bc9e209cd1164 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.533 elapsed=4.677 queued=0.000 tok/s=56.3 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-214c00d9845385ae3cca14dc731bdb96 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=50 accepted=205/219 ttft=0.144 decode=4.534 elapsed=4.677 queued=0.000 tok/s=56.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-8fd6344ba048efdeb90cac14562cf145 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.315 elapsed=8.529 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9c08445978eff00264e4b7b42f8d6f5c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.324 elapsed=8.537 queued=0.000 tok/s=30.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-319a16032200f329d1ad8d97e2cbb3a3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.214 decode=8.309 elapsed=8.524 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-cd377cc7365b96c8a2ba6c0f06099752 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0c5820aa8286 finish=length rounds=116 accepted=139/205 ttft=0.213 decode=8.313 elapsed=8.526 queued=0.000 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-a7edd64a8cf332e1f4626c474f37bfd5 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.838 decode=0.456 elapsed=25.294 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-dd976ae8ec38fbdbf897443219e9feba prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.815 decode=0.456 elapsed=25.271 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-7626782266afae1adc0a669b6c3a5942 prompt=32768 cached=0 thinking=False effort=none tokens=16 sha=04e3ca755337 finish=length rounds=6 accepted=9/13 ttft=24.815 decode=0.456 elapsed=25.270 queued=0.000 tok/s=32.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-40857d9eec8431e1efcaaf8a863aaeb7 prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.873 decode=0.063 elapsed=0.935 queued=0.000 tok/s=16.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b34bc038a2902aa33d7046936d647b91 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.451 decode=0.064 elapsed=1.515 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-de638aec7f8a3eebf87e706e10632117 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.449 decode=0.063 elapsed=1.512 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-196ba951f9c65c725a5b8ebe6185002b prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.441 decode=0.063 elapsed=1.504 queued=0.000 tok/s=15.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-8fcd02a742e0443bc5e68fab5dcdcc97 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.270 decode=0.064 elapsed=6.334 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-db0eb97603371aed4bdfc3518e396f78 prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=6.259 decode=0.065 elapsed=6.323 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-27805087925cf8415c7d853319fb5bfb prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=6.273 decode=0.064 elapsed=6.337 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f4fc2353ad1491aa345177cb66470b2d prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=12.436 decode=0.064 elapsed=12.500 queued=0.000 tok/s=15.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4470df72963388c8363ff641b15f603e prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.434 decode=0.065 elapsed=12.498 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f132297685c2b1fd1a9e16df60faa8e1 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=12.462 decode=0.065 elapsed=12.527 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-45a539c36afaf421bbfaa3a70d6dd389 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.860 decode=0.066 elapsed=24.926 queued=0.000 tok/s=15.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-61ca17740d3830f3e73b312a44ea90a7 prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=24.871 decode=0.065 elapsed=24.935 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9be0561b81a50a2683dea52933fab7fd prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=24.858 decode=0.065 elapsed=24.923 queued=0.000 tok/s=15.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9e7796f9d8fa0beb438c1464d1562d43 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.282 decode=0.065 elapsed=50.347 queued=0.000 tok/s=15.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7716a38e14094f1225e1f2ef9969b85a prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=50.195 decode=0.066 elapsed=50.261 queued=0.000 tok/s=15.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c03d15b0b0584f77a9c929e19a617724 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=50.234 decode=0.065 elapsed=50.299 queued=0.000 tok/s=15.3 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +channels per peer +host-d:25424:25424 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-d:25424:25424 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-d:25424:25424 [0] NCCL INFO ncclCommInitRank comm 0x4bce970 rank 0 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-d:25424:25424 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 2 total 0.45 (kernels 0.26, alloc 0.04, bootstrap 0.04, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-d:25424:25424 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/0/Shared +host-d:25424:25424 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/0/Shared +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbe8d0444000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbe8d0485000) +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2050 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 2050 mtu 3 GID 3 (0/12D290AFFFF0000) ctsFifoRkey=0x1a4240 ctsFifoLkey=0x1a4240 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 2050 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2051 pkey=0 pd=0xfbe8d0053b70 oooRq=0 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f1 ndevs=1 nmdevs=1 qp_num=2052 pkey=0 pd=0xfbe8d0053b70 +host-d:25424:25443 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 2050 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-d:25424:25424 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/server-rank1.log new file mode 100644 index 0000000000..ce65aa3793 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp2/speed/r3r9b/server-rank1.log @@ -0,0 +1,116 @@ +[kvp-serve] 2026-10-23T03:38:22+00:00 rank 1 of 2 context 131072 parallel 4 binary tensorfold-native-r9 0c7727776fda6e28 env TF_KV_PAGES=unset TF_GLM_MM=unset args --prompt-cache-gib 0 +host-e:2433:2433 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-e:2433:2433 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-e:2433:2433 [0] NCCL INFO cudaDriverVersion 13030 +host-e:2433:2433 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-e:2433:2433 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-e:2433:2433 [0] NCCL INFO ncclOsDlopen(libnccl-net.so) failed: libnccl-net.so: cannot open shared object file: No such file or directory +host-e:2433:2433 [0] NCCL INFO NET/Plugin: Could not find: libnccl-net.so +host-e:2433:2433 [0] NCCL INFO Count not open libibverbs.so +host-e:2433:2433 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-e:2433:2433 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-e:2433:2433 [0] NCCL INFO NCCL_SOCKET_IFNAME set to enp1s0f0np0 +host-e:2433:2433 [0] NCCL INFO NCCL_IB_HCA set to rocep1s0f0 +host-e:2433:2433 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x2786e7a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-e:2433:2433 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-e:2433:2433 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [RO]; OOB enp1s0f0np0:<0> +host-e:2433:2433 [0] NCCL INFO Initialized NET plugin IB +host-e:2433:2433 [0] NCCL INFO Assigned NET plugin IB to comm +host-e:2433:2433 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-e:2433:2433 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-e:2433:2433 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-e:2433:2433 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-e:2433:2433 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-e:2433:2433 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-e:2433:2433 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-e:2433:2433 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-e:2433:2433 [0] NCCL INFO Using network IB +host-e:2433:2433 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-e:2433:2433 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x24c3c970 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init START +host-e:2433:2433 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-e:2433:2433 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2433:2433 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2433:2433 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-e:2433:2433 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-e:2433:2433 [0] NCCL INFO Rank 1: 1 Net devices +host-e:2433:2433 [0] NCCL INFO Rank 1: 0 CollNet devices +host-e:2433:2433 [0] NCCL INFO comm 0x24c3c970 rank 1 nRanks 2 nNodes 2 localRanks 1 localRank 0 MNNVL 0 +host-e:2433:2433 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] -1/-1/-1->1->0 [2] -1/-1/-1->1->0 [3] -1/-1/-1->1->0 +host-e:2433:2433 [0] NCCL INFO P2P Chunksize set to 262144 +host-e:2433:2433 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 2 +host-e:2433:2433 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-e:2433:2433 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-e:2433:2433 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-e:2433:2446 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-e:2433:2445 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-e:2433:2445 [0] NCCL INFO proxy listening socket at <53843> +host-e:2433:2447 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-e:2433:2433 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2433:2433 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2433:2433 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2433:2433 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/0 +host-e:2433:2433 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2433:2433 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2433:2433 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2433:2433 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb7dac010000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb7dac054000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb7dac09c000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb7dac0e0000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1782 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1782 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x180fcc ctsFifoLkey=0x180fcc +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1782 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1783 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1783 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17ca87 ctsFifoLkey=0x17ca87 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1783 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb7dac196000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb7dac1da000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb7dac222000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb7dac266000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1784 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1785 pkey=0 pd=0xfb7dac053b70 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1782 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1786 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1787 pkey=0 pd=0xfb7dac053b70 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1783 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1788 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1789 pkey=0 pd=0xfb7dac053b70 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1790 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1790 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x17b06d ctsFifoLkey=0x17b06d +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1790 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1791 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1791 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x174502 ctsFifoLkey=0x174502 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1791 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1791 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1790 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1792 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1793 pkey=0 pd=0xfb7dac053b70 +host-e:2433:2433 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-e:2433:2433 [0] NCCL INFO Connected all trees +host-e:2433:2433 [0] NCCL INFO Connected binomial trees +host-e:2433:2433 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-e:2433:2433 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-e:2433:2433 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 512 | 512 +host-e:2433:2433 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 1 p2p channels per peer +host-e:2433:2433 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-e:2433:2433 [0] NCCL INFO ncclCommInitRank comm 0x24c3c970 rank 1 nranks 2 cudaDev 0 nvmlDev 0 busId f01000 commId 0x550bed879af798cb - Init COMPLETE +host-e:2433:2433 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 2 total 0.45 (kernels 0.28, alloc 0.05, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.11, rest 0.00) +host-e:2433:2433 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/0/Shareinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: RoCE all-gather: 1 HCA(s), shards up to 4096 KiB by RDMA writes +info: rank 1: rank 0 ended; leaving +d +host-e:2433:2433 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/0/Shared +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfb7dac441000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfb7dac482000) +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1794 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1794 mtu 3 GID 3 (0/22D290AFFFF0000) ctsFifoRkey=0x171cdc ctsFifoLkey=0x171cdc +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1794 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1795 pkey=0 pd=0xfb7dac053b70 oooRq=0 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=1 nmdevs=1 qp_num=1796 pkey=0 pd=0xfb7dac053b70 +host-e:2433:2445 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1794 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-e:2433:2433 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/build.log b/docs/receipts/glm-kv-pages-8e705835/tp3/build.log new file mode 100644 index 0000000000..6b87795634 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/build.log @@ -0,0 +1,11 @@ +-- run 2026-10-23 09:22:57 +2026-10-23 09:22:58 tree 10698886 glm: TF_GLM_NCCL_ID_FILE hands rank 0's NCCL id to the others in a file, and NCCL; changed: zig/glm_prompt_host.zig zig/glm_snapshot_host.zig zig/src/core/contract/glm_cuda_records.zig zig/src/core/contract/registry.zig zig/src/core/heat.zig zig/src/core/kv_pool.zig zig/src/core/kv_pool_test.zig zig/src/families/glm/cuda.zig zig/src/families/glm/cuda_dsa_block.zig zig/src/families/glm/cuda_engine.zig zig/src/families/glm/cuda_follow.zig zig/src/families/glm/cuda_kv8_test.zig zig/src/families/glm/cuda_lanes.zig zig/src/families/glm/cuda_latent.zig zig/src/families/glm/cuda_native.zig zig/src/families/glm/cuda_pages.zig zig/src/families/glm/cuda_pages_test.zig zig/src/families/glm/cuda_prompts.zig zig/src/families/glm/cuda_ranks.zig zig/src/families/glm/cuda_snapshot_layout.zig zig/src/families/glm/cuda_snapshot_pages.zig zig/src/families/glm/cuda_snapshots.zig zig/src/families/glm/cuda_sparse.zig zig/src/families/glm/cuda_state.zig zig/src/native/cuda.zig zig/src/native/cuda_kv_pool.zig zig/src/native/cuda_kv_safety_test.zig zig/tests/cuda/contract_glm_attn.zig zig/tests/cuda/contract_glm_index.zig zig/tests/cuda/contract_glm_pages.zig zig/tests/cuda/contract_tests.zig zig/tests/cuda/glm_paged_tests.zig zig/tests/cuda/glm_ref_attn.zig zig/tests/cuda/glm_ref_fp8.zig zig/tests/cuda/glm_ref_index.zig zig/tests/cuda/main.zig +2026-10-23 09:22:58 fmt rc=0 +2026-10-23 09:22:58 -- tp3: 10698886 glm: TF_GLM_NCCL_ID_FILE hands rank 0's NCCL id to the others in a file, and NCCL +2026-10-23 09:27:15 build rc=0 257s: +2026-10-23 09:29:50 native rc=0 155s: +2026-10-23 09:30:10 host rc=0 20s: [default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays failed command: ./.zig-cache/o/2cd8a4d37c1f86a338f34338bdf017f8/test --cache-dir=./.zig-cache --seed=0x6b6f0efa --listen=- Build Summary: 155/155 steps succeeded; 1097/1101 tests passed (4 skipped) +2026-10-23 09:30:15 kvpool rc=0 5s: Build Summary: 5/5 steps succeeded; 17/17 tests passed +2026-10-23 09:30:38 test rc=0 23s: failed command: ./.zig-cache/o/73032413fa2cbd32f6d0043fbdc810a6/test --cache-dir=./.zig-cache --seed=0x139a65b --listen=- Build Summary: 276/276 steps succeeded; 857/863 tests passed (6 skipped) | | +- compile lib hip-mock-failed debug native success 52ms MaxRSS:63M +2026-10-23 09:30:39 lean rc=1 1s: +2026-10-23 09:30:39 === kvp-base tp3 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/chain.log b/docs/receipts/glm-kv-pages-8e705835/tp3/chain.log new file mode 100644 index 0000000000..e95d8fddda --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/chain.log @@ -0,0 +1,12 @@ +2026-10-23 10:18:38 -- R6 chain on kvp-tp3-r9 from check; rank.env without TF_GLM_MM (1.1.2's default) +2026-10-23 10:18:52 binaries from 04's tp3: tensorfold-native eed66a8b83e10eb3, tf-cuda-test 6530f1e42385bf43 +2026-10-23 10:19:19 setup rc 0: up on host-a: /hub is /.cache/huggingface/h up on host-b: /hub is /.cache/huggingface/h up on host-c: /hub is /.cache/huggingface/h +2026-10-23 10:20:01 check: === kvp3-check pg3 rc 0 0 0 +2026-10-23 10:46:02 auto: === kvp3-phase r6auto rc 0 +2026-10-23 11:02:13 all: === kvp3-phase r6all rc 0 +2026-10-23 11:21:03 off: === kvp3-phase r6off rc 0 +2026-10-23 11:21:06 off's replay against auto's record (logs/r6-compare-off.log): rc=0 B t4 drafted prompt 19808 cached 16384: ttft 4.97 s vs fresh 26.47 s ( 5.3x) PASS: 43/43 replies equal and as cached as planned timing: 0 resumed replies no sooner than fresh +2026-10-23 11:24:11 admit: a 37.2 GiB prompt cache (the all step's room less its streams' fixed parts, and 0.9 GiB of pages but on R9's floor) +2026-10-23 11:31:18 admit: === kvp3-phase r6admit rc 0 +2026-10-23 11:31:55 stopped on 01-03; GPU processes there: 0 +== R6 finished diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/check/glm-paged-check-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp3/check/glm-paged-check-rank0.log new file mode 100644 index 0000000000..fcf8e98e7d --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/check/glm-paged-check-rank0.log @@ -0,0 +1,9798 @@ +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:85:85 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:85:85 [0] NCCL INFO cudaDriverVersion 13030 +host-a:85:85 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:85:85 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:85:85 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:85:85 [0] NCCL INFO Count not open libibverbs.so +host-a:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:85:85 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:85:85 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x467239b0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:85:85 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x467650b0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:85:85 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x467a67b0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:85:85 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x467e7eb0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:85:85 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:85:85 [0] NCCL INFO Initialized NET plugin IB +host-a:85:85 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:85:85 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:85:85 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:85:85 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:85:85 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:85:85 [0] NCCL INFO Using network IB +host-a:85:85 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:85:85 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x43aec310 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x1d746c4586df2bd5 - Init START +host-a:85:85 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:85:85 [0] NCCL INFO NET/GIN_IB_GDAinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +KI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:85:85 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:85:85 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:85:85 [0] NCCL INFO Rank 0: 6 Net devices +host-a:85:85 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:85:85 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:85:85 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:85:85 [0] NCCL INFO comm 0x43aec310 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:85:85 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:85:85 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:85:85 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:85:85 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:85:85 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:85:85 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:85:85 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:85:85 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:85:95 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-a:85:95 [0] NCCL INFO proxy listening socket at <50849> +host-a:85:96 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-a:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:85:85 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:85:85 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:85:85 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:85:85 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 1, globalGinSupport 0, cuMemGdrSupport 0 +host-a:85:85 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:85:85 [0] NCCL INFO ncclCommInitRank comm 0x43aec310 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x1d746c4586df2bd5 - Init COMPLETE +host-a:85:85 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 2.98 (kernels 0.24, alloc 0.03, bootstrap 2.70, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.00, rest 0.00) +host-a:85:85 [0] NCCL INFO Symmetric VA size=124GB +host-a:85:105 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-a:85:85 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:85:85 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:85:85 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:85:85 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c090010000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c090054000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c09009c000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c0900e0000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6897 pkey=0 pd=0xf6c09000fef0 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6898 pkey=0 pd=0xf6c09000fef0 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6897 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b6866 ctsFifoLkey=0x1b6866 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6897 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6898 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b6866 ctsFifoLkey=0x1b6866 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6898 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c09018d000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c0901d1000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c090219000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c09025d000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1997 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1998 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1999 pkey=0 pd=0xf6c09016bc70 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1869 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1870 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1871 pkey=0 pd=0xf6c0902bf930 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6929 pkey=0 pd=0xf6c090343c80 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6930 pkey=0 pd=0xf6c090343c80 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6929 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ce7e3 ctsFifoLkey=0x1ce7e3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6929 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6930 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ce7e3 ctsFifoLkey=0x1ce7e3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6930 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2000 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2001 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2002 pkey=0 pd=0xf6c09016bc70 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6897 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6898 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1872 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1873 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1874 pkey=0 pd=0xf6c0902bf930 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6931 pkey=0 pd=0xf6c090343c80 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6932 pkey=0 pd=0xf6c090343c80 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6931 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6931 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6932 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6932 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6899 pkey=0 pd=0xf6c09000fef0 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6900 pkey=0 pd=0xf6c09000fef0 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6899 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6899 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6900 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6900 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6929 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6930 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6931 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6932 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6899 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6900 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:85 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:85:85 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:85:85 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:85:85 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:85:85 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c0905b9000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c0905fa000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6901 pkey=0 pd=0xf6c09000fef0 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6902 pkey=0 pd=0xf6c09000fef0 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6901 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1afaf7 ctsFifoLkey=0x1afaf7 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6901 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6902 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1afaf7 ctsFifoLkey=0x1afaf7 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6902 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6933 pkey=0 pd=0xf6c090343c80 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6934 pkey=0 pd=0xf6c090343c80 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6933 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1dc3c4 ctsFifoLkey=0x1dc3c4 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6933 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6934 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1dc3c4 ctsFifoLkey=0x1dc3c4 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6934 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c0906e9000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c090731000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6901 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6902 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6933 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6934 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6935 pkey=0 pd=0xf6c090343c80 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6936 pkey=0 pd=0xf6c090343c80 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6937 pkey=0 pd=0xf6c090343c80 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6903 pkey=0 pd=0xf6c09000fef0 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6904 pkey=0 pd=0xf6c09000fef0 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6905 pkey=0 pd=0xf6c09000fef0 +host-a:85:85 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:85:85 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:85:85 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:85:85 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c090865000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf6c0908aa000) +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c0908ef000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c090933000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1875 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1876 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1877 pkey=0 pd=0xf6c0902bf930 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2003 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2004 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2005 pkey=0 pd=0xf6c09016bc70 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2006 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2007 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2006 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2006 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2007 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2007 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1878 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1879 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1878 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1878 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1879 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1879 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2006 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2007 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1878 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1879 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:85 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:85:85 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:85:85 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c090b1f000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c090b60000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c090ba1000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1880 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1881 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1880 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1880 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1881 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1881 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf6c090c3b000) +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1880 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1881 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2008 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2009 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2008 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2008 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2009 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2009 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1882 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1883 pkey=0 pd=0xf6c0902bf930 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1882 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176f2c ctsFifoLkey=0x176f2c +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1882 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1883 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176f2c ctsFifoLkey=0x176f2c +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1883 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2010 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2011 pkey=0 pd=0xf6c09016bc70 oooRq=0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2010 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a6a8 ctsFifoLkey=0x19a6a8 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2010 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2011 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a6a8 ctsFifoLkey=0x19a6a8 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2011 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCLinfo: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +info: kv pool idle: promised 0, free 37 + store 0 == total 37 +PASS pages at TP=3: on the engine's own sequence, 7 prompts at the pages' edges (511 to 5,000 tokens) and 91 window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order (12 jumps) and NaN before use; the pool idle after + INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2008 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2009 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1882 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1883 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2010 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:95 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2011 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:85:85 [0] NCCL INFO Connected all trees diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/check/glm-paged-check-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp3/check/glm-paged-check-rank1.log new file mode 100644 index 0000000000..c4647568e1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/check/glm-paged-check-rank1.log @@ -0,0 +1,6299 @@ +host-b:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:85:85 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:85:85 [0] NCCL INFO cudaDriverVersion 13030 +host-b:85:85 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:85:85 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:85:85 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:85:85 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:85:85 [0] NCCL INFO Count not open libibverbs.so +host-b:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:85:85 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:85:85 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0xd5148a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:85:85 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0xd555fa0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:85:85 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0xd5976a0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:85:85 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0xd5d8da0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:85:85 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:85:85 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:85:85 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:85:85 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:85:85 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:85:85 [0] NCCL INFO Initialized NET plugin IB +host-b:85:85 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:85:85 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:85:85 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:85:85 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:85:85 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:85:85 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:85:85 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:85:85 [0] NCCL INFO Using network IB +host-b:85:85 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:85:85 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0xa8dd210 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x1d746c4586df2bd5 - Init START +host-b:85:85 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:85:85 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:85:85 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:85:85 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:85:85 [0] NCCL INFO Rank 1: 6 Net devices +host-b:85:85 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:85:85 [0] NCCL INFO comm 0xa8dd210 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:85:85 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:85:85 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:85:85 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:85:85 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:85:85 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:85:94 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-b:85:94 [0] NCCL INFO proxy listening socket at <35159> +host-b:85:95 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 4 +host-b:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:85:85 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:85:85 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:85:85 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:85:85 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 1, globalGinSupport 0, cuMemGdrSupport 0 +host-b:85:85 [0] NCCL INFO ncclCommInitRank comm 0xa8dd210 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x1d746c4586df2bd5 - Init COMPLETE +host-b:85:85 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 0.90 (kernels 0.24, alloc 0.04, bootstrap 0.60, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.00, rest 0.00) +host-b:85:85 [0] NCCL INFO Symmetric VA size=124GB +host-b:85:104 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-b:85:85 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:85:85 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:85:85 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:85:85 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:85:85 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:85:85 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:85:85 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:85:85 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea2580010000) +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea2580054000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea258009c000) +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2125 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2126 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2125 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2125 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2126 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1be3e1 ctsFifoLkey=0x1be3e1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2126 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6673 pkey=0 pd=0xea2580053e50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6674 pkey=0 pd=0xea2580053e50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6675 pkey=0 pd=0xea2580053e50 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2253 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2254 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2253 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2253 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2254 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d9e96 ctsFifoLkey=0x1d9e96 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2254 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea258022e000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea2580276000) +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea25802ba000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea2580302000) +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea2580346000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6705 pkey=0 pd=0xea2580191dd0 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6706 pkey=0 pd=0xea2580191dd0 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6707 pkey=0 pd=0xea2580191dd0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2125 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2126 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2253 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2254 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6708 pkey=0 pd=0xea2580191dd0 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6709 pkey=0 pd=0xea2580191dd0 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6710 pkey=0 pd=0xea2580191dd0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2127 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2128 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2127 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8c8b ctsFifoLkey=0x1b8c8b +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2127 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2128 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8c8b ctsFifoLkey=0x1b8c8b +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2128 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6676 pkey=0 pd=0xea2580053e50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6677 pkey=0 pd=0xea2580053e50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6678 pkey=0 pd=0xea2580053e50 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2255 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2256 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2255 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1ddcd3 ctsFifoLkey=0x1ddcd3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2255 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2256 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1ddcd3 ctsFifoLkey=0x1ddcd3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2256 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2127 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2128 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2255 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2256 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:85 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:85:85 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:85:85 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:85:85 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:85:85 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea25805b3000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea25805f8000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea258063d000) +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea258067e000) +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6679 pkey=0 pd=0xea2580053e50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6680 pkey=0 pd=0xea2580053e50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6681 pkey=0 pd=0xea2580053e50 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6711 pkey=0 pd=0xea2580191dd0 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6712 pkey=0 pd=0xea2580191dd0 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6713 pkey=0 pd=0xea2580191dd0 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6714 pkey=0 pd=0xea2580191dd0 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6715 pkey=0 pd=0xea2580191dd0 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6714 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c7c4 ctsFifoLkey=0x19c7c4 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6714 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6715 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c7c4 ctsFifoLkey=0x19c7c4 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6715 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6682 pkey=0 pd=0xea2580053e50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6683 pkey=0 pd=0xea2580053e50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6682 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17ca87 ctsFifoLkey=0x17ca87 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6682 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6683 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17ca87 ctsFifoLkey=0x17ca87 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6683 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6714 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6715 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6682 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6683 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:85 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:85:85 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:85:85 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:85:85 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea2580869000) +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xea25808aa000) +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2129 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2130 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2129 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bd6d3 ctsFifoLkey=0x1bd6d3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2129 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2130 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bd6d3 ctsFifoLkey=0x1bd6d3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2130 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2257 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2258 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2257 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1ddad7 ctsFifoLkey=0x1ddad7 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2257 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2258 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1ddad7 ctsFifoLkey=0x1ddad7 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2258 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea258099d000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea25809e5000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2129 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2130 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2257 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2258 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2259 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2260 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2261 pkey=0 pd=0xea2580123d60 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2131 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2132 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2133 pkey=0 pd=0xea258000fe50 +host-b:85:85 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:85:85 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:85:85 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:85:85 [0] NCPASS rank 1 made rank 0's pool and mapped its pages through every step +CL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea2580b19000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea2580b5e000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea2580ba3000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xea2580beb000) +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2134 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2135 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2136 pkey=0 pd=0xea258000fe50 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2262 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2263 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2264 pkey=0 pd=0xea2580123d60 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2137 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2138 pkey=0 pd=0xea258000fe50 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2139 pkey=0 pd=0xea258000fe50 +host-b:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2265 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2266 pkey=0 pd=0xea2580123d60 oooRq=0 +host-b:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2267 pkey=0 pd=0xea2580123d60 +host-b:85:85 [0] NCCL INFO Connected all trees diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/check/glm-paged-check-rank2.log b/docs/receipts/glm-kv-pages-8e705835/tp3/check/glm-paged-check-rank2.log new file mode 100644 index 0000000000..04578ec069 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/check/glm-paged-check-rank2.log @@ -0,0 +1,6519 @@ +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:85:85 [0] NCCL INFO cudaDriverVersion 13030 +host-c:85:85 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:85:85 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:85:85 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:85:85 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:85:85 [0] NCCL INFO Count not open libibverbs.so +host-c:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:85:85 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:85:85 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:85:85 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0xeec28e0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:85:85 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0xef03fe0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:85:85 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0xef456e0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:85:85 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0xef86de0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:85:85 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:85:85 [0] NCCL INFO Initialized NET plugin IB +host-c:85:85 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:85:85 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:85:85 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:85:85 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:85:85 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:85:85 [0] NCCL INFO Using network IB +host-c:85:85 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:85:85 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0xc28b110 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x1d746c4586df2bd5 - Init START +host-c:85:85 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:85:85 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:85:85 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:85:85 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:85:85 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:85:85 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:85:85 [0] NCCL INFO Rank 2: 6 Net devices +host-c:85:85 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:85:85 [0] NCCL INFO comm 0xc28b110 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:85:85 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:85:85 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:85:85 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:85:85 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:85:95 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-c:85:94 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-c:85:94 [0] NCCL INFO proxy listening socket at <45517> +host-c:85:85 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:85:85 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:85:85 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:85:85 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:85:85 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 1, globalGinSupport 0, cuMemGdrSupport 0 +host-c:85:85 [0] NCCL INFO ncclCommInitRank comm 0xc28b110 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x1d746c4586df2bd5 - Init COMPLETE +host-c:85:85 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.31 (kernels 0.24, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.01, rest 0.00) +host-c:85:85 [0] NCCL INFO Symmetric VA size=124GB +host-c:85:104 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-c:85:85 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99608010000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99608054000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe9960809c000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe996080e0000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99608128000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe9960816c000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe996081b4000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe996081f8000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2253 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2254 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2253 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d1c1b ctsFifoLkey=0x1d1c1b +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2253 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2254 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d1c1b ctsFifoLkey=0x1d1c1b +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2254 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2125 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2126 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2125 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bb4b1 ctsFifoLkey=0x1bb4b1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2125 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2126 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bb4b1 ctsFifoLkey=0x1bb4b1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2126 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2255 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2256 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2255 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d4d4d ctsFifoLkey=0x1d4d4d +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2255 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2256 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d4d4d ctsFifoLkey=0x1d4d4d +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2256 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2127 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2128 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2127 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bc2bf ctsFifoLkey=0x1bc2bf +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2127 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2128 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bc2bf ctsFifoLkey=0x1bc2bf +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2128 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1869 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1870 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1871 pkey=0 pd=0xe9960831a0d0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2253 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2254 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1997 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1998 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=1999 pkey=0 pd=0xe9960840cdf0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2125 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2126 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2255 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2256 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2127 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2128 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1872 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1873 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1874 pkey=0 pd=0xe9960831a0d0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2000 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2001 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2002 pkey=0 pd=0xe9960840cdf0 +host-c:85:85 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:85:85 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:85:85 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:85:85 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:85:85 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe996085ac000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe996085ed000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2129 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2130 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2129 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bedea ctsFifoLkey=0x1bedea +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2129 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2130 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bedea ctsFifoLkey=0x1bedea +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2130 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2257 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2258 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2257 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5e5e ctsFifoLkey=0x1d5e5e +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2257 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2258 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5e5e ctsFifoLkey=0x1d5e5e +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2258 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe996086e7000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe9960872f000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2129 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2130 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2257 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2258 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2259 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2260 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2261 pkey=0 pd=0xe99608053d50 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2131 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2132 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2133 pkey=0 pd=0xe9960809be40 +host-c:85:85 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:85:85 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:85:85 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:85:85 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99608864000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe996088a9000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe996088ee000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99608932000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1875 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1876 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1877 pkey=0 pd=0xe9960831a0d0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2003 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2004 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2005 pkey=0 pd=0xe9960840cdf0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2006 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2007 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2006 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2006 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2007 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2007 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1878 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1879 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1878 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1878 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1879 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1879 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2006 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2007 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1878 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1879 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:85 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:85:85 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:85:85 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99608b20000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99608b61000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1880 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1881 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1880 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1804c2 ctsFifoLkey=0x1804c2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1880 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1881 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1804c2 ctsFifoLkey=0x1804c2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1881 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2134 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2135 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2136 pkey=0 pd=0xe9960809be40 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99608c75000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1880 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1881 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2008 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2009 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2008 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19eeeb ctsFifoLkey=0x19eeeb +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2008 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2009 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19eeeb ctsFifoLkey=0x19eeeb +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2009 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99608d10000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99608d58000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99608d9c000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe99608de4000) +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe99608e28000) +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2262 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2263 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2264 pkey=0 pd=0xe99608053d50 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1882 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1883 pkey=0 pd=0xe9960831a0d0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1882 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17a05e ctsFifoLkey=0x17a05e +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1882 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1883 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17a05e ctsFifoLkey=0x17a05e +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1883 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2137 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2138 pkey=0 pd=0xe9960809be40 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2139 pkey=0 pd=0xe9960809be40 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2010 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2011 pkey=0 pd=0xe9960840cdf0 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2010 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x199b9a ctsFifoLkey=0x199b9a +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2010 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2011 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x199b9a ctsFifoLkey=0x199b9a +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2011 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2265 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2266 pkey=0 pd=0xe99608053d50 oooRq=0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=PASS rank 2 made rank 0's pool and mapped its pages through every step +roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2267 pkey=0 pd=0xe99608053d50 +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2008 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2009 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1882 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1883 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2010 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:94 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2011 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:85:85 [0] NCCL INFO Connected all trees diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/ring.patch b/docs/receipts/glm-kv-pages-8e705835/tp3/ring.patch new file mode 100644 index 0000000000..82eee4825c --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/ring.patch @@ -0,0 +1,521 @@ +From fd15d2526c037499844cc2be10870926537670bf Mon Sep 17 00:00:00 2001 +From: +Date: Fri, 16 Oct 2026 15:35:38 +0900 +Subject: [PATCH 1/2] core/heat: with TF_HEAT_CEILING a chunk also waits while + its expected rise would pass the ceiling; GLM reads it and marks each prompt + pass's start + +--- + RUNBOOK.md | 2 + + zig/glm_prompt_host.zig | 2 +- + zig/src/core/heat.zig | 143 +++++++++++++++++++++----- + zig/src/families/glm/cuda_engine.zig | 8 +- + zig/src/families/glm/cuda_prompts.zig | 2 +- + zig/src/families/glm/cuda_ranks.zig | 29 +++++- + 6 files changed, 154 insertions(+), 32 deletions(-) + +diff --git a/RUNBOOK.md b/RUNBOOK.md +index 101faf4c3..44f6494b2 100644 +--- a/RUNBOOK.md ++++ b/RUNBOOK.md +@@ -168,6 +168,8 @@ The server logs `heat_wait_s`, a cancel ends the wait, and the reply's tokens do + On three GB10s serving GLM-5.3-Flash NVFP4 at TP=3, a 1,035,295-token prompt without the bands kept one host's hottest + zone at 94 C or more for most of the prompt (97.7 C at the top); `TF_HEAT_HIGH=92` and `TF_HEAT_LOW=88` held all three + at 93.0 C or under, and the first token came 10% later. ++For GLM, `TF_HEAT_CEILING` (above the high band, with both) also makes a chunk wait while the hottest zone plus the ++last rise of a chunk run straight after another would pass it, until that rise fits under it and the low band holds. + Archives whose names contain `host-only` are CPU verification artifacts and cannot serve CUDA inference. + + ### GB10 CPU pinning for long CUDA runs +diff --git a/zig/glm_prompt_host.zig b/zig/glm_prompt_host.zig +index 1c094d175..3d0709f8f 100644 +--- a/zig/glm_prompt_host.zig ++++ b/zig/glm_prompt_host.zig +@@ -85,7 +85,7 @@ const engine = struct { + if (slot >= 32) return error.BadSlot; + e.cur = &e.seqs[slot]; + } +- pub fn gate(_: *Engine, _: ?u8) !void {} ++ pub fn gate(_: *Engine, _: ?u8, _: bool) !void {} + pub fn exchange(_: *Engine) u8 { + return 0; + } +diff --git a/zig/src/core/heat.zig b/zig/src/core/heat.zig +index eec72a634..19dc487c8 100644 +--- a/zig/src/core/heat.zig ++++ b/zig/src/core/heat.zig +@@ -1,10 +1,11 @@ +-//! Opt-in wait before a prompt chunk when the hottest thermal zone is above a band. ++//! Opt-in wait before a prompt chunk when the hottest thermal zone is above a band or would rise past a ceiling. + const std = @import("std"); + const platform = @import("sys"); + + pub const default_root = "/sys/class/thermal"; + +-pub const Bands = struct { high_c: f64, low_c: f64 }; ++/// Degrees. `ceiling_c` also waits for a chunk's rise; a family that sets it calls `Gate.startPrompt` at each prompt. ++pub const Bands = struct { high_c: f64, low_c: f64, ceiling_c: ?f64 = null }; + + /// Asked after each pause: true ends the wait with error.Cancelled (the request was cancelled meanwhile). + pub const Stop = struct { ptr: *anyopaque, check: *const fn (*anyopaque) bool }; +@@ -15,6 +16,12 @@ pub const Gate = struct { + /// Set after the first unreadable zone, so later chunks skip the read and the log. + no_zone: bool = false, + root: []const u8 = default_root, ++ /// The gathered maximum the last chunk began at; null: the next chunk is a prompt's first. ++ began: ?f64 = null, ++ /// The same when that chunk began straight after another; null: after a rest (a wait, or the gap before a prompt). ++ after: ?f64 = null, ++ /// The last rise of a chunk begun straight after another, in degrees: what the ceiling expects of the next. ++ loaded: f64 = 0, + + /// TF_HEAT_HIGH and TF_HEAT_LOW in degrees, both or neither (neither: no read); TF_HEAT_ROOT moves the zones. + pub fn fromEnv() !Gate { +@@ -22,26 +29,55 @@ pub const Gate = struct { + return .{ .bands = try bandsFrom(envSpan("TF_HEAT_HIGH"), envSpan("TF_HEAT_LOW")), .root = root }; + } + ++ /// A prompt begins: its first chunk has no rise, and it begins after the gap before the prompt. ++ pub fn startPrompt(g: *Gate) void { ++ g.began = null; ++ g.after = null; ++ } ++ + /// `read` returns the gathered maximum in celsius. `sleep_fn` stands in for the 2 s pause. + pub fn waitUntil(g: *Gate, read: *const fn (*anyopaque) anyerror!f64, sleep_fn: *const fn (*anyopaque) void, ctx: *anyopaque, stop: ?Stop) !void { + const bands = g.bands orelse return; + if (g.no_zone) return; +- var waiting = false; +- while (true) { +- const max_c = read(ctx) catch |err| { +- if (err == error.NoZone) { +- g.no_zone = true; +- noteNoZone(g.root); +- return; +- } +- return err; +- }; +- if (allow(bands, max_c, waiting)) return; +- waiting = true; ++ var max_c = (try g.reading(read, ctx)) orelse return; ++ const floor = g.floorFor(bands, max_c) orelse return; ++ while (max_c > floor) { + sleep_fn(ctx); + g.waited_s += 2; + if (stop) |s| if (s.check(s.ptr)) return error.Cancelled; ++ max_c = (try g.reading(read, ctx)) orelse return; + } ++ g.began = max_c; // the next chunk begins after a rest ++ g.after = null; ++ } ++ ++ /// One gathered reading; null once no zone is readable (logged once, and the chunk runs). ++ fn reading(g: *Gate, read: *const fn (*anyopaque) anyerror!f64, ctx: *anyopaque) !?f64 { ++ return read(ctx) catch |err| { ++ if (err != error.NoZone) return err; ++ g.no_zone = true; ++ noteNoZone(g.root); ++ return null; ++ }; ++ } ++ ++ /// Before a chunk at `top`: null to run now, else the reading its wait lasts until (low, or lower to fit a rise). ++ fn floorFor(g: *Gate, b: Bands, top: f64) ?f64 { ++ const inf = std.math.inf(f64); ++ const rise: f64 = if (g.began) |at| (if (at == -inf) 0 else @max(0, top - at)) else 0; ++ if (g.after) |at| { ++ if (at != -inf and top != -inf) g.loaded = rise; // the last chunk began straight after another ++ } ++ const ahead: f64 = if (g.began == null) 0 else g.loaded; ++ const ceiling = b.ceiling_c orelse inf; ++ if (top <= b.high_c and top + ahead <= ceiling) { ++ g.after = if (g.began == null) null else top; ++ g.began = top; ++ return null; ++ } ++ const floor = @min(b.low_c, ceiling - @max(rise, ahead)); ++ noteWait(b, top, ahead, floor); ++ return floor; + } + + /// The real pause, on this process's own hottest zone (`gatheredMax` takes several readings), until `stop` says so. +@@ -76,6 +112,16 @@ pub fn bandsFrom(high: ?[]const u8, low: ?[]const u8) !?Bands { + return bands; + } + ++/// TF_HEAT_CEILING's text on `bands`: in degrees, above the high band, and only with the bands. ++pub fn ceilingFrom(bands: ?Bands, text: ?[]const u8) !?Bands { ++ const t = text orelse return bands; ++ var b = bands orelse return error.HeatBands; ++ const c = try degrees(t); ++ if (c <= b.high_c) return error.HeatBands; ++ b.ceiling_c = c; ++ return b; ++} ++ + pub fn degrees(text: []const u8) !f64 { + return std.fmt.parseFloat(f64, std.mem.trim(u8, text, " \t\r\n")); + } +@@ -86,12 +132,6 @@ pub fn zoneCelsius(text: []const u8) !f64 { + return @as(f64, @floatFromInt(milli)) / 1000.0; + } + +-/// Run when idle and at or under high, or when already waiting and at or under low. +-pub fn allow(bands: Bands, max_c: f64, waiting: bool) bool { +- if (waiting) return max_c <= bands.low_c; +- return max_c <= bands.high_c; +-} +- + pub fn gatheredMax(xs: []const f64) f64 { + var m = xs[0]; + for (xs[1..]) |x| m = @max(m, x); +@@ -120,6 +160,16 @@ pub fn note(seconds_: f64) void { + std.log.info("heat_wait_s {d:.1}", .{seconds_}); + } + ++/// Why a chunk waits, at the wait's start: above the high band, or its expected rise past the ceiling. ++fn noteWait(b: Bands, top: f64, ahead: f64, floor: f64) void { ++ if (@import("builtin").is_test) return; ++ if (top > b.high_c) { ++ std.log.info("heat: hottest zone {d:.1} C above {d} C; the prefill waits for {d:.1} C", .{ top, b.high_c, floor }); ++ } else { ++ std.log.info("heat: hottest zone {d:.1} C plus a chunk's {d:.1} C rise, one run straight after another, above {d} C; the prefill waits for {d:.1} C", .{ top, ahead, b.ceiling_c orelse 0, floor }); ++ } ++} ++ + /// Test builds count the warning. A real process logs it. + var no_zone_notes: u32 = 0; + +@@ -149,10 +199,9 @@ test "bands parse as degrees and a lone value is refused" { + + test "the chunk runs under the high band, and a hot reading waits until the low band" { + const bands = Bands{ .high_c = 92, .low_c = 88 }; +- try std.testing.expect(allow(bands, 92, false)); +- try std.testing.expect(!allow(bands, 92.5, false)); +- try std.testing.expect(!allow(bands, 90, true)); +- try std.testing.expect(allow(bands, 88, true)); ++ var at = Gate{ .bands = bands }; ++ try std.testing.expectEqual(@as(?f64, null), at.floorFor(bands, 92)); // at the high band the chunk runs ++ try std.testing.expectEqual(@as(?f64, 88), at.floorFor(bands, 92.5)); // above it, a wait until the low band + try std.testing.expectEqual(@as(f64, 94), gatheredMax(&.{ 80, 94, 91 })); + const Script = struct { + temps: []const f64, +@@ -195,6 +244,52 @@ test "the chunk runs under the high band, and a hot reading waits until the low + try std.testing.expectEqual(@as(u32, 1), stuck.sleeps); // a cancel ends the wait after the pause it came in + } + ++/// A gate's gathered maxima, one a read (the last repeats), and the pauses taken. ++const Readings = struct { ++ temps: []const f64, ++ i: usize = 0, ++ sleeps: u32 = 0, ++ fn read(ptr: *anyopaque) !f64 { ++ const r: *Readings = @ptrCast(@alignCast(ptr)); ++ const t = r.temps[r.i]; ++ if (r.i + 1 < r.temps.len) r.i += 1; ++ return t; ++ } ++ fn sleep(ptr: *anyopaque) void { ++ @as(*Readings, @ptrCast(@alignCast(ptr))).sleeps += 1; ++ } ++ /// One chunk's gate over `temps`: the pauses it took. ++ fn chunk(g: *Gate, temps: []const f64) !u32 { ++ var r = Readings{ .temps = temps }; ++ try g.waitUntil(read, sleep, &r, null); ++ return r.sleeps; ++ } ++}; ++ ++test "a ceiling comes with both bands and above the high one" { ++ const b = Bands{ .high_c = 92, .low_c = 88 }; ++ try std.testing.expect((try ceilingFrom(null, null)) == null); ++ try std.testing.expectEqual(b, (try ceilingFrom(b, null)).?); ++ try std.testing.expectEqual(@as(?f64, 93), (try ceilingFrom(b, " 93\n")).?.ceiling_c); ++ try std.testing.expectError(error.HeatBands, ceilingFrom(null, "93")); ++ try std.testing.expectError(error.HeatBands, ceilingFrom(b, "92")); ++} ++ ++test "with a ceiling, a chunk begun straight after another waits until the last such rise fits under it" { ++ var g = Gate{ .bands = .{ .high_c = 92, .low_c = 88, .ceiling_c = 93 } }; ++ g.startPrompt(); ++ try std.testing.expectEqual(@as(u32, 0), try Readings.chunk(&g, &.{70})); // a prompt's first chunk: no rise ++ try std.testing.expectEqual(@as(u32, 0), try Readings.chunk(&g, &.{80})); // chunk 1 began after the gap: not kept ++ try std.testing.expectEqual(@as(u32, 2), try Readings.chunk(&g, &.{ 90, 86, 83 })); // 90 + 10 > 93: wait to 83 ++ try std.testing.expectEqual(@as(u32, 2), try Readings.chunk(&g, &.{ 91, 84, 83 })); // its own 8 C, but 10 C counts ++ try std.testing.expectEqual(@as(f64, 8), g.waited_s); ++ g.startPrompt(); ++ try std.testing.expectEqual(@as(u32, 0), try Readings.chunk(&g, &.{ 90, 70 })); // a new prompt's first takes none ++ var bands = Gate{ .bands = .{ .high_c = 92, .low_c = 88 } }; // the same readings with the bands alone ++ bands.startPrompt(); ++ for ([_]f64{ 70, 80, 90 }) |t| try std.testing.expectEqual(@as(u32, 0), try Readings.chunk(&bands, &.{t})); ++} ++ + const env = if (@import("builtin").os.tag == .windows) @import("sys").c else struct { + extern "c" fn setenv(name: [*:0]const u8, value: [*:0]const u8, overwrite: i32) i32; + extern "c" fn unsetenv(name: [*:0]const u8) i32; +diff --git a/zig/src/families/glm/cuda_engine.zig b/zig/src/families/glm/cuda_engine.zig +index a15f5cc18..097be10c0 100644 +--- a/zig/src/families/glm/cuda_engine.zig ++++ b/zig/src/families/glm/cuda_engine.zig +@@ -79,7 +79,7 @@ pub const Engine = struct { + slot: u32 = 0, + capacity: usize, + mtp: bool, +- heat_gate: heat.Gate, // upstream's bands (off unless set), read before each prompt chunk on the gathered maximum ++ heat_gate: heat.Gate, // upstream's bands and ceiling (off unless set), read before each prompt chunk on the gathered maximum + shared_buf: cuda.DeviceBuffer, // one f64 every rank all-reduces to its maximum + snaps: std.AutoHashMapUnmanaged(u64, Snap) = .empty, // kept prompt states by id (the same ids on every rank) + snap_pages: ?snapshots.pages.Pool = null, +@@ -304,6 +304,7 @@ pub const Engine = struct { + /// decode.prefill: `ids` in 2,048-row chunks, then the first token from the last row; `from` restores a kept state. + pub fn prefill(e: *Engine, ids: []const u32, from: ?Resume, cancel: ?Cancel, marks: ?Marks) !u32 { + var start = try e.prefillStart(ids.len, from); ++ e.heat_gate.startPrompt(); // its chunks get no `from`: a resumed pass's first is marked here + while (true) : (start += fw.prefill_rows) { + if (try e.prefillChunk(ids[start..@min(ids.len, start + fw.prefill_rows + 1)], start, ids.len, null, cancel, marks)) |first| return first; + } +@@ -341,7 +342,7 @@ pub const Engine = struct { + p.prompt.?.invalid = true; + }; + var stamp = prof_mod.span(f.prof, e.stream, .gate); +- try e.gate(cancel); ++ try e.gate(cancel, start == 0 or from != null); + stamp.end(); + if (f.prof) |p| p.mark(e.stream, .gate); + try f.run(as_i32[0..rows]); +@@ -384,9 +385,10 @@ pub const Engine = struct { + } + + /// The chunk gate on every rank: a kept state at the mark, the heat wait, rank 0's cancel; the id exchanged first. +- pub fn gate(e: *Engine, cancel: ?Cancel) !void { ++ pub fn gate(e: *Engine, cancel: ?Cancel, first: bool) !void { + try snapshots.gate(e); + if (e.heat_gate.bands != null) { ++ if (first) e.heat_gate.startPrompt(); // a pass's first chunk carries no rise in + var w: Wait = .{ .e = e, .cancel = cancel }; + const before = e.heat_gate.waited_s; + e.heat_gate.waitUntil(Wait.read, Wait.sleep, &w, .{ .ptr = &w, .check = Wait.stop }) catch |err| return w.failed orelse err; +diff --git a/zig/src/families/glm/cuda_prompts.zig b/zig/src/families/glm/cuda_prompts.zig +index b66392f44..da12cc079 100644 +--- a/zig/src/families/glm/cuda_prompts.zig ++++ b/zig/src/families/glm/cuda_prompts.zig +@@ -36,7 +36,7 @@ pub fn pass(e: anytype, parts: []const engine.Prompt, firsts: []u32, comptime Fo + @memcpy(ids[rows..][0..p.ids.len], p.ids); + rows += p.ids.len; + } +- try e.gate(null); ++ try e.gate(null, true); // fresh prompts at position 0: the pass carries no rise in + var one: [1]Rank = .{.{ .w = &e.w, .st = segs[0].st, .b = &e.bp, .segs = segs[0..parts.len] }}; + var f: Forward = .{ .d = e.d, .m = &e.m, .stream = e.stream, .ranks = &one, .exchange = e.exchange(), .pieces = &e.pieces, .split = e.settings.split, .ring = e.settings.ring, .kda_wide = e.settings.kda_wide, .taps = if (e.drafter) |dr| dr.taps(true) else null }; + try f.run(@ptrCast(ids[0..rows])); +diff --git a/zig/src/families/glm/cuda_ranks.zig b/zig/src/families/glm/cuda_ranks.zig +index 7300deb65..02ab85da5 100644 +--- a/zig/src/families/glm/cuda_ranks.zig ++++ b/zig/src/families/glm/cuda_ranks.zig +@@ -15,7 +15,7 @@ pub const Settings = struct { + ring: bool = false, // TF_GLM_PREFILL_REDUCE=ring: split, its exchanges passed along the ring (cuda_relay) + kda_wide: bool = true, // TF_GLM_KDA_DECODE_WIDE: 1 (decode windows on KDA's three-kernel chain) or 0 + pieces: u32 = 3, // TF_GLM_OVERLAP_PIECES: a prompt chunk's exchanges in this many row pieces (1: whole). +- heat: ?heat.Bands = null, // TF_HEAT_HIGH and TF_HEAT_LOW (upstream's names): a prompt chunk waits on every rank's hottest zone ++ heat: ?heat.Bands = null, // TF_HEAT_HIGH, TF_HEAT_LOW (and TF_HEAT_CEILING): a prompt chunk waits on every rank's hottest zone + layers: u32 = 0, // TF_GLM_LAYERS: the first this many layers only, a thin model for one-GPU checks (0: all) + dense_bits: u8 = 16, // TF_GLM_DENSE_BITS: the dense linears packed at load to 4, 6 or 8 bits (16: bf16) + head_bits: u8 = 16, // TF_GLM_HEAD_BITS: the head, likewise +@@ -78,10 +78,14 @@ pub const Settings = struct { + }, + }; + }; +- s.heat = heat.bandsFrom(get("TF_HEAT_HIGH"), get("TF_HEAT_LOW")) catch { ++ const bands = heat.bandsFrom(get("TF_HEAT_HIGH"), get("TF_HEAT_LOW")) catch { + problem.* = "TF_HEAT_HIGH and TF_HEAT_LOW: both or neither, in degrees, with low at or under high"; + return error.BadSetting; + }; ++ s.heat = heat.ceilingFrom(bands, get("TF_HEAT_CEILING")) catch { ++ problem.* = "TF_HEAT_CEILING: degrees above TF_HEAT_HIGH, with both bands"; ++ return error.BadSetting; ++ }; + if (s.prompt_fixed_k and !s.tiled) { + problem.* = "TF_GLM_PROMPT_K=shape requires TF_GLM_MM=1"; + return error.BadSetting; +@@ -121,6 +125,7 @@ pub const Fingerprint = extern struct { + capacity: u64, // tokens a sequence holds + heat_high: f64, // the heat bands in degrees (0 and 0: off): every rank must wait alike + heat_low: f64, ++ heat_ceiling: f64, // TF_HEAT_CEILING (0: none) + config: [32]u8, // sha256 of config.json + program: [32]u8, // sha256 of this executable + checkpoint: [64]u8, // hex digest of the checkpoint's metadata and every shard byte, through the digest cache +@@ -147,6 +152,7 @@ pub const Fingerprint = extern struct { + capacity, + heat_high, + heat_low, ++ heat_ceiling, + config, + program, + checkpoint, +@@ -178,6 +184,7 @@ pub const Fingerprint = extern struct { + .capacity = capacity, + .heat_high = if (s.heat) |b| b.high_c else 0, + .heat_low = if (s.heat) |b| b.low_c else 0, ++ .heat_ceiling = if (s.heat) |b| b.ceiling_c orelse 0 else 0, + .config = config, + .program = program, + .checkpoint = std.fmt.bytesToHex(checkpoint_digest, .lower), +@@ -213,6 +220,7 @@ pub fn mismatch(all: []const Fingerprint) ?Mismatch { + "capacity", + "heat_high", + "heat_low", ++ "heat_ceiling", + "config", + "program", + "checkpoint", +@@ -321,6 +329,7 @@ fn explain(f: Fingerprint.Field) []const u8 { + .capacity => "the sequence capacity (the context)", + .heat_high => "TF_HEAT_HIGH", + .heat_low => "TF_HEAT_LOW", ++ .heat_ceiling => "TF_HEAT_CEILING", + .config => "config.json", + .program => "the executable", + .checkpoint => "checkpoint metadata or shard bytes", +@@ -332,6 +341,7 @@ fn fake(name: [:0]const u8) ?[]const u8 { + if (std.mem.eql(u8, name, "TF_GLM_OVERLAP_PIECES")) return "2"; + if (std.mem.eql(u8, name, "TF_HEAT_HIGH")) return "92"; + if (std.mem.eql(u8, name, "TF_HEAT_LOW")) return "88"; ++ if (std.mem.eql(u8, name, "TF_HEAT_CEILING")) return "93"; + if (std.mem.eql(u8, name, "TF_GLM_DENSE_BITS")) return "4"; + if (std.mem.eql(u8, name, "TF_GLM_HEAD_BITS")) return "8"; + return null; +@@ -347,6 +357,11 @@ fn bad(name: [:0]const u8) ?[]const u8 { + return null; + } + ++fn lowCeiling(name: [:0]const u8) ?[]const u8 { ++ if (std.mem.eql(u8, name, "TF_HEAT_CEILING")) return "92"; ++ return fake(name); ++} ++ + fn typo(name: [:0]const u8) ?[]const u8 { + if (std.mem.eql(u8, name, "TF_GLM_DENSE_BITS")) return "abc"; + return null; +@@ -357,8 +372,11 @@ test "settings from the environment: the defaults, the choices, and a refusal na + const s = try Settings.read(&fake, &problem); + try std.testing.expect(!s.split and s.kda_wide and s.pieces == 2 and s.heat.?.high_c == 92 and s.heat.?.low_c == 88); + try std.testing.expect(s.dense_bits == 4 and s.head_bits == 8); ++ try std.testing.expectEqual(@as(?f64, 93), s.heat.?.ceiling_c); + try std.testing.expectError(error.BadSetting, Settings.read(&bad, &problem)); + try std.testing.expectEqualStrings("TF_GLM_PREFILL_REDUCE: split, ring or gather", problem); ++ try std.testing.expectError(error.BadSetting, Settings.read(&lowCeiling, &problem)); ++ try std.testing.expectEqualStrings("TF_HEAT_CEILING: degrees above TF_HEAT_HIGH, with both bands", problem); + const r = try Settings.read(&ringed, &problem); + try std.testing.expect(r.split and r.ring); + try std.testing.expectEqual(@as(u32, 2), Fingerprint.of(2, 0, r, 8192, true, @splat(0), @splat(0), hashBytes("/c")).split); +@@ -386,6 +404,11 @@ test "fingerprints: agreeing ranks pass; the first differing field and rank are + try std.testing.expectEqual(Fingerprint.Field.heat_high, mismatch(&all).?.field); + all[1] = Fingerprint.of(2, 1, .{ .pages = true }, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); + try std.testing.expectEqual(Mismatch{ .field = .pages, .rank = 1 }, mismatch(&all).?); // TF_KV_PAGES on one rank ++ const ceiling_on: Settings = .{ .heat = .{ .high_c = 92, .low_c = 88, .ceiling_c = 93 } }; ++ all[0] = Fingerprint.of(2, 0, heat_on, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); ++ all[1] = Fingerprint.of(2, 1, ceiling_on, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); ++ try std.testing.expectEqual(Fingerprint.Field.heat_ceiling, mismatch(&all).?.field); ++ all[0] = Fingerprint.of(2, 0, .{}, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); + all[1] = Fingerprint.of(2, 1, .{}, 8192, true, h0, h0, hashBytes("/hub/snapshots/other")); + try std.testing.expectEqual(Fingerprint.Field.checkpoint, mismatch(&all).?.field); + all[1] = Fingerprint.of(2, 1, .{}, 8192, false, h0, h0, hashBytes("/hub/snapshots/423acf37")); // --no-drafts on rank 1 only +@@ -509,7 +532,7 @@ test "every media limit is part of the ranks' agreement" { + test "every scheduling and identity field that differs stops the ranks" { + const zero = Fingerprint.of(2, 0, .{}, 8192, true, @splat(0), @splat(0), hashBytes("/same")); + const one = Fingerprint.of(2, 1, .{}, 8192, true, @splat(0), @splat(0), hashBytes("/same")); +- inline for (.{ "layout", "world", "rank", "pages", "split", "kda_wide", "pieces", "layers", "mtp", "vision", "video_tokens", "max_images", "max_videos", "max_frames", "capacity", "heat_high", "heat_low" }) |name| { ++ inline for (.{ "layout", "world", "rank", "pages", "split", "kda_wide", "pieces", "layers", "mtp", "vision", "video_tokens", "max_images", "max_videos", "max_frames", "capacity", "heat_high", "heat_low", "heat_ceiling" }) |name| { + var all = [_]Fingerprint{ zero, one }; + @field(all[1], name) += 1; + try std.testing.expectEqual(@field(Fingerprint.Field, name), mismatch(&all).?.field); +-- +2.56.0.windows.2 + + +From 10698886af0b9db9acfda5687d5a94b9b9059a95 Mon Sep 17 00:00:00 2001 +From: +Date: Mon, 19 Oct 2026 15:19:38 +0900 +Subject: [PATCH 2/2] glm: TF_GLM_NCCL_ID_FILE hands rank 0's NCCL id to the + others in a file, and NCCL_COMM_ID then names only the ranks' lines (local, + not for upstream: TP=3 on the ring's /32 until #636) + +1.1.0's tp_watch reads NCCL_COMM_ID for the ranks' lines, and NCCL's bootstrap needs an interface in that address's +subnet, which a /32 ring has only on rank 0. With TF_GLM_NCCL_ID_FILE set, each rank logs the NCCL_COMM_ID it takes +out of its environment after the lines are up and before NCCL's first call; rank 0 writes its unique id to the file +as hex and the others read it there (5b6aa233's form), waiting up to 10 minutes. +--- + zig/src/families/glm/cuda_native.zig | 37 +++++++++++++++++++++++++--- + 1 file changed, 34 insertions(+), 3 deletions(-) + +diff --git a/zig/src/families/glm/cuda_native.zig b/zig/src/families/glm/cuda_native.zig +index 25130bf22..7e098f59b 100644 +--- a/zig/src/families/glm/cuda_native.zig ++++ b/zig/src/families/glm/cuda_native.zig +@@ -126,6 +126,36 @@ const Owned = struct { + }; + + extern "c" fn setenv(name: [*:0]const u8, value: [*:0]const u8, overwrite: c_int) c_int; ++extern "c" fn unsetenv(name: [*:0]const u8) c_int; ++ ++/// Local, not for upstream (TP=3 on the ring's /32 until #636): with TF_GLM_NCCL_ID_FILE, NCCL_COMM_ID names only the ++/// ranks' lines; it leaves NCCL's environment, whose bootstrap looks for an interface in that address's subnet. ++fn commIdByFile(rank: usize) ?[]const u8 { ++ const f = std.mem.span(std.c.getenv("TF_GLM_NCCL_ID_FILE") orelse return null); ++ const was = std.mem.span(std.c.getenv("NCCL_COMM_ID") orelse ""); ++ std.log.info("rank {d}: TF_GLM_NCCL_ID_FILE {s}; NCCL_COMM_ID {s} leaves NCCL's environment", .{ rank, f, was }); ++ _ = unsetenv("NCCL_COMM_ID"); ++ return f; ++} ++ ++/// Local: rank 0 writes its unique id to `path` as hex; the others read it there, waiting up to 10 minutes. ++fn idByFile(io: std.Io, path: []const u8, rank: usize, lib: *const cuda.nccl.Library, id: *cuda.nccl.UniqueId) !void { ++ if (rank == 0) { ++ try lib.check(lib.api.ncclGetUniqueId(id), "ncclGetUniqueId"); ++ const hex = std.fmt.bytesToHex(id.internal, .lower); ++ return std.Io.Dir.cwd().writeFile(io, .{ .sub_path = path, .data = &hex }); ++ } ++ var hex: [256]u8 = undefined; ++ for (0..600) |_| { ++ const got = std.Io.Dir.cwd().readFile(io, path, &hex) catch hex[0..0]; ++ if (got.len == hex.len) { ++ _ = try std.fmt.hexToBytes(&id.internal, &hex); ++ return; ++ } ++ std.Io.sleep(io, .fromMilliseconds(1000), .awake) catch {}; ++ } ++ return error.NoCommId; ++} + + /// Every peer's send/recv connected now, before the weights fill memory (connected on first use, ibv_reg_mr can fail). + fn warmPeers(d: *const cuda.Driver, g: *const cuda.tp_group.Group) !void { +@@ -179,14 +209,15 @@ pub fn open(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: [ + set.pages = try paging(if (std.c.getenv("TF_KV_PAGES")) |v| std.mem.span(v) else null, p.world, o.kv8); // every rank + // the ranks' lines before NCCL (cuda/tp_watch.zig): a rank that fails, refuses or dies stops every rank + if (p.world > 1) try cuda.tp_watch.start(io, p.rank, p.world); +- return openRank(gpa, io, ctx, dir, o, p, set, roce_bytes) catch |err| { ++ const id_file = commIdByFile(p.rank); // local: after the lines read NCCL_COMM_ID, before NCCL's first call ++ return openRank(gpa, io, ctx, dir, o, p, set, roce_bytes, id_file) catch |err| { + cuda.tp_watch.fail(@errorName(err)); // the other ranks stop with this rank's error + return err; + }; + } + + /// `open` once the ranks' lines are up: NCCL, the engine and the link, then a follower's loop or rank 0's lanes. +-fn openRank(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: []const u8, o: Options, p: Place, set: ranks.Settings, roce_bytes: usize) !Loaded { ++fn openRank(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: []const u8, o: Options, p: Place, set: ranks.Settings, roce_bytes: usize, id_file: ?[]const u8) !Loaded { + const own = try gpa.create(Owned); + errdefer gpa.destroy(own); + own.gpa = gpa; +@@ -198,7 +229,7 @@ fn openRank(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: [ + var lib = try cuda.nccl.Library.open(); + errdefer lib.close(); + var id: cuda.nccl.UniqueId = undefined; +- try lib.check(lib.api.ncclGetUniqueId(&id), "ncclGetUniqueId"); // NCCL_COMM_ID: the same id on every rank ++ if (id_file) |f| try idByFile(io, f, p.rank, &lib, &id) else try lib.check(lib.api.ncclGetUniqueId(&id), "ncclGetUniqueId"); // NCCL_COMM_ID: the same id on every rank + var comm: cuda.nccl.Comm = null; + try lib.check(lib.api.ncclCommInitRank(&comm, @intCast(p.world), id, @intCast(p.rank)), "ncclCommInitRank"); + own.group = .{ .lib = lib, .comm = comm, .rank = p.rank, .world = p.world }; +-- +2.56.0.windows.2 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/admit.json b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/admit.json new file mode 100644 index 0000000000..b7d0dadf8d --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/admit.json @@ -0,0 +1 @@ +{"alone": [{"prompt": "p65536-0", "seed": 1000, "sent": 756027.874775344, "first": 756071.994693837, "last": 756075.856286231, "tokens": 128, "pieces": [[756071.994693837, 3], [756072.070464986, 9], [756072.124290139, 14], [756072.178344306, 10], [756072.233886141, 17], [756072.296894709, 10], [756072.364175198, 4], [756072.424449911, 5], [756072.481450086, 4], [756072.542819121, 5], [756072.604111296, 10], [756072.665491179, 9], [756072.72757241, 5], [756072.79144755, 5], [756072.863110468, 11], [756072.931929231, 7], [756072.998096011, 11], [756073.066355394, 35], [756073.139485281, 8], [756073.205467061, 7], [756073.260611796, 10], [756073.316349963, 4], [756073.370919426, 17], [756073.42585017, 16], [756073.480925836, 9], [756073.532781074, 3], [756073.587576399, 10], [756073.642577748, 13], [756073.69718153, 7], [756073.750581616, 14], [756073.804935168, 9], [756073.859123366, 11], [756073.924573216, 5], [756073.983090554, 5], [756074.036624, 9], [756074.089168638, 16], [756074.142433486, 17], [756074.194601531, 23], [756074.249310545, 4], [756074.303973816, 3], [756074.370410702, 4], [756074.439868681, 5], [756074.502590775, 6], [756074.566778794, 9], [756074.636030029, 11], [756074.702608654, 16], [756074.770151447, 5], [756074.827732599, 6], [756074.890519651, 11], [756074.952272699, 12], [756075.01342939, 8], [756075.077078811, 5], [756075.146600233, 3], [756075.217173437, 8], [756075.284230015, 3], [756075.345364947, 2], [756075.406084327, 5], [756075.460236264, 8], [756075.51455037, 6], [756075.569087043, 9], [756075.623753998, 13], [756075.679839072, 14], [756075.743079038, 9], [756075.803772516, 9], [756075.856286231, 5]], "completed": 756075.856371763, "ttft_s": 44.11991849297192, "decode_tps": 32.887986883067704, "prompt_tokens": 65536, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "d8a8421597ee", "token_sha256": "d8a8421597ee17ad5b7801dacb29f07eb6692a742fb9fc92104dd0e7527a4a3d", "output_sha256": "0dfe13857bf300b4a865cec4f36e215954dd3fa63c9e610b0d86d3613508c25e", "elapsed_s": 47.981596419005655, "chunk_offsets_s": [44.11991849297192, 44.195689641986974, 44.249514794908464, 44.3035689619137, 44.35911079694051, 44.422119364957325, 44.489399853977375, 44.54967456695158, 44.60667474195361, 44.668043776997365, 44.729335951968096, 44.790715834940784, 44.85279706597794, 44.916672205901705, 44.98833512398414, 45.057153886999004, 45.123320666956715, 45.191580049926415, 45.26470993692055, 45.33069171698298, 45.38583645201288, 45.44157461891882, 45.49614408193156, 45.55107482592575, 45.60615049197804, 45.65800573001616, 45.71280105493497, 45.76780240400694, 45.82240618590731, 45.87580627191346, 45.93015982396901, 45.98434802191332, 46.049797871964984, 46.108315209974535, 46.16184865590185, 46.21439329395071, 46.267658141907305, 46.319826186983846, 46.37453520100098, 46.42919847194571, 46.49563535791822, 46.56509333697613, 46.6278154309839, 46.69200345000718, 46.761254684999585, 46.82783331000246, 46.89537610299885, 46.95295725495089, 47.015744306962006, 47.07749735494144, 47.13865404599346, 47.20230346696917, 47.27182488900144, 47.342398092965595, 47.409454670967534, 47.470589602948166, 47.53130898298696, 47.58546092000324, 47.639775026007555, 47.69431169901509, 47.74897865392268, 47.80506372801028, 47.86830369394738, 47.92899717192631, 47.98151088692248], "chunk_gaps_ms": {"count": 64, "p50": 60.70642900886014, "p95": 70.57320396415889, "max": 75.7711490150541}}, {"prompt": "p65536-1", "seed": 1001, "sent": 756075.857355659, "first": 756119.433744651, "last": 756122.552193051, "tokens": 128, "pieces": [[756119.433744651, 2], [756119.516946546, 11], [756119.576275047, 14], [756119.636865596, 6], [756119.70499132, 20], [756119.774436386, 18], [756119.841284765, 18], [756119.907564877, 28], [756119.97251082, 2], [756120.03315174, 14], [756120.092080659, 16], [756120.151301072, 7], [756120.204445216, 17], [756120.272077043, 10], [756120.327718661, 4], [756120.388774622, 9], [756120.449860517, 8], [756120.513011727, 9], [756120.575679195, 11], [756120.640774461, 6], [756120.710162542, 12], [756120.779840164, 12], [756120.850310733, 11], [756120.918374784, 12], [756120.986720605, 21], [756121.05707296, 7], [756121.126374041, 10], [756121.195848188, 11], [756121.265059545, 18], [756121.333207678, 15], [756121.39879365, 11], [756121.460080214, 10], [756121.519831427, 6], [756121.573151552, 6], [756121.628497618, 10], [756121.689346134, 20], [756121.74752771, 20], [756121.809554855, 7], [756121.86841143, 6], [756121.92057523, 18], [756121.976137292, 7], [756122.035891197, 16], [756122.096457921, 15], [756122.157352488, 12], [756122.218715102, 13], [756122.281610683, 2], [756122.337577481, 7], [756122.396260275, 10], [756122.450842821, 9], [756122.501817453, 1], [756122.552193051, 6]], "completed": 756122.552346805, "ttft_s": 43.57638899201993, "decode_tps": 40.725381251297534, "prompt_tokens": 65536, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "bf47482e16cb", "token_sha256": "bf47482e16cb91a3294931e7a597aa8938db10afebffe6941e0abe5a20c9d02d", "output_sha256": "506ecff1415d0489be9cdf8709f2dace158282371918e04ed030e81d69827fa7", "elapsed_s": 46.694991145981476, "chunk_offsets_s": [43.57638899201993, 43.659590887022205, 43.71891938801855, 43.77950993704144, 43.84763566101901, 43.917080726940185, 43.98392910603434, 44.05020921793766, 44.11515516100917, 44.175796080962755, 44.23472499998752, 44.29394541296642, 44.34708955697715, 44.414721383946016, 44.47036300203763, 44.53141896298621, 44.59250485803932, 44.655656068003736, 44.71832353598438, 44.7834188019624, 44.85280688293278, 44.92248450499028, 44.99295507394709, 45.06101912504528, 45.12936494604219, 45.19971730094403, 45.269018381950445, 45.33849252900109, 45.40770388604142, 45.47585201903712, 45.54143799096346, 45.60272455494851, 45.66247576801106, 45.71579589298926, 45.771141959005035, 45.83199047495145, 45.890172051033005, 45.9521991959773, 46.01105577102862, 46.063219571020454, 46.11878163297661, 46.17853553802706, 46.23910226195585, 46.299996829009615, 46.36135944293346, 46.42425502394326, 46.48022182204295, 46.53890461602714, 46.59348716202658, 46.644461793941446, 46.69483739195857], "chunk_gaps_ms": {"count": 50, "p50": 61.18622951908037, "p95": 70.35235490184277, "max": 83.2018950022757}}, {"prompt": "p65536-2", "seed": 1002, "sent": 756122.553051691, "first": 756166.025516691, "last": 756169.299324171, "tokens": 128, "pieces": [[756166.025516691, 4], [756166.106562037, 7], [756166.16056237, 7], [756166.213109952, 14], [756166.26832834, 9], [756166.322286834, 16], [756166.376914601, 10], [756166.430351081, 8], [756166.484517585, 4], [756166.538314469, 2], [756166.592393551, 19], [756166.64645081, 10], [756166.69942321, 11], [756166.752999557, 9], [756166.808022271, 4], [756166.864334893, 9], [756166.925988869, 5], [756166.987518213, 12], [756167.050367928, 12], [756167.110403883, 15], [756167.17155908, 16], [756167.234990791, 3], [756167.289652577, 13], [756167.352655071, 14], [756167.411424893, 5], [756167.465419582, 5], [756167.519209269, 14], [756167.574282817, 16], [756167.633624636, 25], [756167.696733414, 10], [756167.75561656, 17], [756167.820147242, 5], [756167.890331491, 9], [756167.955254194, 14], [756168.023778952, 14], [756168.088295239, 4], [756168.14338263, 9], [756168.205406649, 9], [756168.265659929, 17], [756168.327030434, 8], [756168.390654143, 18], [756168.460785151, 23], [756168.522955437, 5], [756168.576102238, 9], [756168.629337228, 8], [756168.684204339, 10], [756168.737387555, 15], [756168.792820022, 17], [756168.854461318, 17], [756168.914992013, 9], [756168.978719658, 14], [756169.039863551, 9], [756169.123448667, 28], [756169.190317598, 19], [756169.248332749, 14], [756169.299324171, 1]], "completed": 756169.299432328, "ttft_s": 43.47246499999892, "decode_tps": 38.792751490689845, "prompt_tokens": 65536, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "30f96d96d0b7", "token_sha256": "30f96d96d0b71ddd72f2ccfbcb05bce868c47cc84c22cbf27d7b26c6a33cc946", "output_sha256": "cb508766a2f4dc5642532a8d981a8d3b8e900dc060b072caa7c1c3e94292cc37", "elapsed_s": 46.74638063693419, "chunk_offsets_s": [43.47246499999892, 43.55351034598425, 43.607510678935796, 43.66005826101173, 43.71527664898895, 43.76923514297232, 43.823862909921445, 43.877299389918335, 43.93146589398384, 43.98526277800556, 44.03934185998514, 44.093399118981324, 44.14637151896022, 44.19994786591269, 44.254970579990186, 44.311283201910555, 44.372937178006396, 44.434466521954164, 44.497316236956976, 44.55735219200142, 44.6185073889792, 44.681939099915326, 44.736600885982625, 44.799603379913606, 44.85837320191786, 44.912367890938185, 44.96615757793188, 45.021231125923805, 45.08057294494938, 45.143681722925976, 45.2025648689596, 45.26709555101115, 45.337279799976386, 45.40220250294078, 45.47072726092301, 45.53524354798719, 45.590330938925035, 45.652354958001524, 45.71260823798366, 45.77397874300368, 45.83760245190933, 45.907733459956944, 45.96990374592133, 46.023050546995364, 46.07628553698305, 46.131152648013085, 46.1843358639162, 46.239768330939114, 46.301409626961686, 46.36194032197818, 46.42566796694882, 46.48681185999885, 46.57039697596338, 46.637265907018445, 46.695281057967804, 46.74627247999888], "chunk_gaps_ms": {"count": 55, "p50": 58.883146033622324, "p95": 70.18424896523356, "max": 83.58511596452445}}], "together": [{"prompt": "p65536-0", "seed": 1000, "sent": 756169.30082372, "first": 756170.75382812, "last": 756174.525387157, "tokens": 128, "pieces": [[756170.75382812, 3], [756170.830196062, 9], [756170.88499475, 14], [756170.938711044, 10], [756170.994112356, 17], [756171.055421356, 10], [756171.115106058, 4], [756171.169391408, 5], [756171.225441545, 4], [756171.28695729, 5], [756171.348084296, 10], [756171.40919407, 9], [756171.470697431, 5], [756171.534296514, 5], [756171.604702982, 11], [756171.674231288, 7], [756171.739730242, 11], [756171.808080396, 35], [756171.874177074, 8], [756171.932208873, 7], [756171.985579297, 10], [756172.03926467, 4], [756172.092522057, 17], [756172.147740291, 16], [756172.203784304, 9], [756172.254723098, 3], [756172.308459029, 10], [756172.362418057, 13], [756172.415807344, 7], [756172.468938063, 14], [756172.523228184, 9], [756172.576229612, 11], [756172.642269688, 5], [756172.70114207, 5], [756172.754476943, 9], [756172.807795768, 16], [756172.860855658, 17], [756172.91322882, 23], [756172.967190411, 4], [756173.022228142, 3], [756173.082582638, 4], [756173.143823104, 5], [756173.203965607, 6], [756173.267191494, 9], [756173.336086103, 11], [756173.402945211, 16], [756173.46911236, 5], [756173.525264101, 6], [756173.58607839, 11], [756173.647526625, 12], [756173.709137031, 8], [756173.773130028, 5], [756173.841212408, 3], [756173.9120274, 8], [756173.976410324, 3], [756174.029600489, 2], [756174.082055398, 5], [756174.135773913, 8], [756174.18947254, 6], [756174.243794811, 9], [756174.298049804, 13], [756174.354126224, 14], [756174.416136476, 9], [756174.475038529, 9], [756174.525387157, 5]], "completed": 756174.525496209, "ttft_s": 1.4530043998965994, "decode_tps": 33.67307756630784, "prompt_tokens": 65536, "cached_tokens": 63488, "cache_state": "reused", "complete": true, "token_sha": "d8a8421597ee", "token_sha256": "d8a8421597ee17ad5b7801dacb29f07eb6692a742fb9fc92104dd0e7527a4a3d", "output_sha256": "0dfe13857bf300b4a865cec4f36e215954dd3fa63c9e610b0d86d3613508c25e", "elapsed_s": 5.224672488984652, "chunk_offsets_s": [1.4530043998965994, 1.5293723419308662, 1.5841710299719125, 1.6378873239737004, 1.6932886359281838, 1.7545976359397173, 1.8142823379021138, 1.8685676879249513, 1.9246178249595687, 1.986133569967933, 2.047260575927794, 2.108370349975303, 2.169873710954562, 2.2334727939451113, 2.3038792619481683, 2.3734075679676607, 2.438906522002071, 2.50725667597726, 2.5733533538877964, 2.6313851529266685, 2.6847555768908933, 2.738440949935466, 2.7916983369505033, 2.8469165709102526, 2.9029605839168653, 2.9538993779569864, 3.00763530889526, 3.0615943369921297, 3.1149836239637807, 3.1681143429595977, 3.2224044639151543, 3.275405891938135, 3.341445967904292, 3.4003183499444276, 3.453653222997673, 3.5069720479659736, 3.5600319379009306, 3.6124050999060273, 3.666366690886207, 3.721404421958141, 3.7817589179612696, 3.8429993839235976, 3.9031418869271874, 3.966367773944512, 4.035262382938527, 4.102121490985155, 4.168288639979437, 4.224440380930901, 4.285254669957794, 4.346702904906124, 4.408313310937956, 4.47230630798731, 4.54038868797943, 4.61120367993135, 4.675586603931151, 4.72877676889766, 4.781231677974574, 4.834950192947872, 4.88864881999325, 4.942971090902574, 4.997226084000431, 5.053302503889427, 5.115312755922787, 5.174214808968827, 5.2245634369319305], "chunk_gaps_ms": {"count": 64, "p50": 57.09176999516785, "p95": 69.52830601949245, "max": 76.36794203426689}}, {"prompt": "p65536-1", "seed": 1001, "sent": 756169.302428482, "first": 756180.691492059, "last": 756183.819128218, "tokens": 128, "pieces": [[756180.691492059, 2], [756180.77312322, 11], [756180.832871427, 14], [756180.895087921, 6], [756180.963132624, 20], [756181.032751276, 18], [756181.099223697, 18], [756181.166439181, 28], [756181.231496993, 2], [756181.291190358, 14], [756181.351453257, 16], [756181.409766828, 7], [756181.463783197, 17], [756181.529966683, 10], [756181.585890862, 4], [756181.647590465, 9], [756181.709152616, 8], [756181.77339887, 9], [756181.835775843, 11], [756181.900868469, 6], [756181.970955958, 12], [756182.040014185, 12], [756182.11084141, 11], [756182.17989463, 12], [756182.248605908, 21], [756182.320188244, 7], [756182.389717816, 10], [756182.458092162, 11], [756182.527430716, 18], [756182.594942467, 15], [756182.661145221, 11], [756182.723133538, 10], [756182.783290876, 6], [756182.837371151, 6], [756182.892902146, 10], [756182.95452019, 20], [756183.012528626, 20], [756183.073841418, 7], [756183.132157941, 6], [756183.18505397, 18], [756183.240108872, 7], [756183.300446016, 16], [756183.361514832, 15], [756183.422309032, 12], [756183.484047634, 13], [756183.546834473, 2], [756183.602817969, 7], [756183.661678809, 10], [756183.717163233, 9], [756183.768971826, 1], [756183.819128218, 6]], "completed": 756183.819256902, "ttft_s": 11.389063577051274, "decode_tps": 40.60574617554668, "prompt_tokens": 65536, "cached_tokens": 63488, "cache_state": "reused", "complete": true, "token_sha": "bf47482e16cb", "token_sha256": "bf47482e16cb91a3294931e7a597aa8938db10afebffe6941e0abe5a20c9d02d", "output_sha256": "506ecff1415d0489be9cdf8709f2dace158282371918e04ed030e81d69827fa7", "elapsed_s": 14.516828419989906, "chunk_offsets_s": [11.389063577051274, 11.470694737974554, 11.530442944960669, 11.592659438960254, 11.660704142064787, 11.730322794057429, 11.796795214992017, 11.864010698976927, 11.929068511002697, 11.988761876011267, 12.049024775042199, 12.107338345958851, 12.161354715004563, 12.227538201026618, 12.283462380059063, 12.345161983044818, 12.40672413399443, 12.470970388036221, 12.533347361022606, 12.59843998705037, 12.668527476023883, 12.737585703027435, 12.808412928017788, 12.877466147998348, 12.946177426027134, 13.01775976200588, 13.087289334041998, 13.155663680052385, 13.225002233986743, 13.2925139850704, 13.35871673899237, 13.420705056050792, 13.480862393975258, 13.534942668979056, 13.59047366399318, 13.652091708034277, 13.710100144031458, 13.771412936039269, 13.829729459015653, 13.882625487982295, 13.937680389964953, 13.998017534031533, 14.05908635002561, 14.119880549958907, 14.181619151961058, 14.244405991048552, 14.300389487063512, 14.35925032699015, 14.4147347509861, 14.466543343965895, 14.516699736006558], "chunk_gaps_ms": {"count": 50, "p50": 61.71910249395296, "p95": 70.82722499035299, "max": 81.63116092327982}}, {"prompt": "p65536-2", "seed": 1002, "sent": 756169.303471615, "first": 756175.972916989, "last": 756179.244077261, "tokens": 128, "pieces": [[756175.972916989, 4], [756176.053721452, 7], [756176.107413703, 7], [756176.160823371, 14], [756176.214684513, 9], [756176.26735051, 16], [756176.323521366, 10], [756176.377394282, 8], [756176.431802541, 4], [756176.486132195, 2], [756176.540012023, 19], [756176.593726289, 10], [756176.647287217, 11], [756176.701206612, 9], [756176.755244467, 4], [756176.811699713, 9], [756176.873309475, 5], [756176.934490852, 12], [756176.996909215, 12], [756177.05706383, 15], [756177.117887798, 16], [756177.181849725, 3], [756177.236955932, 13], [756177.298987044, 14], [756177.358525103, 5], [756177.412331066, 5], [756177.465920875, 14], [756177.521029674, 16], [756177.579786271, 25], [756177.642878628, 10], [756177.702755636, 17], [756177.767851621, 5], [756177.83739693, 9], [756177.902433365, 14], [756177.970547973, 14], [756178.034485201, 4], [756178.088369245, 9], [756178.150517013, 9], [756178.209934552, 17], [756178.272033409, 8], [756178.335567995, 18], [756178.406266406, 23], [756178.468841071, 5], [756178.522375478, 9], [756178.574593465, 8], [756178.628628512, 10], [756178.681868401, 15], [756178.73682148, 17], [756178.798780679, 17], [756178.859567548, 9], [756178.9232768, 14], [756178.984474268, 9], [756179.068205466, 28], [756179.134571241, 19], [756179.193518528, 14], [756179.244077261, 1]], "completed": 756179.244161546, "ttft_s": 6.669445374049246, "decode_tps": 38.82414478098634, "prompt_tokens": 65536, "cached_tokens": 63488, "cache_state": "reused", "complete": true, "token_sha": "30f96d96d0b7", "token_sha256": "30f96d96d0b71ddd72f2ccfbcb05bce868c47cc84c22cbf27d7b26c6a33cc946", "output_sha256": "cb508766a2f4dc5642532a8d981a8d3b8e900dc060b072caa7c1c3e94292cc37", "elapsed_s": 9.940689931041561, "chunk_offsets_s": [6.669445374049246, 6.750249837059528, 6.803942088037729, 6.857351756072603, 6.911212898092344, 6.96387889503967, 7.020049751037732, 7.073922667070292, 7.128330926061608, 7.1826605800306424, 7.2365404079901055, 7.290254674036987, 7.34381560201291, 7.39773499709554, 7.451772851985879, 7.508228098042309, 7.569837860064581, 7.631019237102009, 7.693437600042671, 7.753592215012759, 7.81441618304234, 7.878378110006452, 7.933484317036346, 7.995515429065563, 8.055053488002159, 8.108859451022, 8.162449260009453, 8.217558059026487, 8.276314656017348, 8.339407013030723, 8.399284020997584, 8.464380006073043, 8.53392531501595, 8.598961750045419, 8.667076357989572, 8.731013586046174, 8.784897629986517, 8.847045398084447, 8.906462937011383, 8.968561794026755, 9.03209638001863, 9.10279479098972, 9.165369456051849, 9.218903862987645, 9.27112185000442, 9.325156897073612, 9.378396786050871, 9.433349865023047, 9.495309064048342, 9.556095933075994, 9.619805185007863, 9.68100265308749, 9.764733851072378, 9.831099626026116, 9.89004691306036, 9.940605645999312], "chunk_gaps_ms": {"count": 55, "p50": 59.417538926936686, "p95": 70.6984109710902, "max": 83.73119798488915}}], "rows": [{"name": "p65536-0", "prompt": 65536, "sent": 0.0011736080050468445, "first": 1.4541780079016462, "last": 5.225737044936977, "error": null, "equal_alone": true, "started_after": []}, {"name": "p65536-1", "prompt": 65536, "sent": 0.0027783699333667755, "first": 11.391841946984641, "last": 14.519478105939925, "error": null, "equal_alone": true, "started_after": ["p65536-0", "p65536-2"]}, {"name": "p65536-2", "prompt": 65536, "sent": 0.0038215029053390026, "first": 6.673266876954585, "last": 9.944427148904651, "error": null, "equal_alone": true, "started_after": ["p65536-0"]}]} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/client.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/client.log new file mode 100644 index 0000000000..d05c19c353 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/client.log @@ -0,0 +1,10 @@ +server up 2026-10-23 11:28:30 +fit: 200 pages; 65536-token prompts take 129 each +alone p65536-0: prompt 65536 ttft 44.11991849297192 sha d8a8421597ee17ad5b7801dacb29f07eb6692a742fb9fc92104dd0e7527a4a3d +alone p65536-1: prompt 65536 ttft 43.57638899201993 sha bf47482e16cb91a3294931e7a597aa8938db10afebffe6941e0abe5a20c9d02d +alone p65536-2: prompt 65536 ttft 43.47246499999892 sha 30f96d96d0b71ddd72f2ccfbcb05bce868c47cc84c22cbf27d7b26c6a33cc946 + p65536-0: prompt 65536 first 1.45 s last 5.23 s equal_alone True started after - + p65536-1: prompt 65536 first 11.39 s last 14.52 s equal_alone True started after ['p65536-0', 'p65536-2'] + p65536-2: prompt 65536 first 6.67 s last 9.94 s equal_alone True started after ['p65536-0'] +together == alone: 3/3; 2 request(s) started only after another ended +=== kvp-client r6admit rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/hostwatch-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/hostwatch-rank0.log new file mode 100644 index 0000000000..940e79fc8b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/hostwatch-rank0.log @@ -0,0 +1,199 @@ +11:24:26 114 +11:24:28 112 +11:24:30 112 +11:24:32 112 +11:24:34 110 +11:24:36 110 +11:24:38 109 +11:24:40 108 +11:24:42 108 +11:24:44 107 +11:24:46 107 +11:24:48 106 +11:24:50 105 +11:24:52 105 +11:24:54 104 +11:24:56 104 +11:24:58 103 +11:25:01 102 +11:25:03 103 +11:25:05 102 +11:25:07 101 +11:25:09 100 +11:25:11 100 +11:25:13 100 +11:25:15 99 +11:25:17 98 +11:25:19 98 +11:25:21 97 +11:25:23 97 +11:25:25 97 +11:25:27 95 +11:25:29 96 +11:25:31 95 +11:25:33 94 +11:25:35 94 +11:25:38 92 +11:25:40 93 +11:25:42 92 +11:25:44 91 +11:25:46 91 +11:25:48 90 +11:25:50 90 +11:25:52 89 +11:25:54 88 +11:25:56 88 +11:25:58 87 +11:26:00 87 +11:26:02 86 +11:26:04 85 +11:26:06 85 +11:26:08 83 +11:26:10 83 +11:26:12 82 +11:26:14 82 +11:26:17 82 +11:26:19 82 +11:26:21 81 +11:26:23 81 +11:26:25 79 +11:26:27 79 +11:26:29 77 +11:26:31 77 +11:26:33 78 +11:26:35 76 +11:26:37 76 +11:26:39 75 +11:26:41 75 +11:26:43 75 +11:26:45 75 +11:26:47 74 +11:26:49 74 +11:26:51 72 +11:26:53 72 +11:26:56 72 +11:26:58 71 +11:27:00 71 +11:27:02 71 +11:27:04 70 +11:27:06 70 +11:27:08 68 +11:27:10 68 +11:27:12 67 +11:27:14 67 +11:27:16 67 +11:27:18 65 +11:27:20 65 +11:27:22 64 +11:27:24 64 +11:27:26 64 +11:27:28 63 +11:27:30 63 +11:27:33 63 +11:27:35 61 +11:27:37 61 +11:27:39 60 +11:27:41 60 +11:27:43 60 +11:27:45 58 +11:27:47 58 +11:27:49 57 +11:27:51 56 +11:27:53 56 +11:27:55 56 +11:27:57 55 +11:27:59 55 +11:28:01 54 +11:28:03 54 +11:28:05 53 +11:28:08 52 +11:28:10 50 +11:28:12 50 +11:28:14 50 +11:28:16 50 +11:28:18 50 +11:28:20 50 +11:28:22 50 +11:28:24 50 +11:28:26 11 +11:28:28 11 +11:28:30 11 +11:28:32 11 +11:28:34 11 +11:28:36 11 +11:28:38 11 +11:28:40 11 +11:28:42 11 +11:28:45 11 +11:28:47 12 +11:28:49 12 +11:28:51 12 +11:28:53 12 +11:28:55 12 +11:28:57 12 +11:28:59 12 +11:29:01 12 +11:29:03 12 +11:29:05 12 +11:29:07 12 +11:29:09 12 +11:29:12 12 +11:29:14 12 +11:29:16 12 +11:29:18 12 +11:29:20 12 +11:29:22 12 +11:29:24 12 +11:29:26 12 +11:29:28 12 +11:29:30 12 +11:29:32 12 +11:29:34 12 +11:29:36 12 +11:29:39 12 +11:29:41 12 +11:29:43 12 +11:29:45 12 +11:29:47 12 +11:29:49 12 +11:29:51 12 +11:29:53 12 +11:29:55 12 +11:29:57 12 +11:29:59 12 +11:30:01 12 +11:30:03 12 +11:30:05 12 +11:30:07 12 +11:30:09 12 +11:30:11 12 +11:30:14 12 +11:30:16 12 +11:30:18 12 +11:30:20 12 +11:30:22 12 +11:30:24 12 +11:30:26 12 +11:30:28 12 +11:30:30 12 +11:30:32 12 +11:30:34 12 +11:30:36 12 +11:30:38 12 +11:30:40 12 +11:30:42 12 +11:30:44 12 +11:30:46 12 +11:30:49 12 +11:30:51 12 +11:30:53 12 +11:30:55 12 +11:30:57 12 +11:30:59 12 +11:31:01 12 +11:31:03 12 +11:31:05 12 +11:31:07 12 +11:31:09 12 +11:31:11 12 +11:31:13 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/hostwatch-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/hostwatch-rank1.log new file mode 100644 index 0000000000..358035ca20 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/hostwatch-rank1.log @@ -0,0 +1,198 @@ +11:24:26 114 +11:24:29 112 +11:24:31 112 +11:24:33 111 +11:24:35 110 +11:24:37 110 +11:24:39 110 +11:24:41 109 +11:24:43 109 +11:24:45 107 +11:24:47 107 +11:24:49 107 +11:24:51 105 +11:24:53 105 +11:24:55 104 +11:24:57 104 +11:24:59 103 +11:25:01 103 +11:25:03 103 +11:25:05 101 +11:25:08 101 +11:25:10 101 +11:25:12 99 +11:25:14 99 +11:25:16 98 +11:25:18 98 +11:25:20 97 +11:25:22 97 +11:25:24 97 +11:25:26 95 +11:25:28 95 +11:25:30 95 +11:25:32 94 +11:25:34 94 +11:25:36 94 +11:25:38 93 +11:25:40 93 +11:25:42 91 +11:25:45 91 +11:25:47 90 +11:25:49 89 +11:25:51 89 +11:25:53 88 +11:25:55 88 +11:25:57 88 +11:25:59 87 +11:26:01 87 +11:26:03 85 +11:26:05 85 +11:26:07 85 +11:26:09 84 +11:26:11 84 +11:26:13 82 +11:26:15 83 +11:26:17 82 +11:26:19 81 +11:26:22 81 +11:26:24 80 +11:26:26 80 +11:26:28 78 +11:26:30 78 +11:26:32 78 +11:26:34 77 +11:26:36 77 +11:26:38 76 +11:26:40 76 +11:26:42 76 +11:26:44 74 +11:26:46 74 +11:26:48 73 +11:26:50 73 +11:26:52 73 +11:26:54 71 +11:26:56 71 +11:26:58 71 +11:27:01 70 +11:27:03 70 +11:27:05 70 +11:27:07 68 +11:27:09 68 +11:27:11 68 +11:27:13 67 +11:27:15 67 +11:27:17 65 +11:27:19 65 +11:27:21 64 +11:27:23 63 +11:27:25 63 +11:27:27 64 +11:27:29 63 +11:27:31 62 +11:27:33 61 +11:27:35 61 +11:27:38 61 +11:27:40 59 +11:27:42 59 +11:27:44 59 +11:27:46 59 +11:27:48 58 +11:27:50 58 +11:27:52 56 +11:27:54 56 +11:27:56 56 +11:27:58 55 +11:28:00 55 +11:28:02 55 +11:28:04 54 +11:28:06 54 +11:28:08 54 +11:28:10 53 +11:28:13 50 +11:28:15 50 +11:28:17 50 +11:28:19 50 +11:28:21 50 +11:28:23 50 +11:28:25 24 +11:28:27 12 +11:28:29 12 +11:28:31 12 +11:28:33 12 +11:28:35 12 +11:28:37 12 +11:28:39 12 +11:28:42 12 +11:28:44 12 +11:28:46 12 +11:28:48 12 +11:28:50 12 +11:28:52 12 +11:28:54 12 +11:28:56 12 +11:28:58 12 +11:29:00 12 +11:29:02 12 +11:29:04 12 +11:29:06 12 +11:29:08 12 +11:29:10 12 +11:29:12 12 +11:29:14 12 +11:29:17 12 +11:29:19 12 +11:29:21 12 +11:29:23 12 +11:29:25 12 +11:29:27 12 +11:29:29 12 +11:29:31 12 +11:29:33 12 +11:29:35 12 +11:29:37 12 +11:29:39 12 +11:29:41 12 +11:29:43 12 +11:29:45 12 +11:29:47 12 +11:29:50 12 +11:29:52 12 +11:29:54 12 +11:29:56 12 +11:29:58 12 +11:30:00 12 +11:30:02 12 +11:30:04 12 +11:30:06 12 +11:30:08 12 +11:30:10 12 +11:30:12 12 +11:30:14 12 +11:30:16 12 +11:30:18 12 +11:30:20 12 +11:30:23 12 +11:30:25 12 +11:30:27 12 +11:30:29 12 +11:30:31 12 +11:30:33 12 +11:30:35 12 +11:30:37 12 +11:30:39 12 +11:30:41 12 +11:30:43 12 +11:30:45 12 +11:30:47 12 +11:30:49 12 +11:30:51 12 +11:30:53 12 +11:30:56 12 +11:30:58 12 +11:31:00 12 +11:31:02 12 +11:31:04 12 +11:31:06 12 +11:31:08 12 +11:31:10 12 +11:31:12 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/hostwatch-rank2.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/hostwatch-rank2.log new file mode 100644 index 0000000000..1a74519356 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/hostwatch-rank2.log @@ -0,0 +1,198 @@ +11:24:27 112 +11:24:29 112 +11:24:31 112 +11:24:33 110 +11:24:35 110 +11:24:37 109 +11:24:39 109 +11:24:42 108 +11:24:44 108 +11:24:46 108 +11:24:48 107 +11:24:50 107 +11:24:52 106 +11:24:54 105 +11:24:56 105 +11:24:58 104 +11:25:00 104 +11:25:02 104 +11:25:04 102 +11:25:06 102 +11:25:08 101 +11:25:10 101 +11:25:12 100 +11:25:14 100 +11:25:16 99 +11:25:18 98 +11:25:21 98 +11:25:23 97 +11:25:25 97 +11:25:27 97 +11:25:29 96 +11:25:31 96 +11:25:33 94 +11:25:35 95 +11:25:37 94 +11:25:39 93 +11:25:41 93 +11:25:43 92 +11:25:45 92 +11:25:47 90 +11:25:49 90 +11:25:51 89 +11:25:53 89 +11:25:56 88 +11:25:58 88 +11:26:00 88 +11:26:02 87 +11:26:04 87 +11:26:06 86 +11:26:08 85 +11:26:10 84 +11:26:12 84 +11:26:14 84 +11:26:16 82 +11:26:18 82 +11:26:20 82 +11:26:22 81 +11:26:24 81 +11:26:26 80 +11:26:28 80 +11:26:30 79 +11:26:33 79 +11:26:35 79 +11:26:37 77 +11:26:39 77 +11:26:41 76 +11:26:43 76 +11:26:45 75 +11:26:47 75 +11:26:49 74 +11:26:51 73 +11:26:53 73 +11:26:55 72 +11:26:57 72 +11:26:59 71 +11:27:01 71 +11:27:03 71 +11:27:05 69 +11:27:07 69 +11:27:09 69 +11:27:12 68 +11:27:14 68 +11:27:16 67 +11:27:18 67 +11:27:20 67 +11:27:22 66 +11:27:24 66 +11:27:26 64 +11:27:28 64 +11:27:30 64 +11:27:32 63 +11:27:34 63 +11:27:36 62 +11:27:38 62 +11:27:40 61 +11:27:42 60 +11:27:44 60 +11:27:46 59 +11:27:49 59 +11:27:51 59 +11:27:53 57 +11:27:55 55 +11:27:57 55 +11:27:59 55 +11:28:01 55 +11:28:03 54 +11:28:05 55 +11:28:07 55 +11:28:09 55 +11:28:11 55 +11:28:13 55 +11:28:15 55 +11:28:17 55 +11:28:19 55 +11:28:21 55 +11:28:23 56 +11:28:25 17 +11:28:28 17 +11:28:30 17 +11:28:32 17 +11:28:34 17 +11:28:36 17 +11:28:38 17 +11:28:40 17 +11:28:42 17 +11:28:44 17 +11:28:46 17 +11:28:48 17 +11:28:50 17 +11:28:52 17 +11:28:54 17 +11:28:56 17 +11:28:58 17 +11:29:00 17 +11:29:03 17 +11:29:05 17 +11:29:07 17 +11:29:09 17 +11:29:11 17 +11:29:13 17 +11:29:15 17 +11:29:17 17 +11:29:19 17 +11:29:21 17 +11:29:23 17 +11:29:25 17 +11:29:27 17 +11:29:29 17 +11:29:31 17 +11:29:33 17 +11:29:35 17 +11:29:38 17 +11:29:40 17 +11:29:42 17 +11:29:44 17 +11:29:46 17 +11:29:48 17 +11:29:50 17 +11:29:52 17 +11:29:54 17 +11:29:56 17 +11:29:58 17 +11:30:00 17 +11:30:02 17 +11:30:04 17 +11:30:06 17 +11:30:08 17 +11:30:11 17 +11:30:13 17 +11:30:15 17 +11:30:17 17 +11:30:19 17 +11:30:21 17 +11:30:23 17 +11:30:25 17 +11:30:27 17 +11:30:29 17 +11:30:31 17 +11:30:33 17 +11:30:35 17 +11:30:37 17 +11:30:39 17 +11:30:41 17 +11:30:44 17 +11:30:46 17 +11:30:48 17 +11:30:50 17 +11:30:52 17 +11:30:54 17 +11:30:56 17 +11:30:58 17 +11:31:00 17 +11:31:02 17 +11:31:04 17 +11:31:06 17 +11:31:08 17 +11:31:10 17 +11:31:12 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/server-rank0.log new file mode 100644 index 0000000000..54d8d7d10c --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/server-rank0.log @@ -0,0 +1,12082 @@ +[kvp-serve] 2026-10-23T02:24:17+00:00 rank 0 of 3 context 98304 parallel 8 binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 37.2 +info: rank 0: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-a:10203:10203 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:10203:10203 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:10203:10203 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:10203:10203 [0] NCCL INFO cudaDriverVersion 13030 +host-a:10203:10203 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:10203:10203 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:10203:10203 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:10203:10203 [0] NCCL INFO Count not open libibverbs.so +host-a:10203:10203 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:10203:10203 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:10203:10203 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:10203:10203 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x40ce37b0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:10203:10203 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x40d24eb0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:10203:10203 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x40d665b0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:10203:10203 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x40da7cb0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:10203:10203 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:10203:10203 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:10203:10203 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:10203:10203 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:10203:10203 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:10203:10203 [0] NCCL INFO Initialized NET plugin IB +host-a:10203:10203 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:10203:10203 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:10203:10203 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:10203:10203 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:10203:10203 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:10203:10203 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:10203:10203 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:10203:10203 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:10203:10203 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:10203:10203 [0] NCCL INFO Using network IB +host-a:10203:10203 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:10203:10203 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x3e0ac0f0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x451f7964b8d8ea1f - Init START +host-a:10203:10203 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:10203:10203 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:10203:10203 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:10203:10203 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:10203:10203 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:10203:10203 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:10203:10203 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:10203:10203 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:10203:10203 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:10203:10203 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:10203:10203 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:10203:10203 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:10203:10203 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:10203:10203 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:10203:10203 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:10203:10203 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:10203:10203 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:10203:10203 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:10203:10203 [0] NCCL INFO Rank 0: 6 Net devices +host-a:10203:10203 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:10203:10203 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:10203:10203 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:10203:10203 [0] NCCL INFO comm 0x3e0ac0f0 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:10203:10203 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:10203:10203 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:10203:10203 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:10203:10203 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:10203:10203 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:10203:10203 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:10203:10203 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:10203:10203 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:10203:10203 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:10203:10203 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:10203:10220 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-a:10203:10220 [0] NCCL INFO proxy listening socket at <48007> +host-a:10203:10221 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 4 +host-a:10203:10222 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-a:10203:10203 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:10203:10203 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:10203:10203 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:10203:10203 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:10203:10203 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:10203:10203 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:10203:10203 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:10203:10203 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b90010000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b90054000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b9009c000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6951 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6952 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6951 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b514e ctsFifoLkey=0x1b514e +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6951 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6952 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b514e ctsFifoLkey=0x1b514e +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6952 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b90145000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b9018d000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b901d1000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b90219000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b9025d000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2057 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2058 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2059 pkey=0 pd=0xe65b900f2d40 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6953 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6954 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6953 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6953 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6954 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6954 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1929 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1930 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1931 pkey=0 pd=0xe65b902f19b0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6983 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6984 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6983 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c2525 ctsFifoLkey=0x1c2525 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6983 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6984 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c2525 ctsFifoLkey=0x1c2525 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6984 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2060 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2061 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2062 pkey=0 pd=0xe65b900f2d40 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6951 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6952 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1932 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1933 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1934 pkey=0 pd=0xe65b902f19b0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6985 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6986 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6985 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2c2a ctsFifoLkey=0x1d2c2a +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6985 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6986 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2c2a ctsFifoLkey=0x1d2c2a +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6986 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6953 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6954 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6983 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6984 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6985 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6986 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10203 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:10203:10203 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:10203:10203 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:10203:10203 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:10203:10203 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b905bb000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b905fc000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1935 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1936 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1935 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1935 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1936 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1936 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b90690000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b906d1000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2063 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2064 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2063 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19e2da ctsFifoLkey=0x19e2da +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2063 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2064 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19e2da ctsFifoLkey=0x19e2da +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2064 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1935 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1936 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2063 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2064 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1937 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1938 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1937 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x1811ce ctsFifoLkey=0x1811ce +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1937 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1938 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x1811ce ctsFifoLkey=0x1811ce +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1938 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2065 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2066 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2065 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c0bd ctsFifoLkey=0x19c0bd +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2065 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2066 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c0bd ctsFifoLkey=0x19c0bd +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2066 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1937 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1938 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2065 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2066 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10203 [0] NCCL INFO Connected all trees +host-a:10203:10203 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:10203:10203 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:10203:10203 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:10203:10203 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b90822000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b90867000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b908ac000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b908f4000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6955 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6956 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6957 pkey=0 pd=0xe65b90053590 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6987 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6988 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6989 pkey=0 pd=0xe65b90362cd0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6958 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6959 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6960 pkey=0 pd=0xe65b90053590 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6990 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6991 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6992 pkey=0 pd=0xe65b90362cd0 +host-a:10203:10203 [0] NCCL INFO Connected binomial trees +host-a:10203:10203 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:10203:10203 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:10203:10203 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:10203:10203 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:10203:10203 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-a:10203:10203 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:10203:10203 [0] NCCL INFO ncclCommInitRank comm 0x3e0ac0f0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x451f7964b8d8ea1f - Init COMPLETE +host-a:10203:10203 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 2.58 (kernels 0.24, alloc 0.03, bootstrap 2.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.29, rest 0.00) +host-a:10203:10203 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:10203:10203 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:10203:10203 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:10203:10203 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:10203:10203 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:10203:10203 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:10203:10203 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:10203:10203 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b90b1a000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b90b5b000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b90ba0000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6961 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6962 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6961 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1bb4b2 ctsFifoLkey=0x1bb4b2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6961 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6962 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1bb4b2 ctsFifoLkey=0x1bb4b2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6962 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b90c3f000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b90c87000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b90ccb000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe65b90d13000) +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe65b90d57000) +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6993 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6994 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6993 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d6b64 ctsFifoLkey=0x1d6b64 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6993 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6994 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d6b64 ctsFifoLkey=0x1d6b64 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6994 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2067 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2068 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2069 pkey=0 pd=0xe65b900f2d40 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6961 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6962 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1939 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1940 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1941 pkey=0 pd=0xe65b902f19b0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6993 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6994 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2070 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2071 pkey=0 pd=0xe65b900f2d40 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2070 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2070 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2071 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2071 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6995 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6996 pkey=0 pd=0xe65b90362cd0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6997 pkey=0 pd=0xe65b90362cd0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1942 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1943 pkey=0 pd=0xe65b902f19b0 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1942 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1942 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1943 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1943 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2070 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2071 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routinginfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 64.80 GiB; 8 streams at once, the engine's own and 7 more taking 200 pages of 512 rows (1.25 GiB) as they need them (enough for one at a 98304-token window), 131 MiB fixed each, of 38.0 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 35.9 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 98304; loaded in 248.4s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 1 states, 922 MiB and 35802 MiB spare of 36724 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 200 + store 0 == total 200 +[tensorfold] done chatcmpl-e75beaf03b479b95b470d54f3882b470 prompt=65536 cached=0 thinking=False effort=none tokens=128 sha=d8a8421597ee finish=length rounds=64 accepted=63/108 ttft=44.105 decode=3.862 elapsed=47.967 queued=0.000 tok/s=32.9 seed=1000 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 2 states, 1844 MiB and 34880 MiB spare of 36724 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 200 + store 0 == total 200 +[tensorfold] done chatcmpl-a853e9633d3fbe63b8775bfb32a3bc29 prompt=65536 cached=0 thinking=False effort=none tokens=128 sha=bf47482e16cb finish=length rounds=50 accepted=77/108 ttft=43.573 decode=3.119 elapsed=46.692 queued=0.000 tok/s=40.7 seed=1001 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 3 states, 2766 MiB and 33958 MiB spare of 36724 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 200 + store 0 == total 200 +[tensorfold] done chatcmpl-93bdf48dc7aa7278190fa327083d6a6f prompt=65536 cached=0 thinking=False effort=none tokens=128 sha=30f96d96d0b7 finish=length rounds=55 accepted=73/98 ttft=43.468 decode=3.274 elapsed=46.742 queued=0.000 tok/s=38.8 seed=1002 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 63488, kept 0; 3 states, 2766 MiB and 33958 MiB spare of 36724 MiB (hits 1, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 200 + store 0 == total 200 +[tensorfold] done chatcmpl-39db3f92738dafce975f22eef1615c47 prompt=65536 cached=63488 thinking=False effort=none tokens=128 sha=d8a8421597ee finish=length rounds=64 accepted=63/108 ttft=1.450 decode=3.772 elapsed=5.221 queued=0.001 tok/s=33.7 seed=1000 +info: prompt cache: 65536 tokens, resumed at 63488, kept 0; 3 states, 2766 MiB and 33958 MiB spare of 36724 MiB (hits 2, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 200 + store 0 == total 200 +[tensorfold] done chatcmpl-89997f2d85949d9d45d02c3a8cb32d68 prompt=65536 cached=63488 thinking=False effort=none tokens=128 sha=30f96d96d0b7 finish=length rounds=55 accepted=73/98 ttft=6.668 decode=3.271 elapsed=9.939 queued=0.000 tok/s=38.8 seed=1002 +info: prompt cache: 65536 tokens, resumed at 63488, kept 0; 3 states, 2766 MiB and 33958 MiB spare of 36724 MiB (hits 3, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 200 + store 0 == total 200 +[tensorfold] done chatcmpl-ab0b6c1b05d73831dc07094ce2106e0a prompt=65536 cached=63488 thinking=False effort=none tokens=128 sha=bf47482e16cb finish=length rounds=50 accepted=77/108 ttft=11.386 decode=3.128 elapsed=14.514 queued=5.223 tok/s=40.6 seed=1001 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1942 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1943 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:10203:10220 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6963 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6964 pkey=0 pd=0xe65b90053590 oooRq=0 +host-a:10203:10220 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6965 pkey=0 pd=0xe65b90053590 +host-a:10203:10203 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/server-rank1.log new file mode 100644 index 0000000000..0821877ee3 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/server-rank1.log @@ -0,0 +1,3184 @@ +[kvp-serve] 2026-10-23T02:24:16+00:00 rank 1 of 3 context 98304 parallel 8 binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 37.2 +info: rank 1: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-b:8956:8956 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:8956:8956 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:8956:8956 [0] NCCL INFO cudaDriverVersion 13030 +host-b:8956:8956 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:8956:8956 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:8956:8956 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:8956:8956 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:8956:8956 [0] NCCL INFO Count not open libibverbs.so +host-b:8956:8956 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:8956:8956 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:8956:8956 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:8956:8956 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x1827f430 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:8956:8956 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x182c0b30 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:8956:8956 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x18302230 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:8956:8956 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x18343930 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:8956:8956 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:8956:8956 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:8956:8956 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:8956:8956 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:8956:8956 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:8956:8956 [0] NCCL INFO Initialized NET plugin IB +host-b:8956:8956 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:8956:8956 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:8956:8956 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:8956:8956 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:8956:8956 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:8956:8956 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:8956:8956 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:8956:8956 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:8956:8956 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:8956:8956 [0] NCCL INFO Using network IB +host-b:8956:8956 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:8956:8956 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x15647db0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x451f7964b8d8ea1f - Init START +host-b:8956:8956 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:8956:8956 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:8956:8956 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:8956:8956 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:8956:8956 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:8956:8956 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:8956:8956 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:8956:8956 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:8956:8956 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:8956:8956 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:8956:8956 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:8956:8956 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:8956:8956 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:8956:8956 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:8956:8956 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:8956:8956 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:8956:8956 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:8956:8956 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:8956:8956 [0] NCCL INFO Rank 1: 6 Net devices +host-b:8956:8956 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:8956:8956 [0] NCCL INFO comm 0x15647db0 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:8956:8956 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:8956:8956 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:8956:8956 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:8956:8956 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:8956:8956 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:8956:8956 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:8956:8972 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-b:8956:8971 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-b:8956:8971 [0] NCCL INFO proxy listening socket at <53137> +host-b:8956:8973 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 11 +host-b:8956:8956 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:8956:8956 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:8956:8956 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:8956:8956 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:8956:8956 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:8956:8956 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:8956:8956 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:8956:8956 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e74010000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e74054000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2185 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2186 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2185 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2185 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2186 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2186 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e74101000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6729 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6730 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6731 pkey=0 pd=0xe08e74053e50 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2313 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2314 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2313 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de9e6 ctsFifoLkey=0x1de9e6 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2313 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2314 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de9e6 ctsFifoLkey=0x1de9e6 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2314 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e7422e000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e74276000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e742ba000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e74302000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e74346000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6732 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6733 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6734 pkey=0 pd=0xe08e74053e50 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2185 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2186 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2313 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2314 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2187 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2188 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2187 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b7977 ctsFifoLkey=0x1b7977 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2187 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2188 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b7977 ctsFifoLkey=0x1b7977 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2188 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6761 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6762 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6763 pkey=0 pd=0xe08e740df390 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6764 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6765 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6766 pkey=0 pd=0xe08e740df390 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2187 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2188 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2315 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2316 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2315 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1dcecc ctsFifoLkey=0x1dcecc +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2315 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2316 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1dcecc ctsFifoLkey=0x1dcecc +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2316 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2315 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2316 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8956 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:8956:8956 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:8956:8956 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:8956:8956 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:8956:8956 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e745b0000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e745f5000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e7463a000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e7467f000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2189 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2190 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2191 pkey=0 pd=0xe08e7400fec0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2317 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2318 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2319 pkey=0 pd=0xe08e740aef00 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2192 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2193 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2194 pkey=0 pd=0xe08e7400fec0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2320 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2321 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2322 pkey=0 pd=0xe08e740aef00 +host-b:8956:8956 [0] NCCL INFO Connected all trees +host-b:8956:8956 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:8956:8956 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:8956:8956 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:8956:8956 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e748a7000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e748e8000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6735 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6736 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6735 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6735 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6736 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6736 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6767 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6768 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6767 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x163e3b ctsFifoLkey=0x163e3b +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6767 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6768 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x163e3b ctsFifoLkey=0x163e3b +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6768 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e749df000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e74a23000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6735 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6736 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6737 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6738 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6737 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17d592 ctsFifoLkey=0x17d592 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6737 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6738 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17d592 ctsFifoLkey=0x17d592 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6738 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6769 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6770 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6769 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6769 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6770 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6770 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6767 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6768 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6737 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6738 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6769 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6770 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8956 [0] NCCL INFO Connected binomial trees +host-b:8956:8956 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:8956:8956 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:8956:8956 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:8956:8956 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:8956:8956 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-b:8956:8956 [0] NCCL INFO ncclCommInitRank comm 0x15647db0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x451f7964b8d8ea1f - Init COMPLETE +host-b:8956:8956 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 0.64 (kernels 0.25, alloc 0.05, bootstrap 0.04, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.29, rest 0.00) +host-b:8956:8956 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:8956:8956 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:8956:8956 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:8956:8956 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:8956:8956 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:8956:8956 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:8956:8956 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:8956:8956 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e74b1b000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e74b5c000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e74ba1000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e74be5000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2195 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2196 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2195 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2195 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2196 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2196 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2323 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2324 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2323 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1deee4 ctsFifoLkey=0x1deee4 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2323 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2324 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1deee4 ctsFifoLkey=0x1deee4 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2324 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e74ce1000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6739 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6740 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6741 pkey=0 pd=0xe08e74053e50 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6771 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6772 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6773 pkey=0 pd=0xe08e740df390 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6774 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6775 pkey=0 pd=0xe08e740df390 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6774 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19e6e5 ctsFifoLkey=0x19e6e5 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6774 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6775 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19e6e5 ctsFifoLkey=0x19e6e5 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6775 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e74e74000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2195 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2196 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2323 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2324 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6774 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6775 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2325 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2326 pkey=0 pd=0xe08e740aef00 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2327 pkey=0 pd=0xe08e740aef00 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe08e74f2a000) +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe08e74f6e000) +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6742 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6743 pkey=0 pd=0xe08e74053e50 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6742 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17ca87 ctsFifoLkey=0x17ca87 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6742 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6743 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17ca87 ctsFifoLkey=0x17ca87 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6743 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2197 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2198 pkey=0 pd=0xe08e7400fec0 oooRq=0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2199 pkey=0 pd=0xe08e740info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 1: rank 0 ended; leaving +0fec0 +host-b:8956:8971 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6742 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8971 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6743 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:8956:8956 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/server-rank2.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/server-rank2.log new file mode 100644 index 0000000000..d2e3868738 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6admit/server-rank2.log @@ -0,0 +1,6715 @@ +[kvp-serve] 2026-10-23T02:24:15+00:00 rank 2 of 3 context 98304 parallel 8 binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=all TF_GLM_MM=unset args --prompt-cache-gib 37.2 +info: rank 2: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-c:8896:8896 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:8896:8896 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:8896:8896 [0] NCCL INFO cudaDriverVersion 13030 +host-c:8896:8896 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:8896:8896 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:8896:8896 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:8896:8896 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:8896:8896 [0] NCCL INFO Count not open libibverbs.so +host-c:8896:8896 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:8896:8896 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:8896:8896 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:8896:8896 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x2e2ab430 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:8896:8896 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x2e2ecb30 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:8896:8896 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x2e32e230 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:8896:8896 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x2e36f930 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:8896:8896 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:8896:8896 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:8896:8896 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:8896:8896 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:8896:8896 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:8896:8896 [0] NCCL INFO Initialized NET plugin IB +host-c:8896:8896 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:8896:8896 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:8896:8896 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:8896:8896 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:8896:8896 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:8896:8896 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:8896:8896 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:8896:8896 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:8896:8896 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:8896:8896 [0] NCCL INFO Using network IB +host-c:8896:8896 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:8896:8896 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x2b673db0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x451f7964b8d8ea1f - Init START +host-c:8896:8896 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:8896:8896 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:8896:8896 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:8896:8896 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:8896:8896 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:8896:8896 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:8896:8896 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:8896:8896 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:8896:8896 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:8896:8896 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:8896:8896 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:8896:8896 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:8896:8896 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:8896:8896 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:8896:8896 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:8896:8896 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:8896:8896 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:8896:8896 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:8896:8896 [0] NCCL INFO Rank 2: 6 Net devices +host-c:8896:8896 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:8896:8896 [0] NCCL INFO comm 0x2b673db0 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:8896:8896 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:8896:8896 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:8896:8896 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:8896:8896 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:8896:8896 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:8896:8896 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:8896:8911 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-c:8896:8911 [0] NCCL INFO proxy listening socket at <44813> +host-c:8896:8912 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-c:8896:8913 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-c:8896:8896 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:8896:8896 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:8896:8896 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:8896:8896 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:8896:8896 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:8896:8896 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:8896:8896 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:8896:8896 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a0010000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a0054000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a009c000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a00e0000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a0128000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a016c000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a01b4000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a01f8000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2313 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2314 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2313 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2313 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2314 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2314 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2185 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2186 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2185 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcfcc ctsFifoLkey=0x1bcfcc +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2185 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2186 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcfcc ctsFifoLkey=0x1bcfcc +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2186 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2315 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2316 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2315 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3f41 ctsFifoLkey=0x1d3f41 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2315 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2316 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3f41 ctsFifoLkey=0x1d3f41 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2316 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2187 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2188 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2187 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2187 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2188 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2188 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1929 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1930 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1931 pkey=0 pd=0xfcf8a031a220 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2313 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2314 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2057 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2058 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2059 pkey=0 pd=0xfcf8a040cf60 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2185 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2186 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2315 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2316 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2187 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2188 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1932 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1933 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1934 pkey=0 pd=0xfcf8a031a220 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2060 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2061 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2062 pkey=0 pd=0xfcf8a040cf60 +host-c:8896:8896 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:8896:8896 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:8896:8896 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:8896:8896 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:8896:8896 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:8896:8896 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:8896:8896 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:8896:8896 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:8896:8896 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a05aa000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a05eb000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a0633000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1935 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1936 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1935 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1935 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1936 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1936 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2063 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2064 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2063 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2063 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2064 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2064 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a072d000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2189 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2190 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2191 pkey=0 pd=0xfcf8a009bf70 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2317 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2318 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2319 pkey=0 pd=0xfcf8a0053eb0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a0867000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1935 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1936 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2063 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2064 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1937 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1938 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1937 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1937 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1938 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1938 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a08fe000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a0946000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a098a000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2192 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2193 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2194 pkey=0 pd=0xfcf8a009bf70 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2065 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2066 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2065 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2065 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2066 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2066 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2320 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2321 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2322 pkey=0 pd=0xfcf8a0053eb0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1937 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1938 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2065 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2066 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8896 [0] NCCL INFO Connected all trees +host-c:8896:8896 [0] NCCL INFO Connected binomial trees +host-c:8896:8896 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:8896:8896 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:8896:8896 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:8896:8896 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:8896:8896 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-c:8896:8896 [0] NCCL INFO ncclCommInitRank comm 0x2b673db0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x451f7964b8d8ea1f - Init COMPLETE +host-c:8896:8896 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.52 (kernels 0.24, alloc 0.03, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.23, rest 0.00) +host-c:8896:8896 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:8896:8896 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:8896:8896 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:8896:8896 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:8896:8896 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:8896:8896 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:8896:8896 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:8896:8896 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a0b1d000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a0b5e000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a0ba3000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a0be4000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a0c2c000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a0c70000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfcf8a0cb8000) +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfcf8a0cfc000) +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1939 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1940 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1941 pkey=0 pd=0xfcf8a031a220 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2323 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2324 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2323 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5f4c ctsFifoLkey=0x1d5f4c +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2323 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2324 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5f4c ctsFifoLkey=0x1d5f4c +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2324 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2067 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2068 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2069 pkey=0 pd=0xfcf8a040cf60 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2195 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2196 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2195 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcecb ctsFifoLkey=0x1bcecb +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2195 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2196 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcecb ctsFifoLkey=0x1bcecb +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2196 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2323 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2324 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2070 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2071 pkey=0 pd=0xfcf8a040cf60 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2070 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19a6a4 ctsFifoLkey=0x19a6a4 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2070 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2071 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19a6a4 ctsFifoLkey=0x19a6a4 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2071 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2195 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2196 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2325 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2326 pkey=0 pd=0xfcf8a0053eb0 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2327 pkey=0 pd=0xfcf8a0053eb0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2197 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2198 pkey=0 pd=0xfcf8a009bf70 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2199 pkey=0 pd=0xfcf8a009bf70 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2070 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2071 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1942 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1943 pkey=0 pd=0xfcf8a031a220 oooRq=0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1942 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17cb89 ctsFifoLkey=0x17cb89 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1942 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1943 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17cb89 ctsFifoLkey=0x17cb89 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1943 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subninfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 2: rank 0 ended; leaving +et-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1942 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8911 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1943 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:8896:8896 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/client.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/client.log new file mode 100644 index 0000000000..6d68d2b0bd --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/client.log @@ -0,0 +1,19 @@ +server up 2026-10-23 10:53:19 +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 48.4, "per_stream_tps": 48.4, "ttft_s_max": 0.11, "timing_valid": true, "end_to_end_tps": 47.543664963237774, "makespan_s": 5.384523893939331, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 59.2, "per_stream_tps": 32.1, "ttft_s_max": 0.27, "timing_valid": true, "end_to_end_tps": 58.710111694637405, "makespan_s": 8.72081461304333, "steady_tps": 61.5, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 77.7, "per_stream_tps": 20.6, "ttft_s_max": 0.68, "timing_valid": true, "end_to_end_tps": 77.36691730463205, "makespan_s": 13.235631400020793, "steady_tps": 80.2, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 95.9, "per_stream_tps": 13.0, "ttft_s_max": 1.76, "timing_valid": true, "end_to_end_tps": 95.73566142458608, "makespan_s": 21.392237432999536, "steady_tps": 103.2, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 35.0, "per_stream_tps": 35.0, "ttft_s_max": 0.16, "timing_valid": true, "end_to_end_tps": 34.401383436275324, "makespan_s": 7.4415611940203235, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 53.2, "per_stream_tps": 27.8, "ttft_s_max": 0.38, "timing_valid": true, "end_to_end_tps": 52.56448534246739, "makespan_s": 9.740416873944923, "steady_tps": 56.0, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 68.5, "per_stream_tps": 18.1, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 68.04737329986936, "makespan_s": 15.048339860048145, "steady_tps": 73.5, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 85.1, "per_stream_tps": 11.4, "ttft_s_max": 2.15, "timing_valid": true, "end_to_end_tps": 84.88056809998638, "makespan_s": 24.12801947304979, "steady_tps": 91.5, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 70.4, "per_stream_tps": 70.4, "ttft_s_max": 0.11, "timing_valid": true, "end_to_end_tps": 68.62781851106324, "makespan_s": 3.7302657370455563, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 81.7, "per_stream_tps": 41.6, "ttft_s_max": 0.27, "timing_valid": true, "end_to_end_tps": 80.65530033383348, "makespan_s": 6.348001902922988, "steady_tps": 83.5, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 115.1, "per_stream_tps": 30.0, "ttft_s_max": 0.68, "timing_valid": true, "end_to_end_tps": 114.07378265210859, "makespan_s": 8.976646309019998, "steady_tps": 121.0, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 151.4, "per_stream_tps": 20.4, "ttft_s_max": 1.69, "timing_valid": true, "end_to_end_tps": 150.75340645758675, "makespan_s": 13.585099322954193, "steady_tps": 165.8, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 34.9, "per_stream_tps": 34.9, "ttft_s_max": 0.16, "timing_valid": true, "end_to_end_tps": 34.32441879581066, "makespan_s": 7.458247189060785, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 50.4, "per_stream_tps": 25.5, "ttft_s_max": 0.38, "timing_valid": true, "end_to_end_tps": 49.79673082902159, "makespan_s": 10.281799457035959, "steady_tps": 51.2, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 69.4, "per_stream_tps": 17.9, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 68.91814349041903, "makespan_s": 14.858206390053965, "steady_tps": 71.8, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 87.8, "per_stream_tps": 11.6, "ttft_s_max": 2.09, "timing_valid": true, "end_to_end_tps": 87.50031645235602, "makespan_s": 23.405629636952654, "steady_tps": 92.5, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +logs/r6all-conc.json: 16 cells; alone 60/60 equal (0 unequal, 0 failed); serial 72/72 equal (0 unequal, 0 failed) +=== kvp-client r6all rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/conc.json b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/conc.json new file mode 100644 index 0000000000..5e344ed72e --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/conc.json @@ -0,0 +1,9511 @@ +{ + "label": "", + "tokens": 256, + "mixed": false, + "cells": [ + { + "prompt": "code", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.4, + "per_stream_tps": 48.4, + "ttft_s_max": 0.11, + "timing_valid": true, + "end_to_end_tps": 47.543664963237774, + "makespan_s": 5.384523893939331, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.35063832845031, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 5.384523893939331, + "end_to_end_tps": 47.543664963237774, + "timing_valid": true, + "per_stream_tps": [ + 48.4 + ], + "ttft_s": [ + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + } + ], + "client_intervals": [ + { + "start": 754056.516600501, + "first": 754056.626866763, + "last": 754061.900840568, + "end": 754061.901124395, + "elapsed_s": 5.384523893939331, + "chunk_offsets_s": [ + 0.11026626196689904, + 0.16964860795997083, + 0.22678427794016898, + 0.2872410980053246, + 0.35272972600068897, + 0.4204733340302482, + 0.48378850100561976, + 0.5478598679183051, + 0.6125494160223752, + 0.6791528529720381, + 0.7458794759586453, + 0.814053408917971, + 0.8818078979384154, + 0.948220997932367, + 1.0136469840072095, + 1.0755830989219248, + 1.1328084169654176, + 1.1873176459921524, + 1.238935424014926, + 1.293243914959021, + 1.3530400570016354, + 1.414598967996426, + 1.4743444219930097, + 1.5336230869870633, + 1.5946180409519002, + 1.662678339984268, + 1.7322404609294608, + 1.8001809839624912, + 1.8641600449336693, + 1.9314483209745958, + 1.9967551230220124, + 2.0626050329301506, + 2.1277951939264312, + 2.194596410030499, + 2.2632117699831724, + 2.331865979009308, + 2.4001683769747615, + 2.4683811409631744, + 2.5378494859905913, + 2.6071315249428153, + 2.6758891260251403, + 2.7428095269715413, + 2.8106246719835326, + 2.880641645984724, + 2.949040652019903, + 3.0185182290151715, + 3.0872321979841217, + 3.1554746750043705, + 3.2250771010294557, + 3.294413267984055, + 3.362895507016219, + 3.4321194170042872, + 3.5001744999317452, + 3.5667540340218693, + 3.6335406409343705, + 3.7024424680275843, + 3.766784972976893, + 3.826008021016605, + 3.885928132920526, + 3.9500782350078225, + 4.014602661016397, + 4.079342929995619, + 4.168555489974096, + 4.2395112089579925, + 4.307312289020047, + 4.3753182999789715, + 4.441022385028191, + 4.505294280941598, + 4.594530089991167, + 4.660890900995582, + 4.726973645971157, + 4.794059696025215, + 4.861352355917916, + 4.92810673895292, + 4.998503956012428, + 5.06538574793376, + 5.134277686942369, + 5.203914304031059, + 5.271342277992517, + 5.335884336032905, + 5.384240066981874 + ], + "chunk_gaps_ms": { + "count": 80, + "p50": 66.79391150828451, + "p95": 70.01697400119156, + "max": 89.23580904956907 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.2, + "per_stream_tps": 32.1, + "ttft_s_max": 0.27, + "timing_valid": true, + "end_to_end_tps": 58.710111694637405, + "makespan_s": 8.72081461304333, + "steady_tps": 61.5, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 59.23720517876557, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 8.72081461304333, + "end_to_end_tps": 58.710111694637405, + "timing_valid": true, + "steady_tps": 61.46377036255482, + "per_stream_tps": [ + 34.6, + 29.6 + ], + "ttft_s": [ + 0.27, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + } + ], + "client_intervals": [ + { + "start": 754061.902563672, + "first": 754062.177056538, + "last": 754069.539177362, + "end": 754069.539338963, + "elapsed_s": 7.636775290942751, + "chunk_offsets_s": [ + 0.2744928660104051, + 0.3549917449709028, + 0.4487848919816315, + 0.5432068769587204, + 0.6375172609696165, + 0.7297361209057271, + 0.8185860859230161, + 0.9109864518977702, + 1.0017230269731954, + 1.099111135932617, + 1.1944665369810537, + 1.2896074888994917, + 1.3856244969647378, + 1.4804304050048813, + 1.5760951539268717, + 1.6672144999029115, + 1.758559511974454, + 1.849690473987721, + 1.9433584359940141, + 2.0388880629325286, + 2.1354308519512415, + 2.231553572928533, + 2.326550186960958, + 2.422536921920255, + 2.522902241908014, + 2.619074148009531, + 2.7129495989065617, + 2.8092528179986402, + 2.9033585909055546, + 2.997558605973609, + 3.09300411993172, + 3.1898896319326013, + 3.289567297906615, + 3.3876148998970166, + 3.48375810100697, + 3.5786551779601723, + 3.674053572001867, + 3.7719458770006895, + 3.8670767159201205, + 3.9625642149476334, + 4.060500024002977, + 4.15586345794145, + 4.252089426969178, + 4.351550274994224, + 4.450737279956229, + 4.550871093990281, + 4.651003626990132, + 4.750796941923909, + 4.846470505930483, + 4.94250133796595, + 5.0437405538978055, + 5.141262696008198, + 5.236928755999543, + 5.332057546940632, + 5.428234282997437, + 5.5247170069487765, + 5.622357101994567, + 5.7157201959053054, + 5.812034069909714, + 5.909127407940105, + 6.0096099929651245, + 6.1080778709147125, + 6.206418323912658, + 6.300874355947599, + 6.395435397978872, + 6.496520372922532, + 6.592773525975645, + 6.687693114974536, + 6.783891739905812, + 6.880192157928832, + 6.976431294926442, + 7.073605912970379, + 7.174806920927949, + 7.276114408974536, + 7.371216063969769, + 7.466543079935946, + 7.556537966942415, + 7.636613689945079 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 95.98673495929688, + "p95": 101.0849749436602, + "max": 101.30748804658651 + } + }, + { + "start": 754061.902253413, + "first": 754062.013453779, + "last": 754070.622907896, + "end": 754070.623068026, + "elapsed_s": 8.72081461304333, + "chunk_offsets_s": [ + 0.11120036605279893, + 0.1684573320671916, + 0.35513229004573077, + 0.44886863697320223, + 0.5434702549828216, + 0.6377394549781457, + 0.729820266016759, + 0.8188433529576287, + 0.91124295105692, + 1.0019732860382646, + 1.0993681619875133, + 1.1947235480183735, + 1.28986229095608, + 1.3858835550490767, + 1.4806879600510001, + 1.5765745979733765, + 1.6675955270184204, + 1.7590694529935718, + 1.8501703990623355, + 1.9435709019890055, + 2.0392604030203074, + 2.135691973962821, + 2.2318152230000123, + 2.3270645439624786, + 2.422799692954868, + 2.523166853003204, + 2.619455239037052, + 2.7132013300433755, + 2.809514708002098, + 2.903717683046125, + 2.9979167219717056, + 3.0932669549947605, + 3.1901514750206843, + 3.289812979986891, + 3.3878768710419536, + 3.4840104080503806, + 3.579014813993126, + 3.6744123590178788, + 3.772206168039702, + 3.867319855024107, + 3.9631009720033035, + 4.060700729954988, + 4.156118644052185, + 4.25235003803391, + 4.352062296005897, + 4.451224548043683, + 4.551379497977905, + 4.651481727953069, + 4.751056769047864, + 4.84696121502202, + 4.942745100008324, + 5.043836763012223, + 5.141384872957133, + 5.23701558902394, + 5.332416958059184, + 5.428492862032726, + 5.52507717104163, + 5.622549055959098, + 5.715983271016739, + 5.8122958650346845, + 5.9094864670187235, + 6.00996881199535, + 6.108453330001794, + 6.206471235956997, + 6.300978854065761, + 6.395570407039486, + 6.496886551962234, + 6.592903126962483, + 6.687821835977957, + 6.784036188968457, + 6.880674513988197, + 6.976949188043363, + 7.074130845954642, + 7.175282540963963, + 7.276201114058495, + 7.37169088399969, + 7.466659864992835, + 7.556801122031175, + 7.636741835041903, + 7.702603793004528, + 7.759142640046775, + 7.811625913018361, + 7.864942201995291, + 7.9243974450509995, + 7.982095567043871, + 8.041245701955631, + 8.09227553801611, + 8.145762979984283, + 8.202907448983751, + 8.260911318007857, + 8.323707054951228, + 8.383354202960618, + 8.439578518969938, + 8.50376214098651, + 8.588466499000788, + 8.662899640970863, + 8.72065448295325 + ], + "chunk_gaps_ms": { + "count": 96, + "p50": 95.3994570299983, + "p95": 100.9185730945319, + "max": 186.67495797853917 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 77.7, + "per_stream_tps": 20.6, + "ttft_s_max": 0.68, + "timing_valid": true, + "end_to_end_tps": 77.36691730463205, + "makespan_s": 13.235631400020793, + "steady_tps": 80.2, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 77.71879883479339, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 13.235631400020793, + "end_to_end_tps": 77.36691730463205, + "timing_valid": true, + "steady_tps": 80.1876644083498, + "per_stream_tps": [ + 23.1, + 20.0, + 20.9, + 20.4 + ], + "ttft_s": [ + 0.28, + 0.47, + 0.68, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0b4d014c8872", + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b82191a9a6ec", + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a" + } + ], + "client_intervals": [ + { + "start": 754070.625572753, + "first": 754070.902624023, + "last": 754081.961486919, + "end": 754081.961618809, + "elapsed_s": 11.336046056007035, + "chunk_offsets_s": [ + 0.27705127000808716, + 0.36114499799441546, + 0.574091910966672, + 0.8112894659861922, + 0.9449802639428526, + 1.080271941027604, + 1.2086230090353638, + 1.334044417948462, + 1.4671819949289784, + 1.6089819399639964, + 1.7439931660192087, + 1.882972268969752, + 2.0235475469380617, + 2.1659797709435225, + 2.309127712971531, + 2.439456191030331, + 2.5724674789234996, + 2.702565570943989, + 2.838821287965402, + 2.9791474279481918, + 3.1186265199212357, + 3.260577905923128, + 3.404757265932858, + 3.548326250980608, + 3.691530256997794, + 3.838875069981441, + 3.9803844690322876, + 4.12045375793241, + 4.264254504931159, + 4.403679467970505, + 4.54587760893628, + 4.69055170903448, + 4.835693028988317, + 4.977850929019041, + 5.122958680964075, + 5.271770354942419, + 5.415378812933341, + 5.556248908978887, + 5.700592732988298, + 5.845377938007005, + 5.983432999928482, + 6.12616069696378, + 6.278307602973655, + 6.42390199797228, + 6.5657172709470615, + 6.716975255985744, + 6.865627152030356, + 7.015395186957903, + 7.16181251895614, + 7.305995606002398, + 7.449186378973536, + 7.5850134359207, + 7.729274234967306, + 7.874275841983035, + 8.015691654989496, + 8.156656374922022, + 8.303070890950039, + 8.443140354938805, + 8.58494404295925, + 8.729847823968157, + 8.878575160983019, + 9.02218361897394, + 9.173072863020934, + 9.31674444093369, + 9.459243288030848, + 9.607211753027514, + 9.751190965995193, + 9.895316787995398, + 10.038790778955445, + 10.185537265962921, + 10.334848063997924, + 10.48557802697178, + 10.62934205296915, + 10.777982173021883, + 10.924985509016551, + 11.068000647937879, + 11.20225653797388, + 11.335914165945724 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 143.20400601718575, + "p95": 151.25798503868282, + "max": 237.19755501952022 + } + }, + { + "start": 754070.625924356, + "first": 754071.095661631, + "last": 754083.860654547, + "end": 754083.860796853, + "elapsed_s": 13.234872496919706, + "chunk_offsets_s": [ + 0.4697372749214992, + 0.5739773499080911, + 0.8118889119941741, + 0.9451353529002517, + 1.0802271079737693, + 1.2088107229210436, + 1.334435861906968, + 1.467313323984854, + 1.6084640629123896, + 1.7442125449888408, + 1.8829265889944509, + 2.023692636983469, + 2.1660298199858516, + 2.309502019896172, + 2.4396057130070403, + 2.5726205359678715, + 2.7025001778965816, + 2.838647830998525, + 2.97924267698545, + 3.1187689210055396, + 3.260481217992492, + 3.4047647858969867, + 3.5483912599738687, + 3.691235470934771, + 3.8385798519011587, + 3.980191506911069, + 4.119855575961992, + 4.264056647894904, + 4.403424346004613, + 4.545919129974209, + 4.690093720913865, + 4.835578243946657, + 4.977726863930002, + 5.122867928002961, + 5.271637905971147, + 5.415240427944809, + 5.555835976963863, + 5.700178458006121, + 5.845492130960338, + 5.983294582925737, + 6.126200873986818, + 6.277867438970134, + 6.42370777297765, + 6.565506197977811, + 6.716707637999207, + 6.865320029901341, + 7.0151611369801685, + 7.161673030001111, + 7.305707970983349, + 7.448406035895459, + 7.584991468000226, + 7.7285974129335955, + 7.8741416649427265, + 8.015761367976665, + 8.156568486941978, + 8.302909513935447, + 8.44304065790493, + 8.584651880897582, + 8.729602013947442, + 8.878258262993768, + 9.021958097000606, + 9.172754716943018, + 9.316559191909619, + 9.45882105990313, + 9.60703613597434, + 9.75087310699746, + 9.894850495969877, + 10.038366151973605, + 10.185213806922548, + 10.334525212994777, + 10.485188678954728, + 10.629021665896289, + 10.777660729945637, + 10.924684562953189, + 11.067730581969954, + 11.201957896002568, + 11.335681619937532, + 11.452620452968404, + 11.575164297944866, + 11.696444801986217, + 11.816171276965179, + 11.934383352985606, + 12.053148170001805, + 12.168271177913994, + 12.286443429999053, + 12.407273881952278, + 12.52550904697273, + 12.633664932916872, + 12.731363539001904, + 12.827105261967517, + 12.911336508928798, + 12.97743070893921, + 13.042061165906489, + 13.129601628985256, + 13.184877150924876, + 13.23473019094672 + ], + "chunk_gaps_ms": { + "count": 95, + "p50": 141.71229698695242, + "p95": 150.66346595995128, + "max": 237.911562086083 + } + }, + { + "start": 754070.625313118, + "first": 754071.309059635, + "last": 754083.537225825, + "end": 754083.537334706, + "elapsed_s": 12.912021588068455, + "chunk_offsets_s": [ + 0.6837465170538053, + 0.8119805930182338, + 0.9455424139741808, + 1.0809910519747064, + 1.2093580879736692, + 1.3347661530133337, + 1.4677196480333805, + 1.6091600540094078, + 1.7447836060309783, + 1.8837875260505825, + 2.0243607560405508, + 2.1667103699874133, + 2.3098625199636444, + 2.440266662975773, + 2.57332588697318, + 2.703068056027405, + 2.839420799049549, + 2.979948219959624, + 3.1194370400626212, + 3.261135576060042, + 3.405427047982812, + 3.549106211052276, + 3.6915333779761568, + 3.8392401459859684, + 3.980859433999285, + 4.120544639066793, + 4.264767423039302, + 4.4041666890261695, + 4.546306286007166, + 4.690748927067034, + 4.836231593973935, + 4.978503048070706, + 5.123648048029281, + 5.272429098025896, + 5.4160378109663725, + 5.556714978069067, + 5.700536477030255, + 5.845917431986891, + 5.984065709984861, + 6.126665455056354, + 6.278321331017651, + 6.424363875063136, + 6.566072843037546, + 6.71719529107213, + 6.865680449991487, + 7.015718006994575, + 7.162034186068922, + 7.306130583980121, + 7.449218204012141, + 7.58544662501663, + 7.729577263002284, + 7.874377698986791, + 8.016221803962253, + 8.15688148199115, + 8.303297389065847, + 8.44336487003602, + 8.585490528959781, + 8.730062771006487, + 8.878798763966188, + 9.022531974012963, + 9.17329768196214, + 9.317061420995742, + 9.459471993963234, + 9.60754374100361, + 9.751414263970219, + 9.895638950983994, + 10.039014318026602, + 10.18576437199954, + 10.33506941003725, + 10.485764460987411, + 10.629563751979731, + 10.778205599053763, + 10.925328840967268, + 11.068312603980303, + 11.202433467959054, + 11.336212474037893, + 11.453196123009548, + 11.575735614984296, + 11.697092137066647, + 11.816746818018146, + 11.934930318035185, + 12.053708064020611, + 12.168826383072883, + 12.287091835984029, + 12.407849728013389, + 12.526010524015874, + 12.634320060024038, + 12.73201733699534, + 12.827750051976182, + 12.911912707029842 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 142.13959698099643, + "p95": 150.0375570030883, + "max": 151.65587596129626 + } + }, + { + "start": 754070.625165453, + "first": 754070.736416714, + "last": 754083.259074389, + "end": 754083.259220582, + "elapsed_s": 12.634055129019544, + "chunk_offsets_s": [ + 0.11125126108527184, + 0.1689467750256881, + 0.3613856250885874, + 0.5743406650144607, + 0.811938640079461, + 0.9456219821004197, + 1.0809149550041184, + 1.2092615590663627, + 1.3346643280237913, + 1.4676377910654992, + 1.6094323120778427, + 1.7444594270782545, + 1.8834302900359035, + 2.0240151680773124, + 2.1664442870533094, + 2.309577173087746, + 2.4399062920128927, + 2.572954555042088, + 2.703020903049037, + 2.839270829106681, + 2.979608600027859, + 3.119156653061509, + 3.261031319038011, + 3.4052006780402735, + 3.5487944160122424, + 3.691758868051693, + 3.8390778721077368, + 3.9807525200303644, + 4.120804065023549, + 4.264705342007801, + 4.404234226094559, + 4.546356462058611, + 4.690628973999992, + 4.83614554500673, + 4.978216885006987, + 5.123403325094841, + 5.272644683020189, + 5.415741760050878, + 5.55636811000295, + 5.7008215840905905, + 5.845869799028151, + 5.984272752073593, + 6.126606798032299, + 6.27803737600334, + 6.424102337099612, + 6.565727416076697, + 6.717000936041586, + 6.865498831029981, + 7.015325779095292, + 7.161881672102027, + 7.305904709035531, + 7.449528447003104, + 7.585114157060161, + 7.729609839036129, + 7.874229138018563, + 8.015942089026794, + 8.156643111025915, + 8.303146492107771, + 8.443020146107301, + 8.585582834086381, + 8.729906065040268, + 8.87868004210759, + 9.022504726075567, + 9.17324118502438, + 9.316766250063665, + 9.459153367090039, + 9.607186377048492, + 9.751349656027742, + 9.895899241091684, + 10.039005998056382, + 10.185486161033623, + 10.334788511041552, + 10.485659420024604, + 10.629463894059882, + 10.77810390200466, + 10.924994949018583, + 11.06795348005835, + 11.202137913089246, + 11.336100792046636, + 11.452988152042963, + 11.575562765006907, + 11.696874662069604, + 11.816401247051544, + 11.935264545027167, + 12.053605039021932, + 12.16863206110429, + 12.28692701808177, + 12.407492028083652, + 12.525727721047588, + 12.633908936055377 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 142.42911897599697, + "p95": 151.2735199648887, + "max": 237.59797506500036 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 95.9, + "per_stream_tps": 13.0, + "ttft_s_max": 1.76, + "timing_valid": true, + "end_to_end_tps": 95.73566142458608, + "makespan_s": 21.392237432999536, + "steady_tps": 103.2, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 95.85951710167951, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 21.392237432999536, + "end_to_end_tps": 95.73566142458608, + "timing_valid": true, + "steady_tps": 103.23717894738185, + "per_stream_tps": [ + 14.6, + 12.4, + 12.9, + 13.3, + 14.4, + 13.1, + 12.5, + 12.1 + ], + "ttft_s": [ + 0.92, + 0.47, + 0.69, + 1.76, + 1.46, + 1.19, + 0.11, + 0.28 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0b4d014c8872", + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b82191a9a6ec", + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "3f4413b23169", + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "51fe79c071ab", + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e9b7a497dd30", + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "097c9fe5896b", + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05" + } + ], + "client_intervals": [ + { + "start": 754083.863691151, + "first": 754084.788589102, + "last": 754102.196432115, + "end": 754102.196601108, + "elapsed_s": 18.332909956923686, + "chunk_offsets_s": [ + 0.9248979509575292, + 1.0796150229871273, + 1.3560080740135163, + 1.6526573789305985, + 1.9679951450088993, + 2.174828812945634, + 2.376539057935588, + 2.5857604279881343, + 2.7955855479231104, + 3.019679376971908, + 3.2346487370086834, + 3.458277681027539, + 3.6805280359694734, + 3.8982754059834406, + 4.118547678925097, + 4.329018771997653, + 4.545152941951528, + 4.76632947998587, + 4.986705000977963, + 5.206882264930755, + 5.430942076956853, + 5.6519780840026215, + 5.880438736989163, + 6.104424371966161, + 6.330484153935686, + 6.564467369928025, + 6.7858439239207655, + 7.017927106004208, + 7.2451135870069265, + 7.469788059941493, + 7.6922646559542045, + 7.914750405005179, + 8.140051426948048, + 8.363204653956927, + 8.583244411973283, + 8.807363166939467, + 9.036678995005786, + 9.254363641957752, + 9.475192319019698, + 9.704722660942934, + 9.928041856968775, + 10.146129627944902, + 10.371957150986418, + 10.598846442997456, + 10.827789083006792, + 11.059473140980117, + 11.287020263029262, + 11.517230530967936, + 11.741218131966889, + 11.97543810098432, + 12.204836184973828, + 12.431476515019313, + 12.657786200987175, + 12.879002160974778, + 13.102484670002013, + 13.332932666991837, + 13.562945891986601, + 13.786830020020716, + 14.011231208918616, + 14.240213271928951, + 14.471382300020196, + 14.698553210007958, + 14.930704662925564, + 15.156865612021647, + 15.384668607963249, + 15.617029197979718, + 15.849551358027384, + 16.085214155027643, + 16.31207088602241, + 16.543644924997352, + 16.774421499925666, + 17.00488173693884, + 17.232608571997844, + 17.454872907954268, + 17.671651656972244, + 17.898469234933145, + 18.119578248937614, + 18.33274096401874 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 224.67447293456644, + "p95": 235.66279700025916, + "max": 315.3377660783008 + } + }, + { + "start": 754083.86350179, + "first": 754084.330939947, + "last": 754104.88878508, + "end": 754104.888874073, + "elapsed_s": 21.025372283067554, + "chunk_offsets_s": [ + 0.4674381570657715, + 0.5815563710639253, + 0.8166848149849102, + 1.0796467020409182, + 1.3560305860592052, + 1.6520180279621854, + 1.9680705050705, + 2.174711578991264, + 2.3774211449781433, + 2.5872712730197236, + 2.7960649440065026, + 3.0198302409844473, + 3.234795344993472, + 3.4584377290448174, + 3.680769333965145, + 3.8986219520447776, + 4.118686559027992, + 4.3288135370239615, + 4.545695923035964, + 4.766645082039759, + 4.9875369280343875, + 5.20712839497719, + 5.431271311012097, + 5.652114821015857, + 5.880587634048425, + 6.104574196971953, + 6.330814187997021, + 6.564868237008341, + 6.786444153054617, + 7.017839665059, + 7.245137906051241, + 7.469609481049702, + 7.692584561998956, + 7.914646242978051, + 8.140627191984095, + 8.363303021993488, + 8.583544046035968, + 8.807252701022662, + 9.036758819012903, + 9.25467442907393, + 9.475660786964, + 9.705360122025013, + 9.928179488983005, + 10.14645003003534, + 10.37201587005984, + 10.59908852505032, + 10.827928476035595, + 11.059434868046083, + 11.286742868018337, + 11.517310227034613, + 11.741437749005854, + 11.975742710987106, + 12.205184955033474, + 12.431627170997672, + 12.658016587025486, + 12.878930047969334, + 13.102631565998308, + 13.333159708068706, + 13.56307208398357, + 13.787176134064794, + 14.011510411044583, + 14.240212535019964, + 14.471519067999907, + 14.698679705965333, + 14.930805783020332, + 15.157168238074519, + 15.384800208033994, + 15.617152430000715, + 15.849690782022662, + 16.086026018019766, + 16.31220335001126, + 16.543962031020783, + 16.774661806062795, + 17.005153130041435, + 17.233168816077523, + 17.45543184003327, + 17.672073579975404, + 17.89928397804033, + 18.120366718969308, + 18.333645739010535, + 18.543316790019162, + 18.743248839047737, + 18.944737910060212, + 19.13827580306679, + 19.324826430995017, + 19.511503059999086, + 19.70266603503842, + 19.89021312003024, + 20.078281922033057, + 20.259496117010713, + 20.430239350069314, + 20.57094505603891, + 20.706370600964874, + 20.823841611971147, + 20.93286238401197, + 21.025283290073276 + ], + "chunk_gaps_ms": { + "count": 95, + "p50": 223.70865498669446, + "p95": 236.33523599710315, + "max": 316.05247710831463 + } + }, + { + "start": 754083.863465085, + "first": 754084.551800339, + "last": 754104.294088823, + "end": 754104.294216376, + "elapsed_s": 20.430751290987246, + "chunk_offsets_s": [ + 0.6883352539734915, + 0.816897201933898, + 1.0796201109187678, + 1.3565896479412913, + 1.6522479029372334, + 1.9681993859121576, + 2.174815324950032, + 2.3768101959722117, + 2.5873893229290843, + 2.7958589899353683, + 3.0198053939966485, + 3.2349107549525797, + 3.4584029460093006, + 3.680892950971611, + 3.8985823519760743, + 4.118856256944127, + 4.329111828934401, + 4.545665218960494, + 4.76661652198527, + 4.987188222003169, + 5.207243739976548, + 5.431228622910567, + 5.65225051890593, + 5.880698338965885, + 6.104713606997393, + 6.331151679973118, + 6.564822988933884, + 6.786579386913218, + 7.017756960936822, + 7.245246356003918, + 7.469845419982448, + 7.692551298998296, + 7.91461451491341, + 8.140566568006761, + 8.363261949969456, + 8.58339151693508, + 8.807907187961973, + 9.037063253927045, + 9.254752221982926, + 9.475763779948466, + 9.705865342984907, + 9.928369171917439, + 10.146586271934211, + 10.372141311992891, + 10.599233102984726, + 10.82806631794665, + 11.05984575196635, + 11.287382362992503, + 11.517920264974236, + 11.741819561924785, + 11.975999705959111, + 12.205786208971404, + 12.432210632949136, + 12.658313325955532, + 12.879326867987402, + 13.103141363942996, + 13.333549311966635, + 13.563617273932323, + 13.787418056977913, + 14.011854445911013, + 14.240647723898292, + 14.472348821000196, + 14.699547746917233, + 14.931483997963369, + 15.157628258923069, + 15.385798489907756, + 15.61763281992171, + 15.85050359100569, + 16.08632634091191, + 16.31299294193741, + 16.544070192961954, + 16.77528624399565, + 17.005947842961177, + 17.232977262930945, + 17.455766499973834, + 17.672336270916276, + 17.898963879910298, + 18.120185245992616, + 18.333724492928013, + 18.543413655948825, + 18.743325271992944, + 18.944813462905586, + 19.13863667100668, + 19.3250968339853, + 19.51157226099167, + 19.702483841916546, + 19.89033406600356, + 20.078467588988133, + 20.259728246950544, + 20.43062373797875 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 223.9464040612802, + "p95": 235.82274990621954, + "max": 315.95148297492415 + } + }, + { + "start": 754083.863584142, + "first": 754085.623474582, + "last": 754104.796231277, + "end": 754104.796541968, + "elapsed_s": 20.93295782606583, + "chunk_offsets_s": [ + 1.7598904400365427, + 1.9688335680402815, + 2.174781869049184, + 2.3769678300013766, + 2.5864043380133808, + 2.795379338087514, + 3.020202230080031, + 3.2354367760708556, + 3.4587921660859138, + 3.681160346022807, + 3.898792690015398, + 4.119116579997353, + 4.329208709998056, + 4.545316496049054, + 4.767064222018234, + 4.987951429095119, + 5.207286333083175, + 5.4314387530321255, + 5.6522280550561845, + 5.880647987010889, + 6.10474823100958, + 6.330663500004448, + 6.564500186010264, + 6.786104983068071, + 7.0181533170398325, + 7.245582311064936, + 7.46997089299839, + 7.692802549107, + 7.914797989069484, + 8.140275717014447, + 8.364316312014125, + 8.584322118083946, + 8.808183063054457, + 9.037125319009647, + 9.255002704099752, + 9.47611746401526, + 9.705646573100239, + 9.928384980070405, + 10.146864915033802, + 10.372325266012922, + 10.599343616049737, + 10.82814134308137, + 11.059003344038501, + 11.286725221085362, + 11.517605446046218, + 11.741281733033247, + 11.975595574011095, + 12.205032954108901, + 12.431921783019789, + 12.658031932078302, + 12.878891904023476, + 13.102638191077858, + 13.332951290998608, + 13.563158534001559, + 13.787019030074589, + 14.011512827011757, + 14.240365674020723, + 14.471579262055457, + 14.698758028098382, + 14.931104474002495, + 15.15718179102987, + 15.384590831003152, + 15.616995470016263, + 15.849747232045047, + 16.085036665084772, + 16.312038278090768, + 16.54349917906802, + 16.774635182111524, + 17.005143627058715, + 17.23277892509941, + 17.45512254210189, + 17.67180921009276, + 17.898326658061706, + 18.11975244199857, + 18.332986326073296, + 18.543105045100674, + 18.742651906097308, + 18.944174465024844, + 19.137941561057232, + 19.32492953701876, + 19.51113657699898, + 19.702037709997967, + 19.890070256078616, + 20.07808075309731, + 20.25921048305463, + 20.430248711025342, + 20.570580541039817, + 20.705997831071727, + 20.823654843028635, + 20.93264713510871 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 223.67628698702902, + "p95": 232.4046390131116, + "max": 235.28943303972483 + } + }, + { + "start": 754083.865490416, + "first": 754085.327121409, + "last": 754103.001351413, + "end": 754103.001540839, + "elapsed_s": 19.136050423025154, + "chunk_offsets_s": [ + 1.4616309930570424, + 1.6506990720517933, + 1.9667178690433502, + 2.17294226703234, + 2.37522526900284, + 2.584831379004754, + 2.7939396440051496, + 3.018214051029645, + 3.233633815078065, + 3.456790803000331, + 3.679086310090497, + 3.896791664068587, + 4.117101520998403, + 4.326970769092441, + 4.543501086998731, + 4.765096876071766, + 4.985321996035054, + 5.205419355072081, + 5.42976291407831, + 5.6505431910045445, + 5.878901698044501, + 6.102937078103423, + 6.328956684097648, + 6.562724281102419, + 6.7841316040139645, + 7.016431876108982, + 7.243867639102973, + 7.4681764440611005, + 7.690855523105711, + 7.912802530103363, + 8.138595894095488, + 8.362285989103839, + 8.581630332046188, + 8.805401597055607, + 9.034828993026167, + 9.25263124203775, + 9.473550640046597, + 9.703033605008386, + 9.926293888012879, + 10.144521165057085, + 10.36996642802842, + 10.597160075092688, + 10.826083227060735, + 11.057735540089197, + 11.285164102097042, + 11.516014871071093, + 11.73962096509058, + 11.974004967021756, + 12.20336391509045, + 12.429731842014007, + 12.656365868053399, + 12.87725958507508, + 13.100883614039049, + 13.33127098705154, + 13.561762584024109, + 13.785921595059335, + 14.009938413044438, + 14.238340726005845, + 14.469996607047506, + 14.69718873302918, + 14.929766974062659, + 15.155681072035804, + 15.383589189033955, + 15.615000011050142, + 15.848791383090429, + 16.084508964093402, + 16.310612456058152, + 16.542204735102132, + 16.772902558092028, + 17.00367874908261, + 17.231507009011693, + 17.453271404025145, + 17.669943545013666, + 17.89672005106695, + 18.117694103042595, + 18.33103437209502, + 18.541145971044898, + 18.74056313408073, + 18.942352368030697, + 19.135860997019336 + ], + "chunk_gaps_ms": { + "count": 79, + "p50": 224.15901103522629, + "p95": 233.79137204028666, + "max": 316.0187969915569 + } + }, + { + "start": 754083.863256059, + "first": 754085.050872039, + "last": 754104.569461235, + "end": 754104.569612997, + "elapsed_s": 20.706356937997043, + "chunk_offsets_s": [ + 1.1876159800449386, + 1.3567013449501246, + 1.6527811399428174, + 1.96858434996102, + 2.175314113032073, + 2.3771110150264576, + 2.586793957045302, + 2.796233666012995, + 3.0204873039619997, + 3.2358185070334002, + 3.459087223978713, + 3.681439197040163, + 3.899169606040232, + 4.119501383043826, + 4.3295806009555236, + 4.546072519035079, + 4.7672982410294935, + 4.98784734797664, + 5.207825426012278, + 5.43196384597104, + 5.652971021947451, + 5.88104858295992, + 6.1052070669829845, + 6.331420770962723, + 6.565162239945494, + 6.78705873596482, + 7.018435655045323, + 7.2458786820061505, + 7.470156398951076, + 7.693245033035055, + 7.915244553005323, + 8.140692344983108, + 8.364165046950802, + 8.58406077104155, + 8.807720914017409, + 9.03690339694731, + 9.255505988956429, + 9.476353114005178, + 9.70568180596456, + 9.92910375399515, + 10.146984291961417, + 10.372686116956174, + 10.599703507032245, + 10.8287000519922, + 11.059613973950036, + 11.287233673036098, + 11.51820354803931, + 11.741914186975919, + 11.97617038001772, + 12.205899346037768, + 12.432457995950244, + 12.658647169009782, + 12.879708584048785, + 13.102716192021035, + 13.333970580017194, + 13.563794492045417, + 13.787846221006475, + 14.011959488037974, + 14.240803614025936, + 14.47199049696792, + 14.699265728937462, + 14.931371581042185, + 15.157682035001926, + 15.38569800904952, + 15.617762645008042, + 15.850245428038761, + 16.086116499034688, + 16.312730427947827, + 16.544150177040137, + 16.775453733978793, + 17.005637599970214, + 17.23359789303504, + 17.45603487000335, + 17.67243422404863, + 17.899385755998082, + 18.12034952000249, + 18.333985151024535, + 18.54382625198923, + 18.74318775103893, + 18.94477567099966, + 19.138689678977244, + 19.325161651009694, + 19.51142290001735, + 19.702443170011975, + 19.89031209796667, + 20.078370708040893, + 20.259583301958628, + 20.43027632695157, + 20.57087206398137, + 20.70620517595671 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 223.66014297585934, + "p95": 233.74146898277104, + "max": 315.80321001820266 + } + }, + { + "start": 754083.863378109, + "first": 754083.974014514, + "last": 754104.294180168, + "end": 754104.29443249, + "elapsed_s": 20.43105438107159, + "chunk_offsets_s": [ + 0.11063640506472439, + 0.1697102190228179, + 0.36042489006649703, + 0.5819965190021321, + 0.8172899250639603, + 1.0801355870207772, + 1.3563685090048239, + 1.6526966580422595, + 1.9684037560364231, + 2.1751124950824305, + 2.377372936112806, + 2.5863291840068996, + 2.795290728099644, + 3.0200962600065395, + 3.2352271410636604, + 3.4587027720408514, + 3.6814018670702353, + 3.8989507710793987, + 4.119084547040984, + 4.329025507089682, + 4.545879108016379, + 4.766847403021529, + 4.987193853012286, + 5.2074415649985895, + 5.4315274090040475, + 5.652476008050144, + 5.880886932020076, + 6.104986424092203, + 6.331179104046896, + 6.564650059095584, + 6.787117967032827, + 7.018179796054028, + 7.245507829007693, + 7.469832603004761, + 7.692850325023755, + 7.914953318075277, + 8.140610616072081, + 8.36360828904435, + 8.584045026102103, + 8.808454088051803, + 9.036649682093412, + 9.254985551000573, + 9.475909493048675, + 9.705239961040206, + 9.928687142091803, + 10.146786433062516, + 10.372337089036591, + 10.599499505013227, + 10.828395664109848, + 11.059435651055537, + 11.287423370056786, + 11.518046969082206, + 11.741700376034714, + 11.976007305085659, + 12.205427533015609, + 12.431670819059946, + 12.658435119083151, + 12.879256595042534, + 13.102700527058914, + 13.333576416014694, + 13.563560681068338, + 13.78740930405911, + 14.01187283010222, + 14.240405113087036, + 14.471907200058922, + 14.699088350054808, + 14.931659838999622, + 15.157455392065458, + 15.385224308003671, + 15.61767003510613, + 15.850659479037859, + 16.08542945201043, + 16.312551593058743, + 16.543902110075578, + 16.77557787799742, + 17.006089187110774, + 17.23340926610399, + 17.455458448035643, + 17.672261550091207, + 17.898889398085885, + 18.120025484007783, + 18.333454650011845, + 18.543406406999566, + 18.743110518087633, + 18.944577972055413, + 19.138529341085814, + 19.32509104104247, + 19.511419859016314, + 19.702458433108404, + 19.890373234055005, + 20.078362259082496, + 20.259567380999215, + 20.430802059010603 + ], + "chunk_gaps_ms": { + "count": 92, + "p50": 223.9672334981151, + "p95": 235.29340606182814, + "max": 315.70709799416363 + } + }, + { + "start": 754083.86315409, + "first": 754084.140280365, + "last": 754105.255156561, + "end": 754105.255391523, + "elapsed_s": 21.392237432999536, + "chunk_offsets_s": [ + 0.2771262750029564, + 0.3606960129691288, + 0.582050664932467, + 0.8174079590244219, + 1.0800752990180627, + 1.3568350420100614, + 1.6527802919736132, + 1.9689848659327254, + 2.175452786963433, + 2.3778390219667926, + 2.5872263299534097, + 2.7954470349941403, + 3.020371846971102, + 3.2355660729808733, + 3.4590412880061194, + 3.681335275992751, + 3.899055668967776, + 4.119486422976479, + 4.32949687296059, + 4.546140391961671, + 4.767179711023346, + 4.988152278936468, + 5.208039971999824, + 5.43216864799615, + 5.653039630036801, + 5.881189655978233, + 6.105342893046327, + 6.331359377945773, + 6.56530069001019, + 6.786952942959033, + 7.018363574985415, + 7.245939370011911, + 7.470449569984339, + 7.693284137989394, + 7.915076919947751, + 8.140749785932712, + 8.363893875968643, + 8.584558264003135, + 8.808645738987252, + 9.036949045024812, + 9.255263090948574, + 9.476286952965893, + 9.706266082939692, + 9.929367388947867, + 10.147397415945306, + 10.37272251804825, + 10.599961734027602, + 10.828768964041956, + 11.059582181973383, + 11.287611005012877, + 11.517965513980016, + 11.74187028198503, + 11.976509294938296, + 12.206052483990788, + 12.43240837904159, + 12.658570160972886, + 12.880005353945307, + 13.103559240000322, + 13.333758977940306, + 13.563960828934796, + 13.788056303048506, + 14.012214898015372, + 14.240869662025943, + 14.472390069044195, + 14.699761428986676, + 14.93159855902195, + 15.157833909033798, + 15.38555866398383, + 15.61781714099925, + 15.850444325944409, + 16.085879249032587, + 16.31349625904113, + 16.544473364017904, + 16.77530347602442, + 17.006358198006637, + 17.233666053041816, + 17.45597466197796, + 17.672573728952557, + 17.898948200047016, + 18.120877188048325, + 18.33433923497796, + 18.54405222204514, + 18.743925118003972, + 18.945173691026866, + 19.139117410988547, + 19.325115090003237, + 19.51181189599447, + 19.70321981702, + 19.890774374012835, + 20.078945048968308, + 20.260007977951318, + 20.430870300042443, + 20.57119273894932, + 20.70667774102185, + 20.824154702946544, + 20.933803593041375, + 21.02559514902532, + 21.095721303950995, + 21.15640069998335, + 21.21401480201166, + 21.27276947593782, + 21.33772043103818, + 21.39200247102417 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 223.12269802205265, + "p95": 235.35729409195483, + "max": 316.20457395911217 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 35.0, + "per_stream_tps": 35.0, + "ttft_s_max": 0.16, + "timing_valid": true, + "end_to_end_tps": 34.401383436275324, + "makespan_s": 7.4415611940203235, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 35.027123617893885, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 7.4415611940203235, + "end_to_end_tps": 34.401383436275324, + "timing_valid": true, + "per_stream_tps": [ + 35.0 + ], + "ttft_s": [ + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + } + ], + "client_intervals": [ + { + "start": 754255.553397815, + "first": 754255.714485528, + "last": 754262.994558045, + "end": 754262.994959009, + "elapsed_s": 7.4415611940203235, + "chunk_offsets_s": [ + 0.1610877129714936, + 0.22386323101818562, + 0.2829813070129603, + 0.3369382800301537, + 0.39079602097626776, + 0.4487562470603734, + 0.5003660300280899, + 0.551803988055326, + 0.6031603940064088, + 0.6546776170143858, + 0.7061769040301442, + 0.7575178049737588, + 0.8095783770550042, + 0.8614552830113098, + 0.9137753930408508, + 0.964626369997859, + 1.0157949340064079, + 1.0660829619737342, + 1.1175162959843874, + 1.1682789680780843, + 1.219284298014827, + 1.2695088210748509, + 1.3230077259941027, + 1.3769754359964281, + 1.436623085057363, + 1.4919042299734429, + 1.54373665608, + 1.5981174890184775, + 1.6554577570641413, + 1.7134938400704414, + 1.772660732967779, + 1.831883386010304, + 1.8925697490340099, + 1.9562322669662535, + 2.008888956042938, + 2.0661494790110737, + 2.1252686280058697, + 2.184054397046566, + 2.2431933539919555, + 2.296100493054837, + 2.34953576605767, + 2.4017226110445336, + 2.4530862000538036, + 2.5038250640500337, + 2.5567573240259662, + 2.606961286976002, + 2.661179861985147, + 2.7215004609897733, + 2.7803469189675525, + 2.8368013229919598, + 2.8886463890084997, + 2.942697331076488, + 3.001663965987973, + 3.0599684990011156, + 3.111276215990074, + 3.1627611190779135, + 3.2162778640631586, + 3.2745206360705197, + 3.332447469001636, + 3.384562826016918, + 3.4364003560040146, + 3.487069442984648, + 3.5377781779970974, + 3.5886508190305904, + 3.6397245980333537, + 3.6911168589722365, + 3.7426521939923987, + 3.7933194410288706, + 3.8452538040000945, + 3.8965797930723056, + 3.947414002032019, + 3.9984904680168256, + 4.0496278799837455, + 4.100236614001915, + 4.1523308659670874, + 4.205385975074023, + 4.263174536055885, + 4.322084321989678, + 4.3829249740811065, + 4.443753818050027, + 4.505494366050698, + 4.557582378038205, + 4.609800567035563, + 4.66120300500188, + 4.713324184995145, + 4.7641927460208535, + 4.817306287004612, + 4.8678166850004345, + 4.918690126040019, + 4.972143894061446, + 5.024277747026645, + 5.0764736799756065, + 5.128689932986163, + 5.1793656760128215, + 5.230795298004523, + 5.281412319978699, + 5.333062137011439, + 5.383772248052992, + 5.438039032043889, + 5.497820170014165, + 5.557965215994045, + 5.619674658984877, + 5.681162197026424, + 5.747232049005106, + 5.8120000649942085, + 5.865181255037896, + 5.92530970799271, + 5.9832693099742755, + 6.041598003008403, + 6.091511867009103, + 6.1429817450698465, + 6.194582073017955, + 6.246023406973109, + 6.299329926026985, + 6.35239170701243, + 6.405471744015813, + 6.456539714010432, + 6.509892666013911, + 6.567926028044894, + 6.618921661982313, + 6.670567382010631, + 6.722940900013782, + 6.775022831978276, + 6.827137132058851, + 6.879061751067638, + 6.930961713078432, + 6.98237245506607, + 7.032570753013715, + 7.085062833037227, + 7.136704329052009, + 7.188106653979048, + 7.240713151055388, + 7.291285661049187, + 7.343244455987588, + 7.393865589983761, + 7.441160229966044 + ], + "chunk_gaps_ms": { + "count": 135, + "p50": 52.12117999326438, + "p95": 61.48753804154694, + "max": 66.06985197868198 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 53.2, + "per_stream_tps": 27.8, + "ttft_s_max": 0.38, + "timing_valid": true, + "end_to_end_tps": 52.56448534246739, + "makespan_s": 9.740416873944923, + "steady_tps": 56.0, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 53.242712688284676, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 9.740416873944923, + "end_to_end_tps": 52.56448534246739, + "timing_valid": true, + "steady_tps": 55.95810887939078, + "per_stream_tps": [ + 26.6, + 28.9 + ], + "ttft_s": [ + 0.16, + 0.38 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + } + ], + "client_intervals": [ + { + "start": 754262.996736497, + "first": 754263.157850337, + "last": 754272.73662599, + "end": 754272.736842008, + "elapsed_s": 9.740105510922149, + "chunk_offsets_s": [ + 0.1611138399457559, + 0.22250408097170293, + 0.46375873300712556, + 0.5562213059747592, + 0.6512277419678867, + 0.7434564880095422, + 0.8355639380170032, + 0.9275062979431823, + 1.0209340149303898, + 1.1131594339385629, + 1.2070230270037428, + 1.301654740003869, + 1.3953828759258613, + 1.4917102359468117, + 1.5828656449448317, + 1.67708351300098, + 1.7766441989224404, + 1.8765938959550112, + 1.974935017991811, + 2.0700410549761727, + 2.166350158979185, + 2.2621237699640915, + 2.356617401004769, + 2.4563567920122296, + 2.5498111969791353, + 2.646937893005088, + 2.742972258012742, + 2.838079926907085, + 2.9326575739542022, + 3.029302872950211, + 3.1237281679641455, + 3.222129866015166, + 3.31920420890674, + 3.4170371029758826, + 3.515024589956738, + 3.6117249289527535, + 3.7054474969627336, + 3.801911753951572, + 3.8983900429448113, + 3.9965417069615796, + 4.09343516791705, + 4.187993295956403, + 4.285284120007418, + 4.382113036932424, + 4.478731342940591, + 4.576707789907232, + 4.673773731919937, + 4.7689295780146495, + 4.8641837110044435, + 4.9592312349705026, + 5.053990276996046, + 5.146208751015365, + 5.236976443906315, + 5.329656361951493, + 5.42561792593915, + 5.518343108939007, + 5.613356616930105, + 5.710558176971972, + 5.8050531189655885, + 5.899532429990359, + 5.996022110921331, + 6.091789961908944, + 6.182869688956998, + 6.279147384921089, + 6.372293082997203, + 6.466072818962857, + 6.560835684998892, + 6.658034603926353, + 6.752637611934915, + 6.849984020926058, + 6.947174747940153, + 7.042213519918732, + 7.137926089926623, + 7.235857927938923, + 7.333106991951354, + 7.429532192996703, + 7.524854598916136, + 7.617032737005502, + 7.712389559019357, + 7.808618790004402, + 7.90219813992735, + 7.99594965192955, + 8.088952420977876, + 8.18088540493045, + 8.275251513929106, + 8.373457114910707, + 8.470903635956347, + 8.564567034947686, + 8.655062117963098, + 8.748194279964082, + 8.843885745969601, + 8.940545812016353, + 9.036223388975486, + 9.130052726017311, + 9.21801728894934, + 9.28127764200326, + 9.331638774019666, + 9.384450375917368, + 9.43633358599618, + 9.488854160998017, + 9.540737434988841, + 9.591256135958247, + 9.64232132199686, + 9.693034808966331, + 9.739889492979273 + ], + "chunk_gaps_ms": { + "count": 104, + "p50": 95.0099719921127, + "p95": 98.34112203679979, + "max": 241.25465203542262 + } + }, + { + "start": 754262.996425134, + "first": 754263.377135045, + "last": 754272.215085757, + "end": 754272.215193982, + "elapsed_s": 9.21876884798985, + "chunk_offsets_s": [ + 0.3807099109981209, + 0.46386913396418095, + 0.5563391470350325, + 0.651341310935095, + 0.7439659329829738, + 0.835674291010946, + 0.927551673958078, + 1.0210473289480433, + 1.1132914029294625, + 1.2070937160169706, + 1.3017749009886757, + 1.39575500797946, + 1.491836364963092, + 1.582987773930654, + 1.677574413944967, + 1.7767694320064038, + 1.8770873729372397, + 1.975134331965819, + 2.0704059379640967, + 2.1664950880222023, + 2.262486140942201, + 2.356744857970625, + 2.4566208419855684, + 2.5500739839626476, + 2.6471984869567677, + 2.7432334759505466, + 2.8383422809420153, + 2.933017353992909, + 3.0295276109827682, + 3.1239893060410395, + 3.222392908995971, + 3.3194676999701187, + 3.4172712169820443, + 3.5154556339839473, + 3.6122852379921824, + 3.7060244779568166, + 3.802486606989987, + 3.898512540035881, + 3.997096688952297, + 4.094024405931123, + 4.18809784098994, + 4.286036990932189, + 4.382607984938659, + 4.479395748930983, + 4.577083713957109, + 4.674244616995566, + 4.769444718025625, + 4.865006214939058, + 4.959884410025552, + 5.054618587018922, + 5.146832228987478, + 5.237775714951567, + 5.330277408007532, + 5.426237804000266, + 5.518896377994679, + 5.613765517016873, + 5.711115397978574, + 5.805777269997634, + 5.900206948979758, + 5.9966228529810905, + 6.0925189449917525, + 6.1836039839545265, + 6.279864975018427, + 6.37303187395446, + 6.466644152998924, + 6.561498443013988, + 6.658713730983436, + 6.753131952951662, + 6.850791148957796, + 6.947973444010131, + 7.0426969330292195, + 7.1386107839643955, + 7.236630799015984, + 7.3338367430260405, + 7.43015386199113, + 7.52549398096744, + 7.617109632934444, + 7.712439063936472, + 7.809171707020141, + 7.902951683034189, + 7.996472185011953, + 8.089505178970285, + 8.18158630700782, + 8.275893775979057, + 8.374071856029332, + 8.471394760999829, + 8.565231874003075, + 8.655699915951118, + 8.748784461989999, + 8.844633704982698, + 8.941114168963395, + 9.036994389025494, + 9.130642812000588, + 9.218660622951575 + ], + "chunk_gaps_ms": { + "count": 93, + "p50": 95.20010103005916, + "p95": 98.40360295493156, + "max": 100.3179409308359 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 68.5, + "per_stream_tps": 18.1, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 68.04737329986936, + "makespan_s": 15.048339860048145, + "steady_tps": 73.5, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 68.52066756708115, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 15.048339860048145, + "end_to_end_tps": 68.04737329986936, + "timing_valid": true, + "steady_tps": 73.54878140777869, + "per_stream_tps": [ + 17.6, + 19.2, + 18.0, + 18.2 + ], + "ttft_s": [ + 0.16, + 0.38, + 0.89, + 0.62 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0a24c104609f", + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "28cedcb814fc", + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767" + } + ], + "client_intervals": [ + { + "start": 754272.739005692, + "first": 754272.900525918, + "last": 754287.364343103, + "end": 754287.364435359, + "elapsed_s": 14.625429667066783, + "chunk_offsets_s": [ + 0.16152022604364902, + 0.2248992050299421, + 0.4652660550782457, + 0.7334977349964902, + 1.019619067083113, + 1.150378869031556, + 1.279673904995434, + 1.4004094870761037, + 1.5292329509975389, + 1.6538801840506494, + 1.785755900084041, + 1.916277667041868, + 2.048279672046192, + 2.187117803026922, + 2.3201062810840085, + 2.4613647219957784, + 2.603273985092528, + 2.7439937500748783, + 2.8863236730685458, + 3.026726218056865, + 3.1680191560881212, + 3.3110700130928308, + 3.4523563580587506, + 3.5956932980334386, + 3.73791002004873, + 3.8801545180613175, + 4.024122488102876, + 4.168116634013131, + 4.3103542840108275, + 4.452728831092827, + 4.595321397064254, + 4.739498008042574, + 4.883459818083793, + 5.0267562930239365, + 5.169094423996285, + 5.3140340350801125, + 5.452520082006231, + 5.596425843075849, + 5.742265798035078, + 5.887140992097557, + 6.030577149009332, + 6.173403796041384, + 6.317330181016587, + 6.46507920906879, + 6.6112329420866445, + 6.759835578035563, + 6.9069324400043115, + 7.050778008066118, + 7.19625844003167, + 7.342335308087058, + 7.488724692026153, + 7.629481400013901, + 7.770595103036612, + 7.910736334044486, + 8.05183301307261, + 8.187531819101423, + 8.331018264056183, + 8.47403123206459, + 8.61846776609309, + 8.763225422007963, + 8.906949197058566, + 9.048486792016774, + 9.184355120058171, + 9.325786506058648, + 9.46724902803544, + 9.609695143066347, + 9.753466967027634, + 9.893796759075485, + 10.032957180053927, + 10.174322325037792, + 10.315558910020627, + 10.457059944048524, + 10.602530391071923, + 10.745222764089704, + 10.88834659801796, + 11.02674429200124, + 11.170456611085683, + 11.30877424008213, + 11.44938408199232, + 11.59082873200532, + 11.727595052099787, + 11.86785485909786, + 12.00977927306667, + 12.147738979081623, + 12.284468658035621, + 12.427458730060607, + 12.57529897405766, + 12.714831878081895, + 12.853853978100233, + 12.993825959041715, + 13.131807041005231, + 13.274744968046434, + 13.408123720088042, + 13.548614936997183, + 13.686164943035692, + 13.804058146080934, + 13.920550009002909, + 14.04182354605291, + 14.159693964989856, + 14.280381354037672, + 14.405418766080402, + 14.528930068016052, + 14.625337411067449 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 141.45358599489555, + "p95": 147.74902805220336, + "max": 286.1213320866227 + } + }, + { + "start": 754272.739156461, + "first": 754273.121016667, + "last": 754286.425548879, + "end": 754286.42565912, + "elapsed_s": 13.686502658994868, + "chunk_offsets_s": [ + 0.38186020602006465, + 0.46515503001865, + 0.7335260399850085, + 1.0195029060123488, + 1.1502778440481052, + 1.279590608086437, + 1.4003975350642577, + 1.5285663210088387, + 1.6539085210533813, + 1.785644427058287, + 1.9161689310567454, + 2.0484260900411755, + 2.186748696025461, + 2.320083033060655, + 2.461252610082738, + 2.6028086059959605, + 2.7440028380369768, + 2.8861295270035043, + 3.0266143140615895, + 3.167897043051198, + 3.310957260080613, + 3.452164084999822, + 3.5956008500652388, + 3.7377137310104445, + 3.8801597349811345, + 4.024078616057523, + 4.167911273078062, + 4.310268748085946, + 4.452695119078271, + 4.595097442972474, + 4.739402504055761, + 4.883267016033642, + 5.026417667046189, + 5.168770662043244, + 5.314139444031753, + 5.453019335051067, + 5.596575829084031, + 5.7425338650355116, + 5.887115183984861, + 6.030922336038202, + 6.173562998068519, + 6.317726809065789, + 6.465369276003912, + 6.61137660802342, + 6.760053195990622, + 6.90716813900508, + 7.051088412059471, + 7.196550250984728, + 7.3425639510387555, + 7.4888960220851, + 7.629916363977827, + 7.770770753035322, + 7.910813022986986, + 8.052138920058496, + 8.18791668699123, + 8.330788806080818, + 8.474456228083, + 8.618793961009942, + 8.763513216981664, + 8.906957325991243, + 9.048748522996902, + 9.184837429085746, + 9.326220782008022, + 9.467456278041936, + 9.609927706071176, + 9.7539201710606, + 9.894146186066791, + 10.033444480970502, + 10.174767962074839, + 10.315339852008037, + 10.457304042996839, + 10.602744744974189, + 10.745387790026143, + 10.88852693606168, + 11.026916741975583, + 11.170639364980161, + 11.308948897989467, + 11.449573796009645, + 11.590979246073402, + 11.727830654010177, + 11.868096973048523, + 12.010186717030592, + 12.14792728505563, + 12.28471974900458, + 12.427193655981682, + 12.575520240003243, + 12.715004967991263, + 12.854066508007236, + 12.99422172200866, + 13.132135875988752, + 13.27494559402112, + 13.408159081009217, + 13.549001308972947, + 13.686392417992465 + ], + "chunk_gaps_ms": { + "count": 93, + "p50": 141.96419098880142, + "p95": 147.6424669381231, + "max": 285.9768660273403 + } + }, + { + "start": 754272.739066812, + "first": 754273.62939694, + "last": 754287.786545512, + "end": 754287.786627721, + "elapsed_s": 15.047560909064487, + "chunk_offsets_s": [ + 0.8903301280224696, + 1.019886894035153, + 1.1507818329846486, + 1.2800784209975973, + 1.4006824339739978, + 1.5289797809673473, + 1.654122667037882, + 1.7861853759968653, + 1.9165435100439936, + 2.0483721700729802, + 2.1872099169995636, + 2.32036492403131, + 2.4616087570320815, + 2.60305671999231, + 2.7442816890543327, + 2.886538955033757, + 3.0272559199947864, + 3.168199685984291, + 3.3113560000201687, + 3.452639225986786, + 3.5963460410712287, + 3.738121079048142, + 3.8803988890722394, + 4.024420283967629, + 4.168220108025707, + 4.310778417042457, + 4.45310389099177, + 4.5954905829858035, + 4.7398113399976864, + 4.883849102072418, + 5.026445523020811, + 5.168537396006286, + 5.3144293510122225, + 5.4528907580534, + 5.596855640062131, + 5.742575976997614, + 5.887380931060761, + 6.03105939400848, + 6.173592022038065, + 6.317600631969981, + 6.4652903630631045, + 6.6116727229673415, + 6.7599300590809435, + 6.907038410077803, + 7.051221037050709, + 7.196581083000638, + 7.342598799034022, + 7.48902568698395, + 7.629891323973425, + 7.770815570023842, + 7.910842447075993, + 8.052166857058182, + 8.188078543986194, + 8.330827541998588, + 8.474439973011613, + 8.618933242978528, + 8.763541553984396, + 8.907000942039303, + 9.048795931972563, + 9.184833253035322, + 9.32626689504832, + 9.467587735969573, + 9.609973657992668, + 9.75394674798008, + 9.894299836014397, + 10.033430017065257, + 10.174810170079581, + 10.315380220999941, + 10.457426956039853, + 10.60280045797117, + 10.745429615024477, + 10.888579784077592, + 11.027101656072773, + 11.170978648005985, + 11.30899272300303, + 11.44962717301678, + 11.591032079071738, + 11.727885583066382, + 11.86815332598053, + 12.010243533994071, + 12.147983269998804, + 12.285018263966776, + 12.427217432064936, + 12.575569537002593, + 12.715060729067773, + 12.854120541014709, + 12.994276299024932, + 13.13251509598922, + 13.274991243029945, + 13.408458571997471, + 13.549371985020116, + 13.686906103044748, + 13.80420408397913, + 13.920975533081219, + 14.042201966047287, + 14.159943904029205, + 14.280679341056384, + 14.404920010012574, + 14.52911576605402, + 14.625227442011237, + 14.687482409994118, + 14.7397521979874, + 14.79190627404023, + 14.842814786010422, + 14.894798476016149, + 14.94684088602662, + 14.998379532014951, + 15.047478700056672 + ], + "chunk_gaps_ms": { + "count": 107, + "p50": 141.2438330007717, + "p95": 146.3823599042371, + "max": 148.3521049376577 + } + }, + { + "start": 754272.738287861, + "first": 754273.362991084, + "last": 754287.363906155, + "end": 754287.364024684, + "elapsed_s": 14.625736822956242, + "chunk_offsets_s": [ + 0.6247032230021432, + 0.7345544659765437, + 1.0205878759734333, + 1.151348061976023, + 1.2805258970474824, + 1.401192725985311, + 1.5295302500016987, + 1.6546626719646156, + 1.78655613202136, + 1.9171192119829357, + 2.0492359689669684, + 2.187878226977773, + 2.32088116905652, + 2.4622209230437875, + 2.6036039090249687, + 2.7447981260484084, + 2.887131297029555, + 3.0275559220463037, + 3.1686953379539773, + 3.3120270300423726, + 3.4532814710401, + 3.5966439800104126, + 3.7387279480462894, + 3.880968254059553, + 4.024881744058803, + 4.168677440029569, + 4.311216437024996, + 4.45349680702202, + 4.596104364027269, + 4.740327759995125, + 4.88425529806409, + 5.027009240002371, + 5.169440348050557, + 5.3148169060004875, + 5.453288394026458, + 5.597195866983384, + 5.743057598010637, + 5.887762821977958, + 6.031440485036001, + 6.173903304967098, + 6.318094347952865, + 6.46586140897125, + 6.612006725976244, + 6.76034719904419, + 6.907412732951343, + 7.051652800990269, + 7.197020655032247, + 7.343172227963805, + 7.489497339003719, + 7.6303376000141725, + 7.77137871098239, + 7.911361780017614, + 8.052596139954403, + 8.188290850026533, + 8.331417548004538, + 8.474819032009691, + 8.61933948600199, + 8.763991701998748, + 8.907573460019194, + 9.049356129020452, + 9.18523072800599, + 9.326547233038582, + 9.467917626956478, + 9.610486782970838, + 9.754232189967297, + 9.894573358003981, + 10.033779012039304, + 10.175168573972769, + 10.316379638039507, + 10.457678301958367, + 10.60332422296051, + 10.745986963971518, + 10.889104236965068, + 11.027504683006555, + 11.171275706961751, + 11.309396742028184, + 11.450256075011566, + 11.591601027990691, + 11.728468531975523, + 11.868628449970856, + 12.010547535959631, + 12.148415882023983, + 12.28530544997193, + 12.428229953045957, + 12.576177141978405, + 12.715624014032073, + 12.854612785042264, + 12.994639870012179, + 13.132677529007196, + 13.275552400038578, + 13.408754494041204, + 13.549370576976798, + 13.687057111994363, + 13.804710105061531, + 13.921332831960171, + 14.042613122030161, + 14.160461541032419, + 14.28115752094891, + 14.405982146970928, + 14.529452456976287, + 14.625618293997832 + ], + "chunk_gaps_ms": { + "count": 100, + "p50": 141.34235348319635, + "p95": 146.3251110399142, + "max": 286.03340999688953 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 85.1, + "per_stream_tps": 11.4, + "ttft_s_max": 2.15, + "timing_valid": true, + "end_to_end_tps": 84.88056809998638, + "makespan_s": 24.12801947304979, + "steady_tps": 91.5, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 85.1216537893112, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 24.12801947304979, + "end_to_end_tps": 84.88056809998638, + "timing_valid": true, + "steady_tps": 91.45526094887931, + "per_stream_tps": [ + 11.4, + 12.4, + 11.0, + 11.5, + 10.7, + 11.7, + 11.4, + 11.4 + ], + "ttft_s": [ + 1.18, + 1.48, + 0.16, + 0.89, + 0.38, + 2.15, + 1.8, + 0.62 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0a24c104609f", + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "28cedcb814fc", + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "22e8f3f09dfa", + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c28174794bf0", + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c6d6ec2a35a8", + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "1a4e0d70467b", + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66" + } + ], + "client_intervals": [ + { + "start": 754287.791341908, + "first": 754288.969093765, + "last": 754311.245095437, + "end": 754311.245359872, + "elapsed_s": 23.454017963958904, + "chunk_offsets_s": [ + 1.1777518569724634, + 1.3207554009277374, + 1.645292998990044, + 1.9947778009809554, + 2.3518903999356553, + 2.562262989929877, + 2.771995108923875, + 2.9745993399992585, + 3.1810874299844727, + 3.3884365999838337, + 3.601474781986326, + 3.8134257060009986, + 4.0248348959721625, + 4.235704113962129, + 4.443797915009782, + 4.65617948200088, + 4.872020888957195, + 5.097554256906733, + 5.317293986910954, + 5.533506501931697, + 5.752018764964305, + 5.97819149796851, + 6.198197150952183, + 6.422276520985179, + 6.644697938929312, + 6.8653908299747854, + 7.078260609996505, + 7.297921716002747, + 7.508893605903722, + 7.726782822981477, + 7.944854666013271, + 8.166203194996342, + 8.382915664929897, + 8.599379348917864, + 8.820825494942255, + 9.042971998918802, + 9.26195013790857, + 9.486793434014544, + 9.709952298901044, + 9.93326558091212, + 10.157219588989392, + 10.381449055974372, + 10.609958897926845, + 10.836427696980536, + 11.064430268947035, + 11.288554438971914, + 11.513606456923299, + 11.738987965974957, + 11.960614159936085, + 12.184700681013055, + 12.402461800957099, + 12.619715442997403, + 12.839376307907514, + 13.059405815904029, + 13.277568890945986, + 13.486587321967818, + 13.708920066943392, + 13.93164937500842, + 14.152886397903785, + 14.370428027003072, + 14.59236756793689, + 14.814668183913454, + 15.030635303002782, + 15.249081051908433, + 15.457939369953237, + 15.674733823980205, + 15.890218026004732, + 16.106551692006178, + 16.324245737981983, + 16.549238394014537, + 16.77145451400429, + 16.989011886995286, + 17.208940400974825, + 17.433612301945686, + 17.65332809498068, + 17.87371771701146, + 18.092175097903237, + 18.30775181192439, + 18.529257732909173, + 18.743752116919495, + 18.9578491859138, + 19.1736926069716, + 19.386259501916356, + 19.604707379010506, + 19.81705543899443, + 20.03855014394503, + 20.260249089915305, + 20.477944782935083, + 20.693609673995525, + 20.90766394196544, + 21.125233138911426, + 21.34882634191308, + 21.580108445952646, + 21.807131008943543, + 22.027791352942586, + 22.23210049199406, + 22.440336099010892, + 22.643485707929358, + 22.834410596988164, + 23.01618057198357, + 23.17485059192404, + 23.31736240000464, + 23.453753529000096 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 218.44681299990043, + "p95": 228.0025719664991, + "max": 357.11259895469993 + } + }, + { + "start": 754287.792265532, + "first": 754289.269209589, + "last": 754309.818721641, + "end": 754309.818816714, + "elapsed_s": 22.026551182032563, + "chunk_offsets_s": [ + 1.4769440570380539, + 1.644494111998938, + 1.9933651000028476, + 2.351068105082959, + 2.561143268016167, + 2.7706836740253493, + 2.973762757028453, + 3.1799965570680797, + 3.3876719860127196, + 3.600646311067976, + 3.812133182073012, + 4.023724439088255, + 4.234654457075521, + 4.442924931063317, + 4.654981776024215, + 4.870584125048481, + 5.096436646999791, + 5.315757445991039, + 5.532069385051727, + 5.751620522001758, + 5.976649948046543, + 6.19708819699008, + 6.420964909018949, + 6.644484674092382, + 6.8641939560184255, + 7.077301801997237, + 7.2972633900353685, + 7.508576355059631, + 7.725728894001804, + 7.944544344092719, + 8.165959177073091, + 8.382219995022751, + 8.598501374013722, + 8.819959935033694, + 9.042153192101978, + 9.261233492055908, + 9.485933267045766, + 9.708838353049941, + 9.932296581100672, + 10.156138844089583, + 10.380809275084175, + 10.60897322499659, + 10.835702090058476, + 11.063462997088209, + 11.287676031002775, + 11.512739617028274, + 11.73810347809922, + 11.95986519404687, + 12.18384806602262, + 12.401572081027552, + 12.618496361072175, + 12.83842262008693, + 13.058435455081053, + 13.276527874055319, + 13.485853268066421, + 13.707795241032727, + 13.930776168010198, + 14.151996614062227, + 14.369396817986853, + 14.591377784032375, + 14.813778977026232, + 15.030181907000951, + 15.24822651699651, + 15.457216836046427, + 15.673862344003282, + 15.888734525069594, + 16.106436795089394, + 16.32405976904556, + 16.549189674085937, + 16.771208912017755, + 16.988411898026243, + 17.20830250706058, + 17.43235963501502, + 17.653024636092596, + 17.87309465603903, + 18.091581396991387, + 18.30712814303115, + 18.528422654024325, + 18.74270790000446, + 18.957715330063365, + 19.172718759044074, + 19.385525400051847, + 19.603465432068333, + 19.816076887072995, + 20.037782953004353, + 20.25895778206177, + 20.47700167109724, + 20.692629681085236, + 20.906875392072834, + 21.123959638993256, + 21.347821277100593, + 21.579092662082985, + 21.805891141993925, + 22.02645610901527 + ], + "chunk_gaps_ms": { + "count": 93, + "p50": 219.320798991248, + "p95": 227.76090702973306, + "max": 357.7030050801113 + } + }, + { + "start": 754287.791739607, + "first": 754287.953206567, + "last": 754311.108859238, + "end": 754311.109266138, + "elapsed_s": 23.3175265310565, + "chunk_offsets_s": [ + 0.16146695998031646, + 0.22143726702779531, + 0.46107919595669955, + 0.7316097900038585, + 1.0187624719692394, + 1.3204354459885508, + 1.6446625939570367, + 1.993513389956206, + 2.3508337349630892, + 2.561440023011528, + 2.7708781879628077, + 2.9739628869574517, + 3.18008311802987, + 3.3882388550555333, + 3.6006607270101085, + 3.8122992319986224, + 4.024015001952648, + 4.234922572039068, + 4.443513609003276, + 4.655275171040557, + 4.870876224013045, + 5.096481272019446, + 5.315879447036423, + 5.5324007159797475, + 5.751756571000442, + 5.977589093032293, + 6.197501993039623, + 6.421418210025877, + 6.643988413969055, + 6.864850057987496, + 7.0775067480280995, + 7.297418704023585, + 7.508750293054618, + 7.726546933990903, + 7.944203124963678, + 8.165476597030647, + 8.381590710021555, + 8.598725807969458, + 8.820255554048344, + 9.042053623008542, + 9.261163347051479, + 9.48600689903833, + 9.709932763013057, + 9.932736265007406, + 10.15616501995828, + 10.38065570499748, + 10.60899060103111, + 10.83563901798334, + 11.063439444988035, + 11.287835360970348, + 11.512800498050638, + 11.738016661023721, + 11.959961066953838, + 12.18379067396745, + 12.401594848954119, + 12.618533208966255, + 12.83837156300433, + 13.058658306021243, + 13.276639778981917, + 13.486043717944995, + 13.707980426959693, + 13.931572190951556, + 14.151743844035082, + 14.369630660978146, + 14.591332615003921, + 14.814529896015301, + 15.02995702601038, + 15.24828811804764, + 15.456824959954247, + 15.674561438965611, + 15.888950943015516, + 16.106275305966847, + 16.324100153055042, + 16.549016696983017, + 16.771359330043197, + 16.988711740006693, + 17.208662686985917, + 17.432824391988106, + 17.653260063030757, + 17.873517923988402, + 18.091875463956967, + 18.30775493301917, + 18.529132451978512, + 18.743601397029124, + 18.957555775996298, + 19.17358356702607, + 19.38620153395459, + 19.60389607597608, + 19.81695375905838, + 20.03852584003471, + 20.25992387102451, + 20.477782621979713, + 20.693391129025258, + 20.907483348972164, + 21.124618686037138, + 21.348726548952982, + 21.579821484046988, + 21.806654157000594, + 22.027347828960046, + 22.231765146018006, + 22.44021181098651, + 22.643366234959103, + 22.834223108016886, + 23.016340045956895, + 23.174565133987926, + 23.317119630984962 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 219.10972404293716, + "p95": 270.53059404715896, + "max": 357.32034500688314 + } + }, + { + "start": 754287.792202028, + "first": 754288.681970811, + "last": 754310.807772754, + "end": 754310.808515769, + "elapsed_s": 23.016313740983605, + "chunk_offsets_s": [ + 0.8897687830030918, + 1.0187972079729661, + 1.320096658077091, + 1.644479374983348, + 1.9938160320743918, + 2.3514696600614116, + 2.561263859970495, + 2.770655849017203, + 2.973875925061293, + 3.1800339330220595, + 3.3880148680182174, + 3.6000421450007707, + 3.8126987230498344, + 4.023270578007214, + 4.234752585995011, + 4.443144613062032, + 4.655088784056716, + 4.870594124076888, + 5.096610759967007, + 5.31592968699988, + 5.532463115989231, + 5.751858155010268, + 5.976844941964373, + 6.196796769974753, + 6.42047008802183, + 6.644863364985213, + 6.864214948029257, + 7.077779502025805, + 7.296634280006401, + 7.508917749975808, + 7.7265950290020555, + 7.944851353997365, + 8.16583997500129, + 8.381540340022184, + 8.59893654496409, + 8.82006822398398, + 9.042014134000055, + 9.261359204072505, + 9.485894993995316, + 9.709516135044396, + 9.93219136307016, + 10.15651987097226, + 10.381025179987773, + 10.608991128974594, + 10.835730138001963, + 11.063765863073058, + 11.28787081700284, + 11.51308218005579, + 11.737997043994255, + 11.960132236010395, + 12.184104420011863, + 12.401521728024818, + 12.618956828024238, + 12.838565835962072, + 13.058398158056661, + 13.276675026048906, + 13.486013620975427, + 13.708163083996624, + 13.93128678004723, + 14.152333769015968, + 14.369526850990951, + 14.591472567990422, + 14.813753216061741, + 15.02970998303499, + 15.248256084974855, + 15.457472982001491, + 15.674505213974044, + 15.888914078008384, + 16.106469291029498, + 16.324331931071356, + 16.549656830029562, + 16.771307184011675, + 16.988705691997893, + 17.208758639055304, + 17.43318002601154, + 17.65301465196535, + 17.873202127986588, + 18.09159213199746, + 18.30704295798205, + 18.52912521897815, + 18.74282001296524, + 18.958262694068253, + 19.173473036964424, + 19.385984122985974, + 19.6041238210164, + 19.81642217002809, + 20.037384837050922, + 20.259542379062623, + 20.477659996016882, + 20.693162742070854, + 20.907074752962217, + 21.124550812062807, + 21.348435217980295, + 21.57964778598398, + 21.80657415499445, + 22.02713137899991, + 22.231838201987557, + 22.439875327050686, + 22.642826262046583, + 22.834168163011782, + 23.01557072601281 + ], + "chunk_gaps_ms": { + "count": 100, + "p50": 218.70043996023014, + "p95": 228.03572507109493, + "max": 357.6536279870197 + } + }, + { + "start": 754287.791675271, + "first": 754288.171228747, + "last": 754311.918906353, + "end": 754311.91927474, + "elapsed_s": 24.127599469036795, + "chunk_offsets_s": [ + 0.37955347599927336, + 0.46119340404402465, + 0.7317267670296133, + 1.0188795770518482, + 1.3205499269533902, + 1.6447910260176286, + 1.9936254379572347, + 2.351118825026788, + 2.561549350968562, + 2.7708468589698896, + 2.9739725020481274, + 3.180096845026128, + 3.388199766050093, + 3.600770471035503, + 3.8122840150026605, + 4.023953496012837, + 4.234940283000469, + 4.443414918961935, + 4.655262897955254, + 4.870820223004557, + 5.0964051419869065, + 5.315853270003572, + 5.532612669980153, + 5.751579705043696, + 5.977324769948609, + 6.197046019951813, + 6.421198031050153, + 6.644631427014247, + 6.8643453969853, + 7.077486154972576, + 7.297370975022204, + 7.508478082017973, + 7.726536308997311, + 7.944141475949436, + 8.165484179975465, + 8.381611236953177, + 8.598745199968107, + 8.820191569044255, + 9.04205861396622, + 9.261172131053172, + 9.486000083037652, + 9.709257093025371, + 9.932408662047237, + 10.156445886008441, + 10.38084695499856, + 10.609193242969923, + 10.83576343394816, + 11.063653382007033, + 11.287974577979185, + 11.513201461057179, + 11.738299846998416, + 11.960149851976894, + 12.184190260944888, + 12.401865283027291, + 12.618641081964597, + 12.838561436976306, + 13.05864542501513, + 13.27664640196599, + 13.486032868968323, + 13.708208604948595, + 13.93065450200811, + 14.151962038013153, + 14.369742276961915, + 14.591453608009033, + 14.814198564039543, + 15.030153859057464, + 15.248522870941088, + 15.456943201017566, + 15.673921816982329, + 15.889066752046347, + 16.106283801957034, + 16.324101289035752, + 16.54917068197392, + 16.771295840037055, + 16.988730203942396, + 17.208449035999365, + 17.432657125988044, + 17.653471807949245, + 17.873276656959206, + 18.091796998050995, + 18.307886646012776, + 18.529078659019433, + 18.743571107974276, + 18.95785004994832, + 19.173607295029797, + 19.38638262299355, + 19.604121261043474, + 19.816904605948366, + 20.038319085957482, + 20.259729853016324, + 20.477954639005475, + 20.693595897988416, + 20.907639494049363, + 21.1248308790382, + 21.34888832597062, + 21.580110286013223, + 21.807214737054892, + 22.027123010950163, + 22.232545071979985, + 22.44031149102375, + 22.64339545101393, + 22.8348241209751, + 23.016577007016167, + 23.17489425605163, + 23.31787050096318, + 23.453658169019036, + 23.576750888954848, + 23.695256542996503, + 23.805904158041812, + 23.91082300094422, + 24.003654340980574, + 24.076070872019045, + 24.127231081947684 + ], + "chunk_gaps_ms": { + "count": 112, + "p50": 218.02960196509957, + "p95": 270.5333629855886, + "max": 357.49338706955314 + } + }, + { + "start": 754287.794662514, + "first": 754289.941962137, + "last": 754311.702534336, + "end": 754311.702613809, + "elapsed_s": 23.907951295026578, + "chunk_offsets_s": [ + 2.1472996230004355, + 2.3482780309859663, + 2.558594972011633, + 2.7683617159491405, + 2.971477536018938, + 3.1772179070394486, + 3.385621438967064, + 3.5978279010159895, + 3.8094382459530607, + 4.0210150219500065, + 4.231753646978177, + 4.440639917971566, + 4.652235302957706, + 4.867793731042184, + 5.093603276996873, + 5.312914668000303, + 5.529959782026708, + 5.749327684985474, + 5.9746118820039555, + 6.194146185996942, + 6.41845163796097, + 6.641286612953991, + 6.861974208033644, + 7.075090902042575, + 7.294786166981794, + 7.506274237995967, + 7.723410296952352, + 7.94198793603573, + 8.163309153052978, + 8.379785572993569, + 8.596528155962005, + 8.817830492043868, + 9.039709520991892, + 9.25873614102602, + 9.483570285025053, + 9.706913455040194, + 9.930026159971021, + 10.15367603697814, + 10.377693213988096, + 10.606250751996413, + 10.832665501977317, + 11.060851020971313, + 11.285095512052067, + 11.51006034493912, + 11.735224268049933, + 11.95698718400672, + 12.181078745052218, + 12.399096745997667, + 12.616009810008109, + 12.83578405098524, + 13.056118938024156, + 13.273923657950945, + 13.483199788024649, + 13.705355650978163, + 13.928495650994591, + 14.149235788965598, + 14.367017676006071, + 14.588895377004519, + 14.810908247018233, + 15.027747004991397, + 15.245690701995045, + 15.454532250994816, + 15.671134702977724, + 15.88672365900129, + 16.103747186949477, + 16.321620962000452, + 16.546533506014384, + 16.768522055004723, + 16.986064051976427, + 17.20532353594899, + 17.4297354669543, + 17.650072416989133, + 17.87052155996207, + 18.088752651005052, + 18.304647992015816, + 18.52631717803888, + 18.74063508096151, + 18.955163961974904, + 19.170503858011216, + 19.383078561048023, + 19.600842735962942, + 19.813814881956205, + 20.035757606034167, + 20.25670710601844, + 20.47504355700221, + 20.690649535041302, + 20.9044158810284, + 21.122054358012974, + 21.345867931027897, + 21.577007377985865, + 21.803868195042014, + 22.02390386199113, + 22.22952166898176, + 22.43736770399846, + 22.64064256194979, + 22.832077775965445, + 23.013052735012025, + 23.17169086704962, + 23.314567239023745, + 23.450629404978827, + 23.573655404965393, + 23.692213619942777, + 23.80287157103885, + 23.907871822011657 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 217.87377505097538, + "p95": 225.80954595468938, + "max": 231.13944695796818 + } + }, + { + "start": 754287.792623615, + "first": 754289.594887918, + "last": 754311.867631982, + "end": 754311.86776536, + "elapsed_s": 24.0751417450374, + "chunk_offsets_s": [ + 1.8022643029689789, + 1.9931504590203986, + 2.3514366049785167, + 2.561022531008348, + 2.77097190101631, + 2.973598851938732, + 3.1798407159512863, + 3.3874193120282143, + 3.5996872460236773, + 3.8121824309928343, + 4.0233324189903215, + 4.234093428007327, + 4.442766626016237, + 4.654796413960867, + 4.870508747990243, + 5.0961547730257735, + 5.315764758037403, + 5.53207213699352, + 5.751402248046361, + 5.976473066955805, + 6.196679953951389, + 6.420800043968484, + 6.643732939031906, + 6.86346871801652, + 7.077190808951855, + 7.296600839938037, + 7.508011533995159, + 7.726075120968744, + 7.943626496009529, + 8.165003312984481, + 8.381238530972041, + 8.598618719028309, + 8.819952048012055, + 9.041990358033217, + 9.26099304202944, + 9.485546175041236, + 9.708533694967628, + 9.932021538959816, + 10.155978378956206, + 10.380367751000449, + 10.608645429951139, + 10.83538371999748, + 11.063181875040755, + 11.287356444983743, + 11.512565008015372, + 11.737791251041926, + 11.959543030941859, + 12.183523006970063, + 12.400740906014107, + 12.617596433032304, + 12.83756390796043, + 13.057490983046591, + 13.27565330604557, + 13.484851515037008, + 13.706973826047033, + 13.929771263035946, + 14.15130401600618, + 14.36898998299148, + 14.591105061001144, + 14.812990105943754, + 15.029743231949396, + 15.247907825978473, + 15.456458444939926, + 15.67325193900615, + 15.888715405017138, + 16.105897319037467, + 16.323374610976316, + 16.548994201002643, + 16.771148575935513, + 16.988322152988985, + 17.208123513963073, + 17.432508997037075, + 17.652625304996036, + 17.87250169098843, + 18.091127136955038, + 18.306801963946782, + 18.528618559939787, + 18.74254215497058, + 18.95728358195629, + 19.172886200947687, + 19.385474679991603, + 19.603481095982715, + 19.81611325603444, + 20.036893217009492, + 20.258513665990904, + 20.476595746935345, + 20.69258334499318, + 20.90623634599615, + 21.12380675796885, + 21.347593466984108, + 21.578789026942104, + 21.80599672696553, + 22.02602684102021, + 22.231171875959262, + 22.439011959941126, + 22.642244864953682, + 22.83320311503485, + 23.01493692898657, + 23.17360723693855, + 23.316130726016127, + 23.452512367046438, + 23.575469741947018, + 23.694164101965725, + 23.80478593998123, + 23.9098008639412, + 24.002440346986987, + 24.075008366955444 + ], + "chunk_gaps_ms": { + "count": 106, + "p50": 217.8461914882064, + "p95": 226.73829004634172, + "max": 358.28614595811814 + } + }, + { + "start": 754287.791255267, + "first": 754288.411138358, + "last": 754310.808708122, + "end": 754310.808835628, + "elapsed_s": 23.017580360989086, + "chunk_offsets_s": [ + 0.6198830909561366, + 0.7322162430500612, + 1.0196677120402455, + 1.3210897559765726, + 1.6454753370489925, + 1.9949124900158495, + 2.3525609499774873, + 2.5623065260006115, + 2.771888004965149, + 2.9747262849705294, + 3.180936021031812, + 3.3889939970104024, + 3.6014220140641555, + 3.813477050047368, + 4.024302075966261, + 4.235272879013792, + 4.443724057986401, + 4.656093450030312, + 4.871830343967304, + 5.0972712150542066, + 5.31696811306756, + 5.532981456955895, + 5.752150350017473, + 5.978122874046676, + 6.197503512958065, + 6.421777380048297, + 6.64573433296755, + 6.86471904895734, + 7.077862735022791, + 7.2974958079867065, + 7.509465530980378, + 7.727229755022563, + 7.944614327978343, + 8.16585637605749, + 8.381985545041971, + 8.599320309003815, + 8.820797607069835, + 9.042583786998875, + 9.261699112015776, + 9.486379318055697, + 9.709756760974415, + 9.932878841995262, + 10.157248246017843, + 10.381316062994301, + 10.609733119956218, + 10.836222831043415, + 11.064142395043746, + 11.288457317976281, + 11.51357240905054, + 11.738685755059123, + 11.96052432002034, + 12.184574440005235, + 12.402227303013206, + 12.619121229974553, + 12.838909712038003, + 13.05901691596955, + 13.277243447955698, + 13.486385736963712, + 13.708544800058007, + 13.931401949026622, + 14.152277352986857, + 14.369902519043535, + 14.5919167479733, + 14.81405918695964, + 15.030519734020345, + 15.248898235033266, + 15.457407189067453, + 15.67429217195604, + 15.88952278799843, + 16.10722451400943, + 16.324789359001443, + 16.54980462405365, + 16.772159095969982, + 16.98934053000994, + 17.209248916013166, + 17.433607373968698, + 17.65382801904343, + 17.874063784023747, + 18.092410892015323, + 18.308200953062624, + 18.530026332009584, + 18.743625411996618, + 18.958349782973528, + 19.174106643069535, + 19.386760098976083, + 19.604734931024723, + 19.817401746986434, + 20.038698593038134, + 20.26060077303555, + 20.478643606067635, + 20.694146926980466, + 20.908097386010922, + 21.125403620069847, + 21.349342330009677, + 21.580740196048282, + 21.80709548899904, + 22.028199933003634, + 22.23209967603907, + 22.44104242604226, + 22.644243826973252, + 22.834848538041115, + 23.01745285501238 + ], + "chunk_gaps_ms": { + "count": 101, + "p50": 218.98471598979086, + "p95": 231.39786603860557, + "max": 357.64845996163785 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 70.4, + "per_stream_tps": 70.4, + "ttft_s_max": 0.11, + "timing_valid": true, + "end_to_end_tps": 68.62781851106324, + "makespan_s": 3.7302657370455563, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 70.43615852994674, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 3.7302657370455563, + "end_to_end_tps": 68.62781851106324, + "timing_valid": true, + "per_stream_tps": [ + 70.4 + ], + "ttft_s": [ + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 754327.422224281, + "first": 754327.532020543, + "last": 754331.152320192, + "end": 754331.152490018, + "elapsed_s": 3.7302657370455563, + "chunk_offsets_s": [ + 0.10979626199696213, + 0.17016933800186962, + 0.22739757096860558, + 0.28817855496890843, + 0.35475255199708045, + 0.42127459694165736, + 0.4889442679705098, + 0.5548105619382113, + 0.6191591790411621, + 0.6852498600492254, + 0.7516740310238674, + 0.818145371042192, + 0.8835623339982703, + 0.9482483939500526, + 1.0135633549652994, + 1.079344033030793, + 1.145230680005625, + 1.2097944660345092, + 1.2674413259373978, + 1.3181879799813032, + 1.369561581988819, + 1.4235062489751726, + 1.4839969580061734, + 1.5435671800514683, + 1.6018943339586258, + 1.6609522140352055, + 1.7263235120335594, + 1.791178229963407, + 1.8542516030138358, + 1.9154181589838117, + 1.9976478499593213, + 2.078564745024778, + 2.1609531739959493, + 2.2460583630017936, + 2.3261965150013566, + 2.4127488220110536, + 2.49637343804352, + 2.5815584199735895, + 2.6657283999957144, + 2.750797205022536, + 2.834499342017807, + 2.9197181799681857, + 3.0036737750051543, + 3.0895177549682558, + 3.172975825960748, + 3.2570837900275365, + 3.3410085040377453, + 3.4254183910088614, + 3.5091634560376406, + 3.5906018919777125, + 3.673612901940942, + 3.730095910956152 + ], + "chunk_gaps_ms": { + "count": 51, + "p50": 66.42417097464204, + "p95": 85.21883795037866, + "max": 86.55230700969696 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 81.7, + "per_stream_tps": 41.6, + "ttft_s_max": 0.27, + "timing_valid": true, + "end_to_end_tps": 80.65530033383348, + "makespan_s": 6.348001902922988, + "steady_tps": 83.5, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 81.74826717278044, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 6.348001902922988, + "end_to_end_tps": 80.65530033383348, + "timing_valid": true, + "steady_tps": 83.49073345659595, + "per_stream_tps": [ + 42.0, + 41.2 + ], + "ttft_s": [ + 0.27, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 754331.154278498, + "first": 754331.42894949, + "last": 754337.501804604, + "end": 754337.501961006, + "elapsed_s": 6.347682507941499, + "chunk_offsets_s": [ + 0.2746709919301793, + 0.35677349800243974, + 0.4421766099985689, + 0.5269440039992332, + 0.6174789059441537, + 0.7067504050210118, + 0.7963646759744734, + 0.8829173019621521, + 0.9665397899225354, + 1.0496837289538234, + 1.1368002960225567, + 1.2223686740035191, + 1.31392315402627, + 1.4033502379897982, + 1.4922286929795519, + 1.5835173790110275, + 1.67098596598953, + 1.7634966459590942, + 1.8540578050306067, + 1.9458277740050107, + 2.0372994209174067, + 2.1266103599919006, + 2.2121977779315785, + 2.3007670789957047, + 2.3891218489734456, + 2.475882877013646, + 2.559311506920494, + 2.644473352935165, + 2.7291510179638863, + 2.8140061729354784, + 2.8980714479694143, + 2.9839408359257504, + 3.069306844030507, + 3.1552192399976775, + 3.243894669925794, + 3.332710883929394, + 3.4163008119212463, + 3.502832205966115, + 3.5909409419400617, + 3.680492763989605, + 3.7662591269472614, + 3.852796745020896, + 3.94091240898706, + 4.027896958985366, + 4.114875957020558, + 4.203074421966448, + 4.29168733896222, + 4.37897190800868, + 4.465793703915551, + 4.552852878929116, + 4.640479931025766, + 4.727476369007491, + 4.814438182977028, + 4.90256971900817, + 4.99020582693629, + 5.0778187030227855, + 5.165379353915341, + 5.253446858027019, + 5.341342728002928, + 5.427928778924979, + 5.515357428928837, + 5.604010122013278, + 5.692739903926849, + 5.77954870800022, + 5.866377305006608, + 5.952679881011136, + 6.036990982014686, + 6.121152817970142, + 6.207689699949697, + 6.290630085975863, + 6.347526105935685 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 87.02780649764463, + "p95": 91.47164691239595, + "max": 92.51067996956408 + } + }, + { + "start": 754331.153959103, + "first": 754331.263140255, + "last": 754337.445154586, + "end": 754337.445262859, + "elapsed_s": 6.29130375594832, + "chunk_offsets_s": [ + 0.10918115195818245, + 0.16895073500927538, + 0.357405168004334, + 0.44269517494831234, + 0.5277291309321299, + 0.6180702080018818, + 0.7072488259291276, + 0.7967647429322824, + 0.8834544270066544, + 0.9669664499815553, + 1.0501413010060787, + 1.1372907490003854, + 1.2229250469245017, + 1.314399477909319, + 1.4038101299665868, + 1.492415574961342, + 1.583612451911904, + 1.671768756932579, + 1.764035066938959, + 1.8545881619211286, + 1.9463558749994263, + 2.0378008499974385, + 2.1270661569433287, + 2.212593877920881, + 2.301276442944072, + 2.389571084990166, + 2.4765292030060664, + 2.559845063951798, + 2.644990413915366, + 2.7295560939237475, + 2.814492384903133, + 2.898583116941154, + 2.98454688990023, + 3.0697713919216767, + 3.1557907819515094, + 3.2444194270065054, + 3.333208936965093, + 3.416888672974892, + 3.503484867978841, + 3.591398049960844, + 3.6813653159188107, + 3.766913676983677, + 3.852947241975926, + 3.941400990006514, + 4.028502916917205, + 4.115579755976796, + 4.203689147951081, + 4.292172143934295, + 4.379503320902586, + 4.466369629953988, + 4.553471124963835, + 4.6409676959738135, + 4.727954805945046, + 4.81496241199784, + 4.903138012974523, + 4.990725239971653, + 5.078508517006412, + 5.1658739189151675, + 5.253894269932061, + 5.341979134012945, + 5.42843563191127, + 5.515949865919538, + 5.6045687509467825, + 5.693282116902992, + 5.7796467889565974, + 5.866487929946743, + 5.952813289943151, + 6.037522842991166, + 6.121262643951923, + 6.208227688912302, + 6.291195482946932 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 87.12568745249882, + "p95": 91.47443098481745, + "max": 188.4544329950586 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 115.1, + "per_stream_tps": 30.0, + "ttft_s_max": 0.68, + "timing_valid": true, + "end_to_end_tps": 114.07378265210859, + "makespan_s": 8.976646309019998, + "steady_tps": 121.0, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 115.06327027634916, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 8.976646309019998, + "end_to_end_tps": 114.07378265210859, + "timing_valid": true, + "steady_tps": 120.99711299317494, + "per_stream_tps": [ + 30.7, + 29.8, + 30.2, + 29.6 + ], + "ttft_s": [ + 0.68, + 0.28, + 0.47, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 754337.506581624, + "first": 754338.186625799, + "last": 754346.481646862, + "end": 754346.481763951, + "elapsed_s": 8.97518232697621, + "chunk_offsets_s": [ + 0.6800441750092432, + 0.8007357829483226, + 0.9251843369565904, + 1.0475369199411944, + 1.168800837942399, + 1.2867736209882423, + 1.401786808972247, + 1.51629175292328, + 1.6312790849478915, + 1.7470501519273967, + 1.8697828350123018, + 1.9970555739710107, + 2.124484186992049, + 2.2500363669823855, + 2.376639100955799, + 2.5019914709264413, + 2.6285510199377313, + 2.7568702169228345, + 2.8787302839336917, + 3.0052275439957157, + 3.1330651849275455, + 3.257601691992022, + 3.3804708709940314, + 3.501278384006582, + 3.6194434089120477, + 3.736396853928454, + 3.850916965981014, + 3.961962325964123, + 4.077262748964131, + 4.196478534955531, + 4.31289503199514, + 4.430994791910052, + 4.549257033970207, + 4.667626412934624, + 4.784268175950274, + 4.901981355971657, + 5.02275003597606, + 5.144345539971255, + 5.265708321938291, + 5.386750652920455, + 5.505943558993749, + 5.623990069958381, + 5.743120479979552, + 5.866536304005422, + 5.989934095996432, + 6.111436382983811, + 6.232079925946891, + 6.35324619396124, + 6.472751614986919, + 6.5886187149444595, + 6.708299272926524, + 6.829731895006262, + 6.9505444639362395, + 7.072503250907175, + 7.19421651493758, + 7.315382511005737, + 7.4328460089163855, + 7.551711551961489, + 7.671393501921557, + 7.790934394928627, + 7.911749379942194, + 8.033637446002103, + 8.153285523992963, + 8.26808531791903, + 8.385309229954146, + 8.502961401944049, + 8.618472529924475, + 8.731698724906892, + 8.833933188929223, + 8.917779534007423, + 8.975065237958916 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 119.68125397106633, + "p95": 127.27273895870894, + "max": 128.3191969851032 + } + }, + { + "start": 754337.50546427, + "first": 754337.784981768, + "last": 754346.340425276, + "end": 754346.340545198, + "elapsed_s": 8.83508092793636, + "chunk_offsets_s": [ + 0.27951749798376113, + 0.360694266972132, + 0.5751480549806729, + 0.8016546069411561, + 0.9259929049294442, + 1.0483630560338497, + 1.1695624759886414, + 1.2873626180225983, + 1.4032094939611852, + 1.5172879060264677, + 1.6321240520337597, + 1.748096689931117, + 1.8706518179969862, + 1.9979560460196808, + 2.1255038279341534, + 2.250884567038156, + 2.3773479389492422, + 2.5026001159567386, + 2.6293637950439006, + 2.7576013919897377, + 2.879491297993809, + 3.006084799999371, + 3.133632373996079, + 3.258068431983702, + 3.381464911974035, + 3.5019592859316617, + 3.62033738498576, + 3.7371252450393513, + 3.8516655650455505, + 3.962667837040499, + 4.078278005938046, + 4.19744240003638, + 4.313936754013412, + 4.431656638043933, + 4.550045232987031, + 4.668427139986306, + 4.784883333020844, + 4.902671905932948, + 5.023193879984319, + 5.145026601967402, + 5.266604313976131, + 5.387661540997215, + 5.50716088199988, + 5.624757052981295, + 5.7438045500312, + 5.867236934020184, + 5.9906474149320275, + 6.112253173952922, + 6.233069455018267, + 6.35440601198934, + 6.473449188983068, + 6.589990951004438, + 6.709613109007478, + 6.830814625020139, + 6.9516154009616, + 7.073660028981976, + 7.195277068996802, + 7.3165399129502475, + 7.4339155870256945, + 7.552786585991271, + 7.672394871013239, + 7.792087637935765, + 7.912801036960445, + 8.034662318998016, + 8.1544405099703, + 8.26914284704253, + 8.386374550987966, + 8.504023539018817, + 8.619649259955622, + 8.732766686938703, + 8.83496100595221 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 120.136282464955, + "p95": 127.54778191447258, + "max": 226.5065519604832 + } + }, + { + "start": 754337.505912546, + "first": 754337.973808859, + "last": 754346.424697017, + "end": 754346.42481657, + "elapsed_s": 8.918904024059884, + "chunk_offsets_s": [ + 0.4678963130572811, + 0.5751156380865723, + 0.8013526690192521, + 0.9257069820305333, + 1.0482633910141885, + 1.1695856130681932, + 1.2874905070057139, + 1.4022623650962487, + 1.5169092620490119, + 1.631994499010034, + 1.7477549590403214, + 1.8705024730879813, + 1.997778797056526, + 2.1251920980867, + 2.2506676530465484, + 2.3773593630176038, + 2.5027110460214317, + 2.6291104970732704, + 2.757736641098745, + 2.879643732099794, + 3.005976287066005, + 3.13397004106082, + 3.258059984073043, + 3.3813399190548807, + 3.5018958770669997, + 3.6200670620892197, + 3.737274974002503, + 3.8515464439988136, + 3.9626668450655416, + 4.077966515091248, + 4.197202750016004, + 4.313620991073549, + 4.4317002550233155, + 4.5499651520513, + 4.668244738015346, + 4.784973654081114, + 4.902601698064245, + 5.023366793990135, + 5.144963274011388, + 5.26641829602886, + 5.3873560020001605, + 5.50680217507761, + 5.624621372087859, + 5.743753076996654, + 5.867251767078415, + 5.990560438018292, + 6.112354935030453, + 6.232971342047676, + 6.354299804079346, + 6.473527702037245, + 6.589671140071005, + 6.7090783840976655, + 6.830187579034828, + 6.951306950068101, + 7.073480474995449, + 7.195004058070481, + 7.316228487063199, + 7.433878371026367, + 7.552138884086162, + 7.671867203083821, + 7.791964981006458, + 7.912800029036589, + 8.034529582015239, + 8.154350110096857, + 8.269020222011022, + 8.38616134901531, + 8.504009667085484, + 8.619533210992813, + 8.732829136075452, + 8.834886190015823, + 8.918784470995888 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 119.959153002128, + "p95": 127.4133010301739, + "max": 226.23703093267977 + } + }, + { + "start": 754337.505117642, + "first": 754337.616958786, + "last": 754346.238412575, + "end": 754346.238506783, + "elapsed_s": 8.73338914103806, + "chunk_offsets_s": [ + 0.11184114404022694, + 0.17237126198597252, + 0.3612750730244443, + 0.5757053879788145, + 0.8020715080201626, + 0.9264745899708942, + 1.048779187956825, + 1.1700510899536312, + 1.2879354880424216, + 1.4034559769788757, + 1.5174131239764392, + 1.6322256539715454, + 1.7482152519514784, + 1.8707608280237764, + 1.9980677439598367, + 2.125604070024565, + 2.2511801059590653, + 2.377645094995387, + 2.502998217009008, + 2.6294712050585076, + 2.758023716043681, + 2.879771398031153, + 3.0063896829960868, + 3.1340651150094345, + 3.258476388058625, + 3.3818590600276366, + 3.5024215950397775, + 3.620465610991232, + 3.7375368979992345, + 3.852070400957018, + 3.9630862730555236, + 4.078408424044028, + 4.197540706023574, + 4.3140239550266415, + 4.43206789100077, + 4.5503343410091475, + 4.668726871022955, + 4.785293273977004, + 4.90308047097642, + 5.023618077044375, + 5.145442062988877, + 5.266876477049664, + 5.388059113058262, + 5.507657655049115, + 5.62504908896517, + 5.744085161015391, + 5.867647339007817, + 5.991060251020826, + 6.113395281950943, + 6.23357904504519, + 6.354956945986487, + 6.47401417104993, + 6.590287226019427, + 6.710127450060099, + 6.831076420028694, + 6.951838011969812, + 7.074216915061697, + 7.195531744044274, + 7.316786588984542, + 7.434179382049479, + 7.553072541020811, + 7.672926860977896, + 7.792506329948083, + 7.913274882012047, + 8.035152915981598, + 8.154683712986298, + 8.269604547997005, + 8.386629611020908, + 8.504500088049099, + 8.620068241027184, + 8.73329493298661 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 120.36068458110094, + "p95": 128.55251098517329, + "max": 226.36612004134804 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 151.4, + "per_stream_tps": 20.4, + "ttft_s_max": 1.69, + "timing_valid": true, + "end_to_end_tps": 150.75340645758675, + "makespan_s": 13.585099322954193, + "steady_tps": 165.8, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 151.43408252080238, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 13.585099322954193, + "end_to_end_tps": 150.75340645758675, + "timing_valid": true, + "steady_tps": 165.84959739690183, + "per_stream_tps": [ + 20.8, + 20.1, + 21.4, + 20.2, + 20.5, + 20.3, + 21.1, + 20.1 + ], + "ttft_s": [ + 1.15, + 0.11, + 1.69, + 0.47, + 0.9, + 0.68, + 1.41, + 0.28 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 754346.489961498, + "first": 754347.642880904, + "last": 754359.930594871, + "end": 754359.931185836, + "elapsed_s": 13.44122433802113, + "chunk_offsets_s": [ + 1.1529194060713053, + 1.3076519781025127, + 1.5858289350289851, + 1.8765668050618842, + 2.0587159300921485, + 2.2361829000292346, + 2.413478429079987, + 2.588381184032187, + 2.7677947310730815, + 2.9544893850106746, + 3.1427695740712807, + 3.336109968018718, + 3.532516422099434, + 3.7268664250150323, + 3.924548491020687, + 4.119830774026923, + 4.312290281057358, + 4.50476341904141, + 4.696702599991113, + 4.889678111067042, + 5.080682916101068, + 5.26988442405127, + 5.456585381994955, + 5.635724575025961, + 5.814032880007289, + 5.991291288053617, + 6.164667533012107, + 6.335183576098643, + 6.504055332043208, + 6.67171413206961, + 6.8417412110138685, + 7.015228208038025, + 7.183386806049384, + 7.352332481998019, + 7.5215973450103775, + 7.689485268085264, + 7.858926117070951, + 8.029859235044569, + 8.202365072094835, + 8.372977515100501, + 8.546312335063703, + 8.719328256091103, + 8.891736076097004, + 9.064093047054484, + 9.236827206099406, + 9.411843833047897, + 9.583074714057148, + 9.752205080003478, + 9.922009068075567, + 10.092060098075308, + 10.260411704075523, + 10.429290676023811, + 10.597966492990963, + 10.766788088018075, + 10.938140890095383, + 11.108153216075152, + 11.279463201062754, + 11.45062126510311, + 11.618929815012962, + 11.788387031992897, + 11.958970307023264, + 12.128307522041723, + 12.296579607995227, + 12.46391496504657, + 12.630094344029203, + 12.795467059011571, + 12.951593579025939, + 13.095787158003077, + 13.225596814998426, + 13.339947372092865, + 13.440633373102173 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.27043299842626, + "p95": 196.40645408071578, + "max": 290.7378700328991 + } + }, + { + "start": 754346.48755954, + "first": 754346.600896072, + "last": 754359.284781832, + "end": 754359.284959049, + "elapsed_s": 12.79739950899966, + "chunk_offsets_s": [ + 0.11333653202746063, + 0.17237649206072092, + 0.3616216980153695, + 0.5736104210373014, + 0.798631503013894, + 1.0479034290183336, + 1.3098210220923647, + 1.5878404090180993, + 1.8779957620427012, + 2.0604732100619003, + 2.237887268071063, + 2.4153658060822636, + 2.5901501120533794, + 2.7694858830654994, + 2.9562459620647132, + 3.1446169030386955, + 3.338010641047731, + 3.5344089510617778, + 3.728693418088369, + 3.926292091025971, + 4.121777720050886, + 4.314187305979431, + 4.506932063028216, + 4.698657882050611, + 4.891480623045936, + 5.082687430083752, + 5.271738185081631, + 5.458532519987784, + 5.637621344067156, + 5.815892065060325, + 5.993032328085974, + 6.166519694030285, + 6.337043401086703, + 6.505661393981427, + 6.67360128602013, + 6.844092496088706, + 7.017245203023776, + 7.185047524981201, + 7.354300691979006, + 7.523585219983943, + 7.691483590053394, + 7.8609113349812105, + 8.031681572087109, + 8.204149728058837, + 8.375001198030077, + 8.54822244902607, + 8.721157697029412, + 8.893567709019408, + 9.065724582062103, + 9.238684007083066, + 9.413743035052903, + 9.584392054006457, + 9.753957080072723, + 9.923837212030776, + 10.093749409075826, + 10.262103736051358, + 10.431164468987845, + 10.599855503067374, + 10.769451008993201, + 10.939823801978491, + 11.110169842024334, + 11.281414114986546, + 11.45260226004757, + 11.62084282503929, + 11.790579324006103, + 11.961023797979578, + 12.130337477079593, + 12.298604618990794, + 12.465966920019127, + 12.632111706072465, + 12.79722229205072 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 172.70170198753476, + "p95": 249.2719260044396, + "max": 290.15535302460194 + } + }, + { + "start": 754346.489204755, + "first": 754348.182798444, + "last": 754360.072103914, + "end": 754360.072549086, + "elapsed_s": 13.58334433089476, + "chunk_offsets_s": [ + 1.6935936888912693, + 1.8765511409146711, + 2.059542672941461, + 2.236637111986056, + 2.414463237975724, + 2.5888649160042405, + 2.767961484962143, + 2.9555272669531405, + 3.143755454919301, + 3.337167753954418, + 3.5333448299206793, + 3.727790897944942, + 3.925168191897683, + 4.120662685949355, + 4.3130846719723195, + 4.505708482931368, + 4.697575311991386, + 4.890659719938412, + 5.081675388966687, + 5.270981489913538, + 5.457376108970493, + 5.636517541948706, + 5.814753046957776, + 5.991973583004437, + 6.165458659990691, + 6.3357241249177605, + 6.5048797549679875, + 6.672424890915863, + 6.842653346946463, + 7.015514834900387, + 7.18358885589987, + 7.352854406926781, + 7.521854675957002, + 7.68988440791145, + 7.859225143911317, + 8.029808370978571, + 8.202842467930168, + 8.373305646004155, + 8.546531296917237, + 8.719445937895216, + 8.892662740894593, + 9.064755660947412, + 9.237444683909416, + 9.412349645979702, + 9.583298427984118, + 9.752771180937998, + 9.922615024959669, + 10.092596918926574, + 10.261001741979271, + 10.4299873058917, + 10.59867115598172, + 10.767455886933021, + 10.93909229093697, + 11.10934621898923, + 11.280319736921228, + 11.451572921941988, + 11.62015483493451, + 11.78933913691435, + 11.959281174000353, + 12.128653861000203, + 12.296860474976711, + 12.46410079090856, + 12.630222488893196, + 12.79542949097231, + 12.95275237399619, + 13.097099153907038, + 13.225969602935947, + 13.34027339098975, + 13.442335579893552, + 13.525632687960751, + 13.582899158936925 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 170.96114996820688, + "p95": 194.4460680242628, + "max": 197.37729395274073 + } + }, + { + "start": 754346.488077272, + "first": 754346.956104464, + "last": 754359.585376637, + "end": 754359.585957474, + "elapsed_s": 13.097880201996304, + "chunk_offsets_s": [ + 0.46802719205152243, + 0.5737269190140069, + 0.7983919500838965, + 1.0478394930250943, + 1.3097931190859526, + 1.5880641880212352, + 1.8786196240689605, + 2.06054865103215, + 2.2375361929880455, + 2.4159757640445605, + 2.590034896042198, + 2.768858790048398, + 2.956324523082003, + 3.1451400120276958, + 3.338883370044641, + 3.5345215930137783, + 3.728812652057968, + 3.926385804079473, + 4.121760729001835, + 4.314252763055265, + 4.506803662050515, + 4.698672331054695, + 4.891186269000173, + 5.082364980014972, + 5.2720449570333585, + 5.45812298101373, + 5.63731987006031, + 5.815596815082245, + 5.992551317089237, + 6.166137499036267, + 6.336484196013771, + 6.505180735024624, + 6.67303070507478, + 6.84340209106449, + 7.016583549091592, + 7.184665778069757, + 7.353932369034737, + 7.523020814987831, + 7.690905297989957, + 7.860231362050399, + 8.030789933050983, + 8.203733725007623, + 8.374294360051863, + 8.547520923078991, + 8.720421211095527, + 8.893097721040249, + 9.065440164064057, + 9.238353429012932, + 9.413018389022909, + 9.5841982930433, + 9.753653382067569, + 9.92350647400599, + 10.093453934998251, + 10.261786742019467, + 10.430776369990781, + 10.599481276003644, + 10.768744443077594, + 10.939356422051787, + 11.109854448004626, + 11.281153362011537, + 11.452293634065427, + 11.620518023031764, + 11.789702645037323, + 11.960767699987628, + 12.130157780018635, + 12.298567819059826, + 12.465615830034949, + 12.631716568022966, + 12.797267445013858, + 12.953433435992338, + 13.097299365093932 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 172.50947648426518, + "p95": 249.44754294119775, + "max": 290.5554360477254 + } + }, + { + "start": 754346.489739432, + "first": 754347.394491674, + "last": 754359.829794389, + "end": 754359.82992975, + "elapsed_s": 13.340190317947417, + "chunk_offsets_s": [ + 0.9047522420296445, + 1.0460501320194453, + 1.3080905419774354, + 1.5863657069858164, + 1.876495460048318, + 2.0587474340572953, + 2.2363421650370583, + 2.4136303500272334, + 2.5886922100326046, + 2.767381864017807, + 2.9549101409502327, + 3.142875622957945, + 3.337009415961802, + 3.532690759981051, + 3.726965241949074, + 3.9245784430531785, + 4.1199725839542225, + 4.3124751299619675, + 4.50506564497482, + 4.6969590180087835, + 4.8899394730106, + 5.081028950982727, + 5.2703222839627415, + 5.456919561023824, + 5.635911216028035, + 5.814293971052393, + 5.991404489031993, + 6.164957248023711, + 6.335302856983617, + 6.504050035960972, + 6.671682163956575, + 6.841605689027347, + 7.015367473009974, + 7.183503223001026, + 7.352482771035284, + 7.521702465950511, + 7.689675158006139, + 7.859113606973551, + 8.029898484004661, + 8.202522480976768, + 8.373164620948955, + 8.546568818041123, + 8.719363632961176, + 8.891921998001635, + 9.064281704952009, + 9.236948422971182, + 9.412385341944173, + 9.583112969994545, + 9.752567115006968, + 9.922300909995101, + 10.092202082974836, + 10.260599322034977, + 10.429549877997488, + 10.598280783975497, + 10.766744408058003, + 10.938152841990814, + 11.108479955000803, + 11.279833732987754, + 11.451177941984497, + 11.619515820988454, + 11.788566297967918, + 11.959156851982698, + 12.12834025803022, + 12.296721513033845, + 12.46399143000599, + 12.630228152964264, + 12.795479635009542, + 12.952048302977346, + 13.095865878975019, + 13.226142356055789, + 13.340054956963286 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.3811059598811, + "p95": 197.61320110410452, + "max": 290.12975306250155 + } + }, + { + "start": 754346.487624692, + "first": 754347.167249276, + "last": 754359.715401048, + "end": 754359.715893676, + "elapsed_s": 13.228268984006718, + "chunk_offsets_s": [ + 0.6796245839213952, + 0.7985155989881605, + 1.0477705009980127, + 1.3099091199692339, + 1.588090299977921, + 1.8782492049504071, + 2.0606935640098527, + 2.238120469963178, + 2.415499920025468, + 2.590249617001973, + 2.769893742981367, + 2.956347082974389, + 3.1447312079835683, + 3.3383211409673095, + 3.534542856970802, + 3.728680457919836, + 3.9263210999779403, + 4.121806456940249, + 4.314172793994658, + 4.506959198974073, + 4.6987105389125645, + 4.891541488002986, + 5.082754982984625, + 5.271967820008285, + 5.458749049925245, + 5.637729089939967, + 5.815997603000142, + 5.993050984921865, + 6.166541439015418, + 6.33707880997099, + 6.505733971949667, + 6.673649302916601, + 6.843918974976987, + 7.017137921997346, + 7.1852538789389655, + 7.354329764959402, + 7.523633188917302, + 7.691304932930507, + 7.86064179695677, + 8.031772164977156, + 8.204514979966916, + 8.37479433300905, + 8.548186544911005, + 8.720830127014779, + 8.893651101971045, + 9.0658182320185, + 9.23876381595619, + 9.414643154945225, + 9.584555016015656, + 9.753986936993897, + 9.923919213935733, + 10.093842258909717, + 10.2621874489123, + 10.43148576898966, + 10.599833247018978, + 10.768921996932477, + 10.940164508996531, + 11.110450981999747, + 11.281718086916953, + 11.452920614974573, + 11.62132356595248, + 11.790648172958754, + 11.961169895948842, + 12.130559670971707, + 12.298872653977014, + 12.466145721962675, + 12.632242731982842, + 12.7977591449162, + 12.953972928924486, + 13.098310877918266, + 13.227776355925016 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.71711748233065, + "p95": 249.25490200985223, + "max": 290.1589049724862 + } + }, + { + "start": 754346.491031843, + "first": 754347.905120868, + "last": 754360.014766818, + "end": 754360.014929763, + "elapsed_s": 13.523897920036688, + "chunk_offsets_s": [ + 1.4140890249982476, + 1.584544011973776, + 1.8747874289983884, + 2.0572142850141972, + 2.2343596840510145, + 2.4120490240165964, + 2.5872075100196525, + 2.7666775699472055, + 2.952861738973297, + 3.141190359950997, + 3.3346922759665176, + 3.5310867780353874, + 3.725324252038263, + 3.9228666679700837, + 4.1183548419503495, + 4.310820795944892, + 4.503513311967254, + 4.6952431469690055, + 4.8890264730434865, + 5.079649850958958, + 5.26870377396699, + 5.45558929303661, + 5.634510356001556, + 5.8128094939747825, + 5.990032876958139, + 6.163486658944748, + 6.3340436780126765, + 6.502603222965263, + 6.670313943992369, + 6.840570832951926, + 7.014007813995704, + 7.182130826986395, + 7.351461834972724, + 7.520372694940306, + 7.688255034037866, + 7.857642906019464, + 8.028755018021911, + 8.201263286988251, + 8.371795521001332, + 8.544957524049096, + 8.717892116052099, + 8.890488257049583, + 9.062693531042896, + 9.235806012991816, + 9.411047218949534, + 9.58150641201064, + 9.75120270298794, + 9.920891970046796, + 10.090808582957834, + 10.259137901943177, + 10.427927655982785, + 10.596762754954398, + 10.765554526005872, + 10.936712477938272, + 11.10733364103362, + 11.278590889996849, + 11.449850842938758, + 11.618005151976831, + 11.787205294007435, + 11.957835417008027, + 12.127511963946745, + 12.295554046984762, + 12.462799531989731, + 12.628895965986885, + 12.794440682977438, + 12.950852132984437, + 13.094669756945223, + 13.225228301016614, + 13.338678817031905, + 13.439634612994269, + 13.523734975024126 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.13503196742386, + "p95": 195.48817398026586, + "max": 290.2434170246124 + } + }, + { + "start": 754346.487449763, + "first": 754346.765543951, + "last": 754359.441101665, + "end": 754359.441178209, + "elapsed_s": 12.95372844603844, + "chunk_offsets_s": [ + 0.27809418796095997, + 0.362049077055417, + 0.5740904740523547, + 0.7989639859879389, + 1.0482950969599187, + 1.3104772999649867, + 1.588506462983787, + 1.8789080099668354, + 2.061342802015133, + 2.2389222050551325, + 2.4163192870328203, + 2.591032551950775, + 2.7698652460239828, + 2.956791950040497, + 3.1453219009563327, + 3.3392527650576085, + 3.535062044975348, + 3.7294887689640746, + 3.9269664810271934, + 4.122307869023643, + 4.314677773974836, + 4.507389587000944, + 4.6991371819749475, + 4.892151460982859, + 5.083274555043317, + 5.27231930999551, + 5.4592487659538165, + 5.638127957005054, + 5.816472326987423, + 5.993659949977882, + 6.167103282990865, + 6.337658509961329, + 6.5062571440357715, + 6.673938952968456, + 6.843950351001695, + 7.017626630025916, + 7.185828204033896, + 7.354807161027566, + 7.524072248023003, + 7.691892985953018, + 7.861271866015159, + 8.03215340001043, + 8.205027175950818, + 8.375341680948623, + 8.548718724981882, + 8.72194080799818, + 8.894171874038875, + 9.066358971991576, + 9.239162266952917, + 9.414261727011763, + 9.585281918058172, + 9.754825184005313, + 9.924414786044508, + 10.094527896028012, + 10.262798174051568, + 10.43158904905431, + 10.600620597950183, + 10.769772339961492, + 10.940029931953177, + 11.110339076025411, + 11.281702662003227, + 11.452871846035123, + 11.621106731006876, + 11.790370458038524, + 11.961187095032074, + 12.13051039003767, + 12.29866559500806, + 12.466015415964648, + 12.63214765896555, + 12.797823896980844, + 12.953651902033016 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 172.51718050101772, + "p95": 249.3311109719798, + "max": 290.4015469830483 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 34.9, + "per_stream_tps": 34.9, + "ttft_s_max": 0.16, + "timing_valid": true, + "end_to_end_tps": 34.32441879581066, + "makespan_s": 7.458247189060785, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 34.94084942925766, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 7.458247189060785, + "end_to_end_tps": 34.32441879581066, + "timing_valid": true, + "per_stream_tps": [ + 34.9 + ], + "ttft_s": [ + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 754379.409830953, + "first": 754379.56985733, + "last": 754386.867905436, + "end": 754386.868078142, + "elapsed_s": 7.458247189060785, + "chunk_offsets_s": [ + 0.16002637695055455, + 0.22061927802860737, + 0.2815148689551279, + 0.3429199530510232, + 0.40838963398709893, + 0.4748909720219672, + 0.5379894949728623, + 0.5907896689604968, + 0.6473129970254377, + 0.7039364540250972, + 0.7601269939914346, + 0.8113639539806172, + 0.8618710030568764, + 0.9138539859559387, + 0.9650993139948696, + 1.017266922048293, + 1.0680983109632507, + 1.120936805033125, + 1.1723405820084736, + 1.2232547389576212, + 1.2732182790059596, + 1.3251374369719997, + 1.3864646650617942, + 1.4448881860589609, + 1.5013753690291196, + 1.551367118023336, + 1.604062283062376, + 1.6639262810349464, + 1.7217869319720194, + 1.7803164219949394, + 1.8395530380075797, + 1.9049987989710644, + 1.9666876130504534, + 2.01886343001388, + 2.076076603960246, + 2.135168274049647, + 2.1871564090251923, + 2.2383971450617537, + 2.2901288450229913, + 2.3430237399879843, + 2.3951572199584916, + 2.447201258968562, + 2.5026106890290976, + 2.5541894120397046, + 2.6052650900091976, + 2.6559158200398088, + 2.706877370015718, + 2.757647013058886, + 2.807613866054453, + 2.8586180709535256, + 2.909976167953573, + 2.9600359169999138, + 3.0117900819750503, + 3.0633594130631536, + 3.114268481032923, + 3.165435296948999, + 3.219003253034316, + 3.275665589957498, + 3.327084040036425, + 3.3785907139535993, + 3.43045780796092, + 3.4806105820462108, + 3.5319685180438682, + 3.5829592359950766, + 3.63427238794975, + 3.6848444619681686, + 3.73589977202937, + 3.7866300869500265, + 3.83910612994805, + 3.8900271829916164, + 3.9407811149721965, + 3.9932803580304608, + 4.043293050955981, + 4.094332328997552, + 4.1459467159584165, + 4.19712436303962, + 4.247358066029847, + 4.297366166021675, + 4.348840440041386, + 4.400726797990501, + 4.45215940696653, + 4.502852858044207, + 4.5548273449530825, + 4.60809752298519, + 4.664370176033117, + 4.7158454259624705, + 4.767599430051632, + 4.81987465498969, + 4.878043581964448, + 4.93506360205356, + 4.984990837983787, + 5.036432359018363, + 5.089275366044603, + 5.145102399052121, + 5.194875873043202, + 5.245665917056613, + 5.295867586974055, + 5.346549869980663, + 5.398947463952936, + 5.45005119103007, + 5.5027615879662335, + 5.559258994995616, + 5.610216113040224, + 5.660476054996252, + 5.7126323990523815, + 5.769828116986901, + 5.827676785062067, + 5.88391660596244, + 5.935643858043477, + 5.988588689011522, + 6.048380878055468, + 6.105583908036351, + 6.161260506953113, + 6.2203805149765685, + 6.2854651199886575, + 6.347533153020777, + 6.4000545729650185, + 6.457459604018368, + 6.516345176962204, + 6.573658896028064, + 6.623917895019986, + 6.675862269010395, + 6.726205284008756, + 6.777948120026849, + 6.828055166057311, + 6.8815031700069085, + 6.937431787955575, + 6.994901219964959, + 7.0513109939638525, + 7.1013161829905584, + 7.153264348977245, + 7.204054167959839, + 7.256104175001383, + 7.306228660978377, + 7.357361412025057, + 7.40908810403198, + 7.458074482972734 + ], + "chunk_gaps_ms": { + "count": 136, + "p50": 51.931765978224576, + "p95": 61.688814079388976, + "max": 66.50133803486824 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.4, + "per_stream_tps": 25.5, + "ttft_s_max": 0.38, + "timing_valid": true, + "end_to_end_tps": 49.79673082902159, + "makespan_s": 10.281799457035959, + "steady_tps": 51.2, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.38693295616982, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 10.281799457035959, + "end_to_end_tps": 49.79673082902159, + "timing_valid": true, + "steady_tps": 51.16649597203632, + "per_stream_tps": [ + 25.3, + 25.7 + ], + "ttft_s": [ + 0.16, + 0.38 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 754386.870876135, + "first": 754387.030534764, + "last": 754397.102653556, + "end": 754397.103023447, + "elapsed_s": 10.232147312024608, + "chunk_offsets_s": [ + 0.1596586290979758, + 0.22074655804317445, + 0.4616732340073213, + 0.5523035100195557, + 0.6335615729913116, + 0.7189688900252804, + 0.8118816030910239, + 0.8964946561027318, + 0.9828275970648974, + 1.06581369205378, + 1.153983018011786, + 1.2420578630408272, + 1.3294164940016344, + 1.41725266398862, + 1.499736514990218, + 1.5941918489988893, + 1.6905292810406536, + 1.785098185064271, + 1.8742858720943332, + 1.9659913569921628, + 2.058994599035941, + 2.1487697470001876, + 2.235582004999742, + 2.327176977065392, + 2.4230186600470915, + 2.511898105032742, + 2.5998973650857806, + 2.691170367063023, + 2.780896874028258, + 2.8651686609955505, + 2.9536870940355584, + 3.049575833021663, + 3.1406606250675395, + 3.2338757410179824, + 3.327512428048067, + 3.419162312988192, + 3.5061688600108027, + 3.5881951540941373, + 3.6761441100388765, + 3.757450253004208, + 3.8477909750072286, + 3.9355906170094386, + 4.023780935094692, + 4.115114801097661, + 4.205558451008983, + 4.299986906000413, + 4.391256611095741, + 4.484563135076314, + 4.575873097055592, + 4.66724389209412, + 4.756650413037278, + 4.847751909052022, + 4.936795514076948, + 5.028999148053117, + 5.117985745077021, + 5.211277741007507, + 5.298824085039087, + 5.392095329007134, + 5.484291139058769, + 5.575967952026986, + 5.669640864012763, + 5.759716990054585, + 5.852999466005713, + 5.942487812018953, + 6.033112614997663, + 6.126330259023234, + 6.2173658980755135, + 6.30436702107545, + 6.3995335140498355, + 6.497629153076559, + 6.588870714069344, + 6.678479621070437, + 6.763764024013653, + 6.856024523032829, + 6.950771155999973, + 7.03930304502137, + 7.129075295990333, + 7.216816858039238, + 7.310075654066168, + 7.401818691054359, + 7.4926589370006695, + 7.580917271086946, + 7.6692434140713885, + 7.758985074004158, + 7.849180017015897, + 7.939451202051714, + 8.032735101995058, + 8.126936082029715, + 8.218733696034178, + 8.303294061101042, + 8.391209416091442, + 8.482579378061928, + 8.572244605049491, + 8.654630118049681, + 8.7395070460625, + 8.830657950020395, + 8.920713884057477, + 9.01033320708666, + 9.10199266008567, + 9.196606939076446, + 9.283788416069001, + 9.372535426053219, + 9.461948459036648, + 9.549392018001527, + 9.637843058095314, + 9.723610825021751, + 9.808981005102396, + 9.901564178988338, + 9.992457481101155, + 10.083518575993367, + 10.16573493508622, + 10.23177742108237 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 90.07612604182214, + "p95": 95.16649297438562, + "max": 240.92667596414685 + } + }, + { + "start": 754386.87052058, + "first": 754387.247844154, + "last": 754397.152206596, + "end": 754397.152320037, + "elapsed_s": 10.281799457035959, + "chunk_offsets_s": [ + 0.3773235740372911, + 0.46235364000312984, + 0.5528628909960389, + 0.6343986360589042, + 0.719597534975037, + 0.812623625039123, + 0.896640368970111, + 0.9835803239839152, + 1.0662968970136717, + 1.1546617760322988, + 1.2427824770566076, + 1.3300503069767728, + 1.4180329750524834, + 1.5003973840503022, + 1.5943194980500266, + 1.6906585780670866, + 1.7857749910326675, + 1.8749446449801326, + 1.9665799230569974, + 2.0595900759799406, + 2.149542602011934, + 2.236373563995585, + 2.327823015046306, + 2.4238479799823835, + 2.5124937900109217, + 2.6005772589705884, + 2.691669731051661, + 2.781606817035936, + 2.865818426012993, + 2.9542385860113427, + 3.0502472950611264, + 3.141378103988245, + 3.234566931030713, + 3.3280365120153874, + 3.4199432800523937, + 3.506954835029319, + 3.5890078169759363, + 3.6762395499972627, + 3.757892801077105, + 3.848591334070079, + 3.9361587340245023, + 4.024374603992328, + 4.115996569045819, + 4.206287754001096, + 4.300615183077753, + 4.3919323610607535, + 4.485148084000684, + 4.5764524299884215, + 4.6681018670788035, + 4.757373907021247, + 4.848564972053282, + 4.937640993972309, + 5.029799267067574, + 5.1187364080687985, + 5.212082323967479, + 5.29949481098447, + 5.392721925978549, + 5.48491176799871, + 5.576786199002527, + 5.670240917010233, + 5.759802430984564, + 5.853565535042435, + 5.9430339119862765, + 6.033850301988423, + 6.126983864000067, + 6.2175968600204214, + 6.30522297997959, + 6.399700555019081, + 6.497828338993713, + 6.589444527053274, + 6.678937065065838, + 6.763982217991725, + 6.856551039032638, + 6.950964565039612, + 7.039739800035022, + 7.129654677002691, + 7.217378542991355, + 7.310232582967728, + 7.402261591050774, + 7.493091837037355, + 7.5813714830437675, + 7.669676346005872, + 7.759142115013674, + 7.849456756026484, + 7.939870549016632, + 8.033174401032738, + 8.127423366066068, + 8.21928043605294, + 8.303930641966872, + 8.391650874982588, + 8.483286728034727, + 8.572904802975245, + 8.655197627027519, + 8.740109930979088, + 8.831263698986731, + 8.921140832011588, + 9.010515175992623, + 9.102705818018876, + 9.197112992056645, + 9.284646951011382, + 9.373147016041912, + 9.462527391966432, + 9.55024874501396, + 9.638575735967606, + 9.724164606072009, + 9.809772484004498, + 9.9020933189895, + 9.993126542074606, + 10.084242245997302, + 10.166530670016073, + 10.232800710014999, + 10.28168601600919 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 89.93708598427474, + "p95": 94.47757503949106, + "max": 98.12778397463262 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.4, + "per_stream_tps": 17.9, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 68.91814349041903, + "makespan_s": 14.858206390053965, + "steady_tps": 71.8, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.40355251660989, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 14.858206390053965, + "end_to_end_tps": 68.91814349041903, + "timing_valid": true, + "steady_tps": 71.83014580846583, + "per_stream_tps": [ + 18.0, + 18.3, + 17.8, + 17.6 + ], + "ttft_s": [ + 0.62, + 0.89, + 0.38, + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 754397.154363383, + "first": 754397.773877289, + "last": 754411.963281328, + "end": 754411.963388129, + "elapsed_s": 14.809024746064097, + "chunk_offsets_s": [ + 0.6195139060728252, + 0.728157481062226, + 1.0091500530252233, + 1.1299403780139983, + 1.2531899720197544, + 1.3765676320763305, + 1.5042662100167945, + 1.6231577810831368, + 1.746572497067973, + 1.8684135830262676, + 1.990782689070329, + 2.114568497054279, + 2.237404743093066, + 2.363225609064102, + 2.4938065650640056, + 2.629134093062021, + 2.7626786440378055, + 2.897430791053921, + 3.02931855909992, + 3.158509470988065, + 3.2889407140901312, + 3.4168955670902506, + 3.546360114007257, + 3.6808390410151333, + 3.815589828067459, + 3.9421197870979086, + 4.071022521005943, + 4.196344805997796, + 4.319927826989442, + 4.448211178998463, + 4.580691143986769, + 4.715686973067932, + 4.8491779550677165, + 4.980153955053538, + 5.109447588096373, + 5.232989969081245, + 5.352719627087936, + 5.469811934046447, + 5.592474978999235, + 5.714962886064313, + 5.8428968579974025, + 5.969195440062322, + 6.096134971012361, + 6.22708209103439, + 6.361687388038263, + 6.493069487041794, + 6.621836971025914, + 6.752483960008249, + 6.881456069997512, + 7.0098684940021485, + 7.138886364991777, + 7.266008153092116, + 7.392779138986953, + 7.523670690017752, + 7.651362884091213, + 7.781161642051302, + 7.911620677099563, + 8.043231978081167, + 8.174535917001776, + 8.304926919052377, + 8.435649957042187, + 8.561028114054352, + 8.689470938988961, + 8.820536971092224, + 8.948955316096544, + 9.074747925042175, + 9.204475353006274, + 9.335619274061173, + 9.466927836998366, + 9.600243634078652, + 9.723429899080656, + 9.850963626988232, + 9.978209785069339, + 10.107484074076638, + 10.237765283090994, + 10.36326828901656, + 10.496438867994584, + 10.627578933024779, + 10.759248475078493, + 10.887601330992766, + 11.011529075098224, + 11.137130961986259, + 11.2651239259867, + 11.391840463038534, + 11.517625343985856, + 11.650039389031008, + 11.7882523490116, + 11.92155787406955, + 12.05134963907767, + 12.176645987085067, + 12.30216779303737, + 12.424335409072228, + 12.54370843898505, + 12.666349611012265, + 12.793049188097939, + 12.920301106059924, + 13.046291700098664, + 13.175249521038495, + 13.30242966208607, + 13.430581988068298, + 13.556726744049229, + 13.684420650009997, + 13.809548132005148, + 13.930019317078404, + 14.048193520051427, + 14.171730156056583, + 14.29668179701548, + 14.421422956045717, + 14.541307255043648, + 14.6522407210432, + 14.742623554077, + 14.808917945018038 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 127.69857794046402, + "p95": 134.7507870523259, + "max": 280.9925719629973 + } + }, + { + "start": 754397.155174847, + "first": 754398.041206715, + "last": 754412.012172537, + "end": 754412.012308283, + "elapsed_s": 14.857133436016738, + "chunk_offsets_s": [ + 0.8860318680526689, + 1.0080981070641428, + 1.129254322964698, + 1.2520818170160055, + 1.375835319980979, + 1.5032115440117195, + 1.6226765440078452, + 1.7461570689920336, + 1.8676785039715469, + 1.9900567299919203, + 2.1138717860449106, + 2.236333293025382, + 2.3624874890083447, + 2.493163630948402, + 2.6281304029980674, + 2.762193070957437, + 2.8970811390317976, + 3.029106411966495, + 3.1581032590474933, + 3.288220579037443, + 3.4164915470173582, + 3.545805659960024, + 3.6802506189560518, + 3.8145138169638813, + 3.9415546300588176, + 4.069992783013731, + 4.195277979015373, + 4.319367644959129, + 4.447528467979282, + 4.5800347060430795, + 4.715056582004763, + 4.848496061051264, + 4.979548044968396, + 5.108750076964498, + 5.232361098984256, + 5.352107365033589, + 5.4693085049511865, + 5.591794667998329, + 5.713719386025332, + 5.8417752149980515, + 5.968919645063579, + 6.095680454978719, + 6.226329011027701, + 6.361232278984971, + 6.492412953055464, + 6.6211574600311, + 6.751787489047274, + 6.880481356056407, + 7.008920597028919, + 7.137771858018823, + 7.264989824034274, + 7.392209214041941, + 7.523305214010179, + 7.650680221966468, + 7.780661524971947, + 7.910581931006163, + 8.04259850003291, + 8.174254170036875, + 8.303883005050011, + 8.435238864040002, + 8.560584492981434, + 8.68921344797127, + 8.819462832994759, + 8.9483444460202, + 9.074191582971253, + 9.203932579956017, + 9.334964500041679, + 9.46625200600829, + 9.59969122800976, + 9.722878693020903, + 9.850213316036388, + 9.977507009985857, + 10.107078645029105, + 10.23725214204751, + 10.362733802990988, + 10.49593716696836, + 10.627133489004336, + 10.758715462055989, + 10.887058941996656, + 11.01102540595457, + 11.136463162954897, + 11.26458478497807, + 11.391543500008993, + 11.517202412011102, + 11.649396006017923, + 11.787678600056097, + 11.921077245031483, + 12.05046035000123, + 12.176295791054145, + 12.301089526037686, + 12.423622217960656, + 12.542670252965763, + 12.665958150988445, + 12.792538142995909, + 12.919820445007645, + 13.045745310955681, + 13.17496180604212, + 13.302073818980716, + 13.430340833030641, + 13.556148594012484, + 13.683994228951633, + 13.809199008042924, + 13.92954366398044, + 14.047426120028831, + 14.171222471049987, + 14.296148224035278, + 14.42117170500569, + 14.540813697967678, + 14.651581931044348, + 14.741921979002655, + 14.80853090097662, + 14.856997690047137 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 127.37622403074056, + "p95": 134.4449589960277, + "max": 138.28259403817356 + } + }, + { + "start": 754397.154825643, + "first": 754397.533668365, + "last": 754411.897195179, + "end": 754411.897310092, + "elapsed_s": 14.742484448943287, + "chunk_offsets_s": [ + 0.3788427219260484, + 0.4627244248986244, + 0.7278332859277725, + 1.0083640309749171, + 1.1291484029497951, + 1.2522426838986576, + 1.3764857589267194, + 1.5032710660016164, + 1.623094196897, + 1.7463488309876993, + 1.8682756779016927, + 1.9906311199301854, + 2.1139427789021283, + 2.236580687924288, + 2.362652163952589, + 2.4929746539564803, + 2.628401845926419, + 2.7620206389110535, + 2.8965809589717537, + 3.0290730689885095, + 3.1582617729436606, + 3.2882881650002673, + 3.4165924289263785, + 3.5459449579939246, + 3.680424765916541, + 3.8146465229801834, + 3.9418344089062884, + 4.070134720997885, + 4.195394508889876, + 4.319614095962606, + 4.448060184950009, + 4.580134930904023, + 4.715590379899368, + 4.848675038898364, + 4.979739743983373, + 5.108789213933051, + 5.232796078897081, + 5.352317239972763, + 5.469396425993182, + 5.592062766896561, + 5.713850635918789, + 5.8422236039768904, + 5.968930765986443, + 6.095438036951236, + 6.226410788949579, + 6.361487785936333, + 6.492535594967194, + 6.621330070891418, + 6.751870370935649, + 6.8806631189072505, + 7.0095465399790555, + 7.1378908678889275, + 7.265171025996096, + 7.3923662879969925, + 7.523414815892465, + 7.65086092799902, + 7.78086282289587, + 7.911332257906906, + 8.04284883895889, + 8.174145993893035, + 8.304060063906945, + 8.434999614953995, + 8.560612717992626, + 8.689438794972375, + 8.81948838592507, + 8.948010123916902, + 9.074775636894628, + 9.204205814981833, + 9.335118581890129, + 9.466429304913618, + 9.599979967926629, + 9.723451466998085, + 9.850465110968798, + 9.977915478986688, + 10.107682315981947, + 10.237830931902863, + 10.363291264977306, + 10.496469076955691, + 10.627540804911405, + 10.759018616983667, + 10.887360736960545, + 11.01132520090323, + 11.136853663949296, + 11.264990756986663, + 11.391842191922478, + 11.517505854950286, + 11.649791418923996, + 11.787977834930643, + 11.921362143941224, + 12.050748528912663, + 12.176597026991658, + 12.301356921903789, + 12.423776571988128, + 12.543291731970385, + 12.666256810887717, + 12.792842210968956, + 12.920115327928215, + 13.046320436988026, + 13.175257537979633, + 13.302359501947649, + 13.430632244911976, + 13.556452613906004, + 13.6842980888905, + 13.809491923893802, + 13.929847634979524, + 14.047570618917234, + 14.17152629897464, + 14.297047208994627, + 14.42178369499743, + 14.541102052899078, + 14.65188354998827, + 14.742369535961188 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 128.13709303736687, + "p95": 135.07699698675424, + "max": 280.5307450471446 + } + }, + { + "start": 754397.154101893, + "first": 754397.315518504, + "last": 754411.806284453, + "end": 754411.807036492, + "elapsed_s": 14.652934599085711, + "chunk_offsets_s": [ + 0.16141661105211824, + 0.22298555204179138, + 0.4636240480467677, + 0.7286723169963807, + 1.0092454940313473, + 1.130275292089209, + 1.2533432370983064, + 1.3771238120971248, + 1.5042196980211884, + 1.6235029360977933, + 1.7473388720536605, + 1.8688342260429636, + 1.9912252210779116, + 2.1150520690716803, + 2.237469639047049, + 2.363600140088238, + 2.4938654460711405, + 2.6292753100860864, + 2.76301863102708, + 2.8975104870041832, + 3.029941715998575, + 3.159086932078935, + 3.2893301730509847, + 3.4174853480653837, + 3.546832229010761, + 3.681274261092767, + 3.8155280670616776, + 3.942662944085896, + 4.070998536073603, + 4.196418198058382, + 4.320396583061665, + 4.4486425740178674, + 4.581056602997705, + 4.716075040050782, + 4.849523558048531, + 4.9805785190546885, + 5.1097768869949505, + 5.2333809160627425, + 5.353220991091803, + 5.470425027073361, + 5.5929312540683895, + 5.715004294062965, + 5.842771272989921, + 5.96980176505167, + 6.096523054060526, + 6.227256348007359, + 6.362122719059698, + 6.49343910603784, + 6.62216068606358, + 6.7528169070137665, + 6.881511334097013, + 7.009809773066081, + 7.138772570993751, + 7.265996681060642, + 7.393234743038192, + 7.524306312086992, + 7.65171187906526, + 7.781678686034866, + 7.911579156061634, + 8.043720254092477, + 8.17509187408723, + 8.30500762409065, + 8.436083448003046, + 8.561492774053477, + 8.689945167046972, + 8.820010230061598, + 8.94898310804274, + 9.074662868049927, + 9.204661786090583, + 9.335676026064903, + 9.466953853028826, + 9.600456851068884, + 9.723756110062823, + 9.850987899000756, + 9.978716126061045, + 10.107827884028666, + 10.23806206905283, + 10.363487203023396, + 10.496642246027477, + 10.627784279058687, + 10.759440029039979, + 10.887807380990125, + 11.011746773030609, + 11.137325730989687, + 11.265333496034145, + 11.392059009056538, + 11.51782816206105, + 11.65020244603511, + 11.78836806199979, + 11.921764707076363, + 12.051367446081713, + 12.177123175002635, + 12.302377554005943, + 12.424643715028651, + 12.543684375006706, + 12.666569581022486, + 12.793250038055703, + 12.920505699003115, + 13.046496502007358, + 13.175460307043977, + 13.302626032033004, + 13.430766470031813, + 13.556936234002933, + 13.684629739029333, + 13.809743638033979, + 13.930237589986064, + 14.048412210075185, + 14.171936014085077, + 14.298010625061579, + 14.421646670089103, + 14.541514585027471, + 14.65218256006483 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 128.0077650444582, + "p95": 135.01843705307692, + "max": 280.57317703496665 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 87.8, + "per_stream_tps": 11.6, + "ttft_s_max": 2.09, + "timing_valid": true, + "end_to_end_tps": 87.50031645235602, + "makespan_s": 23.405629636952654, + "steady_tps": 92.5, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 87.76718400770122, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 23.405629636952654, + "end_to_end_tps": 87.50031645235602, + "timing_valid": true, + "steady_tps": 92.46239294728939, + "per_stream_tps": [ + 11.4, + 11.6, + 11.8, + 11.4, + 11.7, + 12.0, + 11.3, + 11.5 + ], + "ttft_s": [ + 0.62, + 1.16, + 1.77, + 0.38, + 1.46, + 2.09, + 0.16, + 0.88 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 754412.015466135, + "first": 754412.636237839, + "last": 754434.972772784, + "end": 754434.972842896, + "elapsed_s": 22.95737676101271, + "chunk_offsets_s": [ + 0.6207717040088028, + 0.727050872053951, + 1.0056069400161505, + 1.299980415031314, + 1.614324918948114, + 1.9427751180483028, + 2.2857272720430046, + 2.4781593299703673, + 2.6680310129886493, + 2.8630938150454313, + 3.058781421976164, + 3.2463787000160664, + 3.4325838059885427, + 3.624023135052994, + 3.8203895329497755, + 4.019213984021917, + 4.21659904695116, + 4.419480271055363, + 4.623927606036887, + 4.830207644961774, + 5.036285105976276, + 5.239670239039697, + 5.447228737059049, + 5.6505685249576345, + 5.852790447999723, + 6.055814824998379, + 6.260990454000421, + 6.463140903040767, + 6.664063308970071, + 6.8636436719680205, + 7.065449430025183, + 7.2695520729757845, + 7.475723854964599, + 7.677541356999427, + 7.881503663025796, + 8.081230201991275, + 8.2770219220547, + 8.471573709975928, + 8.664923792006448, + 8.855916909058578, + 9.043259480036795, + 9.234952331054956, + 9.427860760013573, + 9.622637526947074, + 9.825074297958054, + 10.024831078015268, + 10.223700777045451, + 10.427596586057916, + 10.629943853011355, + 10.831376999034546, + 11.034751954954118, + 11.232125081005506, + 11.42997254699003, + 11.629463308025151, + 11.826486909994856, + 12.022673849016428, + 12.219065638026223, + 12.419857850996777, + 12.618745182058774, + 12.816897258977406, + 13.017633693991229, + 13.216566530056298, + 13.416097947047092, + 13.613766003050841, + 13.811128376983106, + 14.007656597997993, + 14.209288066020235, + 14.409070861060172, + 14.60685805499088, + 14.80195586301852, + 14.996539731975645, + 15.196630288963206, + 15.396575773949735, + 15.601099348044954, + 15.800566300051287, + 15.996831638971344, + 16.19475148897618, + 16.39754688704852, + 16.596638155984692, + 16.794281795970164, + 16.991227173013613, + 17.190149688045494, + 17.388204802991822, + 17.58308625803329, + 17.775415849057026, + 17.975726137054153, + 18.18257393897511, + 18.390111475950107, + 18.597730885026976, + 18.805427687009797, + 19.013378044008277, + 19.213269639061764, + 19.40950188995339, + 19.60347319196444, + 19.798140643979423, + 19.993727945024148, + 20.18631705804728, + 20.37807941203937, + 20.57091745606158, + 20.764576514950022, + 20.961235314025544, + 21.16177081898786, + 21.36022003297694, + 21.556440635002218, + 21.746285932022147, + 21.9380677100271, + 22.124724850058556, + 22.30840925895609, + 22.485812123049982, + 22.65652937500272, + 22.81476115505211, + 22.957306648953818 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 198.449213989079, + "p95": 207.95035699848086, + "max": 342.9521539947018 + } + }, + { + "start": 754412.01536135, + "first": 754413.178138862, + "last": 754435.21268898, + "end": 754435.212834853, + "elapsed_s": 23.197473502950743, + "chunk_offsets_s": [ + 1.1627775119850412, + 1.3002120340242982, + 1.6142338459612802, + 1.942573083913885, + 2.285730744013563, + 2.4779348159208894, + 2.6678070429479703, + 2.863013781956397, + 3.0585347960004583, + 3.246198938926682, + 3.4327595200156793, + 3.6240255510201678, + 3.8199810329824686, + 4.018741643987596, + 4.21626291598659, + 4.419286397984251, + 4.623787940945476, + 4.830073563964106, + 5.035885293968022, + 5.239909200929105, + 5.447048414964229, + 5.651109937927686, + 5.852958704927005, + 6.055862457025796, + 6.261211447999813, + 6.4634307459928095, + 6.664318575989455, + 6.863910473999567, + 7.065842313924804, + 7.269699657917954, + 7.476058113970794, + 7.677360730944201, + 7.881910913973115, + 8.081372587010264, + 8.276919680996798, + 8.471789983916096, + 8.66513457801193, + 8.856111373985186, + 9.043492986005731, + 9.23522354895249, + 9.42816893896088, + 9.622511078021489, + 9.824836183921434, + 10.024855189956725, + 10.224081052932888, + 10.428064493928105, + 10.629799691960216, + 10.83218671893701, + 11.034821587963961, + 11.23226735391654, + 11.430127540021203, + 11.629630125011317, + 11.826536751003005, + 12.022736185928807, + 12.218950036913157, + 12.420452495920472, + 12.6180313519435, + 12.816144131938927, + 13.016952999983914, + 13.217274136957712, + 13.41628673300147, + 13.613994164974429, + 13.811915679951198, + 14.008753855945542, + 14.209045775933191, + 14.409760740003549, + 14.607475852011703, + 14.802393066929653, + 14.997187912929803, + 15.197032372932881, + 15.397254323936068, + 15.601787129999138, + 15.800042102928273, + 15.997092040954158, + 16.195496871951036, + 16.396941537968814, + 16.59679951693397, + 16.794651606935076, + 16.991978092002682, + 17.19091539096553, + 17.38882765697781, + 17.58354584593326, + 17.775976189994253, + 17.976251516956836, + 18.183005334925838, + 18.39059066399932, + 18.598330474924296, + 18.80597967596259, + 19.0137976949336, + 19.213599545997567, + 19.40983138093725, + 19.60395660402719, + 19.798399926978163, + 19.993796232971363, + 20.186218497925438, + 20.37784995394759, + 20.570978096919134, + 20.764520770986564, + 20.960925263003446, + 21.16146721597761, + 21.360223728930578, + 21.55643967492506, + 21.74710384302307, + 21.93789180496242, + 22.124698625993915, + 22.308556923992, + 22.485870699980296, + 22.65640831796918, + 22.814930851920508, + 22.95774354902096, + 23.08666432800237, + 23.197327630012296 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.92151497676969, + "p95": 207.64920103829354, + "max": 343.15766009967774 + } + }, + { + "start": 754412.017874333, + "first": 754413.785609838, + "last": 754435.371653294, + "end": 754435.371746271, + "elapsed_s": 23.353871938074008, + "chunk_offsets_s": [ + 1.7677355050109327, + 1.9404335600556806, + 2.2826104110572487, + 2.4752442790195346, + 2.665229595033452, + 2.8606142880162224, + 3.055939684039913, + 3.2437888210406527, + 3.43031216901727, + 3.6213379750261083, + 3.8176070590270683, + 4.016434391029179, + 4.2139429750386626, + 4.416725622024387, + 4.621210525045171, + 4.827616373077035, + 5.033432024065405, + 5.236792244017124, + 5.444765579071827, + 5.648459770018235, + 5.850488634081557, + 6.053503412054852, + 6.258748161024414, + 6.4608695060014725, + 6.661978410091251, + 6.861501252045855, + 7.063282082090154, + 7.267270772019401, + 7.473284488078207, + 7.67537032905966, + 7.879299962078221, + 8.078934885095805, + 8.274488331051543, + 8.469353849999607, + 8.662673644023016, + 8.85365028004162, + 9.040943267056718, + 9.232746903086081, + 9.425607332028449, + 9.620290945051238, + 9.822619124082848, + 10.022459008032456, + 10.221342356060632, + 10.425444934051484, + 10.62760058301501, + 10.829449302051216, + 11.032262684078887, + 11.22983811609447, + 11.427463052095845, + 11.627325640060008, + 11.82398407207802, + 12.020316308015026, + 12.216688352054916, + 12.41734666004777, + 12.616300440044142, + 12.814571061055176, + 13.014978534076363, + 13.214437007089145, + 13.413862567045726, + 13.6122501330683, + 13.8094267930137, + 14.005574419046752, + 14.206796283018775, + 14.406936986022629, + 14.605119207059033, + 14.800495594041422, + 14.994758339016698, + 15.194663790985942, + 15.39467914798297, + 15.599178289994597, + 15.79745561501477, + 15.994626339059323, + 16.192910208017565, + 16.394730014028028, + 16.594024196034297, + 16.79209288000129, + 16.989662165986374, + 17.18832973507233, + 17.386281521059573, + 17.580993870040402, + 17.773388135014102, + 17.97414322604891, + 18.18079344299622, + 18.3881100009894, + 18.595741763012484, + 18.80326259508729, + 19.011250504059717, + 19.211201060097665, + 19.40740035008639, + 19.601373637095094, + 19.795423979056068, + 19.991703302017413, + 20.184274528059177, + 20.37603627401404, + 20.569045135052875, + 20.7624806080712, + 20.959042318048887, + 21.15942962898407, + 21.358031053096056, + 21.55432042409666, + 21.74420687300153, + 21.935734633007087, + 22.1226998870261, + 22.306733036064543, + 22.48394577007275, + 22.654422652092762, + 22.81270977598615, + 22.955514728091657, + 23.084382979082875, + 23.1952670190949, + 23.286689412081614, + 23.35377896099817 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 198.18222103640437, + "p95": 207.31655799318105, + "max": 342.17685100156814 + } + }, + { + "start": 754412.016753411, + "first": 754412.396031893, + "last": 754434.830515996, + "end": 754434.830635229, + "elapsed_s": 22.813881817972288, + "chunk_offsets_s": [ + 0.3792784819379449, + 0.46347607497591525, + 0.7258154849987477, + 1.0042134230025113, + 1.2986471069743857, + 1.612992858979851, + 1.9415941629558802, + 2.284471052000299, + 2.4768274289090186, + 2.6667772729415447, + 2.8619116119807586, + 3.057459729956463, + 3.2452316499548033, + 3.4318885679822415, + 3.6227846440160647, + 3.8188521749107167, + 4.017695409944281, + 4.215497885015793, + 4.418294499977492, + 4.622734266915359, + 4.828964720945805, + 5.034916469012387, + 5.238553875940852, + 5.445320302969776, + 5.649006878957152, + 5.851368097937666, + 6.054943248978816, + 6.259563416009769, + 6.461633465019986, + 6.66250663890969, + 6.8618681349325925, + 7.064848767942749, + 7.267986073973589, + 7.47403731092345, + 7.675866076955572, + 7.880145138013177, + 8.079822812927887, + 8.27540126699023, + 8.469864478916861, + 8.663219391950406, + 8.854548511910252, + 9.041742697940208, + 9.233493997016922, + 9.426395706948824, + 9.620921463007107, + 9.823108087992296, + 10.023165830993094, + 10.222357693011872, + 10.425996250938624, + 10.628071851911955, + 10.831022611935623, + 11.033114211983047, + 11.230333351995796, + 11.428190625971183, + 11.62866915599443, + 11.824691293993965, + 12.021010473952629, + 12.21735291799996, + 12.419178211945109, + 12.617268655914813, + 12.81480482290499, + 13.015613499912433, + 13.215767993940972, + 13.414192377938889, + 13.612702504964545, + 13.810058207018301, + 14.00704857695382, + 14.208164311945438, + 14.407739553018473, + 14.605846364982426, + 14.800711388001218, + 14.995288839912973, + 15.195435269968584, + 15.395439218962565, + 15.599948488990776, + 15.799055694020353, + 15.995645500021055, + 16.19364362291526, + 16.396056360914372, + 16.594791018986143, + 16.792933270917274, + 16.990883888909593, + 17.189072142005898, + 17.387014567968436, + 17.581893013906665, + 17.774363038945012, + 17.97452530893497, + 18.181505272979848, + 18.388946632971056, + 18.596535385935567, + 18.804185371962376, + 19.012023326940835, + 19.21204199595377, + 19.408069844008423, + 19.602113162982278, + 19.79670795891434, + 19.99232417193707, + 20.185077862930484, + 20.37675103195943, + 20.569756101001985, + 20.763341175974347, + 20.96008767094463, + 21.160440389998257, + 21.35897334991023, + 21.555099742952734, + 21.744885374908336, + 21.936586368014105, + 22.12370416801423, + 22.30767302797176, + 22.484948706929572, + 22.65544752497226, + 22.81376258493401 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 198.42438399791718, + "p95": 262.33941002283245, + "max": 342.87688904441893 + } + }, + { + "start": 754412.01555612, + "first": 754413.474375554, + "last": 754435.30448456, + "end": 754435.304579009, + "elapsed_s": 23.289022888988256, + "chunk_offsets_s": [ + 1.4588194340467453, + 1.614266774035059, + 1.9426520289853215, + 2.2856048539979383, + 2.4781053450424224, + 2.667900627013296, + 2.8630666460376233, + 3.0586220760596916, + 3.246392076020129, + 3.4327832320705056, + 3.624017214984633, + 3.8199975449824706, + 4.01907945505809, + 4.216658407007344, + 4.419455214985646, + 4.623969973996282, + 4.830196877010167, + 5.036158128990792, + 5.239652366028167, + 5.446661900030449, + 5.650835440028459, + 5.852841280051507, + 6.055769992060959, + 6.2611145350383595, + 6.463139206985943, + 6.664252639049664, + 6.863656776025891, + 7.066119516035542, + 7.269633705029264, + 7.475564973079599, + 7.678006928996183, + 7.881778385024518, + 8.08128485805355, + 8.276879520039074, + 8.47163881501183, + 8.665031329030171, + 8.856010333052836, + 9.043226279085502, + 9.234964507049881, + 9.428011930081993, + 9.622575478046201, + 9.824872168013826, + 10.02471064508427, + 10.223830074071884, + 10.427076757070608, + 10.629756827023812, + 10.831408102996647, + 11.03421067004092, + 11.231483843061142, + 11.429339308990166, + 11.629567485069856, + 11.82583175203763, + 12.021999491029419, + 12.21850144106429, + 12.419489431078546, + 12.618718061014079, + 12.81667234399356, + 13.017823552014306, + 13.216588210081682, + 13.415697079035453, + 13.61460300302133, + 13.811152985086665, + 14.007403028081171, + 14.20826072897762, + 14.409316590987146, + 14.607495052041486, + 14.802462460007519, + 14.997167289024219, + 15.196905252058059, + 15.396920209052041, + 15.60146023100242, + 15.799910390051082, + 15.996542740031146, + 16.19526525400579, + 16.396676031057723, + 16.596374697051942, + 16.7943734760629, + 16.99201260402333, + 17.190688460017554, + 17.38856947806198, + 17.583401156007312, + 17.775743516045623, + 17.976416559075005, + 18.18308122304734, + 18.390360966091976, + 18.598095240071416, + 18.805618089041673, + 19.013639613986015, + 19.213594778091647, + 19.40967633901164, + 19.603726873989217, + 19.79824214905966, + 19.993921210058033, + 20.186820103088394, + 20.37815440504346, + 20.571878393064253, + 20.76486053399276, + 20.96166605304461, + 21.161833442980424, + 21.36046633904334, + 21.556771358009428, + 21.746853808988817, + 21.938093038043007, + 22.124865315039642, + 22.30890590301715, + 22.486207855050452, + 22.656561087002046, + 22.814839635044336, + 22.957740508019924, + 23.086595879052766, + 23.197637008037418, + 23.288928440073505 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 198.17846105434, + "p95": 207.27974304463714, + "max": 342.9528250126168 + } + }, + { + "start": 754412.018128271, + "first": 754414.112993771, + "last": 754435.42044215, + "end": 754435.420884474, + "elapsed_s": 23.402756202965975, + "chunk_offsets_s": [ + 2.094865499995649, + 2.2828067970694974, + 2.475236998987384, + 2.665203146985732, + 2.8603145900415257, + 3.0559947569854558, + 3.2437834769953042, + 3.430547483963892, + 3.6214925840031356, + 3.817693285061978, + 4.016474168049172, + 4.214044927968644, + 4.416958809015341, + 4.621302381972782, + 4.827498772065155, + 5.033413095981814, + 5.236862309044227, + 5.444333543069661, + 5.648298601037823, + 5.850301368976943, + 6.0536336690420285, + 6.258408589987084, + 6.46052588801831, + 6.661633815965615, + 6.8611919860122725, + 7.0634041960584, + 7.266968818032183, + 7.472773140063509, + 7.675058549968526, + 7.87850456405431, + 8.078180990065448, + 8.273868613061495, + 8.468179312068969, + 8.661799396038987, + 8.853066627052613, + 9.04027118103113, + 9.231812461977825, + 9.424967022030614, + 9.62007564806845, + 9.821846893057227, + 10.021659929072484, + 10.220860432018526, + 10.424883489962667, + 10.626776096061803, + 10.828959456994198, + 11.03212780400645, + 11.229538258048706, + 11.427266202983446, + 11.626916549052112, + 11.823624468990602, + 12.019922464969568, + 12.216357358032838, + 12.41740389203187, + 12.615832020062953, + 12.81413843401242, + 13.01502623106353, + 13.213953915052116, + 13.412768429028802, + 13.61206851596944, + 13.809203990967944, + 14.005444498034194, + 14.206571610062383, + 14.406813033041544, + 14.604782228008844, + 14.79998453403823, + 14.994629074004479, + 15.194442494073883, + 15.394456667010672, + 15.598971663974226, + 15.797570144990459, + 15.99376745102927, + 16.19218880997505, + 16.3946266049752, + 16.593372191069648, + 16.791273431968875, + 16.988749886048026, + 17.187444559065625, + 17.385492122964934, + 17.58037071302533, + 17.772709617041983, + 17.974050970049575, + 18.179734857054427, + 18.387227241997607, + 18.59502878098283, + 18.80261638201773, + 19.01045988907572, + 19.210561710991897, + 19.40664930397179, + 19.600547117064707, + 19.79574686300475, + 19.991602790076286, + 20.184394848998636, + 20.375890208990313, + 20.569126192014664, + 20.762482193065807, + 20.959186878986657, + 21.15929377300199, + 21.357941341004334, + 21.554163926979527, + 21.744053991977125, + 21.93555728800129, + 22.122407980961725, + 22.306447128998116, + 22.483725705067627, + 22.654203162062913, + 22.812493485980667, + 22.955120597034693, + 23.084099855972454, + 23.195101978024468, + 23.286322145024315, + 23.35348969406914, + 23.402313878992572 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.9691949672997, + "p95": 206.1963900923729, + "max": 207.84350705798715 + } + }, + { + "start": 754412.015778954, + "first": 754412.17713062, + "last": 754434.671675603, + "end": 754434.671790437, + "elapsed_s": 22.656011482933536, + "chunk_offsets_s": [ + 0.16135166597086936, + 0.22210691799409688, + 0.4641190569382161, + 0.7262711849762127, + 1.004539874033071, + 1.299360953969881, + 1.6137644019909203, + 1.9421021200250834, + 2.28445604396984, + 2.4780540330102667, + 2.6676111849956214, + 2.8625475539593026, + 3.058078184025362, + 3.245833319029771, + 3.432719278964214, + 3.623535530990921, + 3.819330738973804, + 4.01876404392533, + 4.215892976964824, + 4.418930233921856, + 4.623436369001865, + 4.82952975097578, + 5.035567722981796, + 5.2389363279799, + 5.44650757801719, + 5.650039143976755, + 5.8523030990036204, + 6.05568637594115, + 6.260493473033421, + 6.462669107015245, + 6.663357462966815, + 6.862902657012455, + 7.065541542950086, + 7.268847393919714, + 7.47548244392965, + 7.677043191972189, + 7.880805255961604, + 8.080847429926507, + 8.276417883927934, + 8.470787542988546, + 8.664331690990366, + 8.85536216699984, + 9.042774626985192, + 9.234419493936002, + 9.427415059995838, + 9.621680733980611, + 9.823964176001027, + 10.024064206983894, + 10.223793418030255, + 10.426920579979196, + 10.62891699501779, + 10.832083820947446, + 11.034039644990116, + 11.231406321981922, + 11.429424062022008, + 11.629818142973818, + 11.825747543945909, + 12.021834417944774, + 12.218391519971192, + 12.419085890986025, + 12.618108695955016, + 12.81603616301436, + 13.016746629960835, + 13.216198239009827, + 13.414907391997986, + 13.61363540997263, + 13.810976102948189, + 14.007862439961173, + 14.208195863990113, + 14.408911579987034, + 14.606759428977966, + 14.802696877974086, + 14.996375921997242, + 15.196458464022726, + 15.396368347923271, + 15.600970497936942, + 15.800090998993255, + 15.996564836008474, + 16.194691583979875, + 16.397069170954637, + 16.595920149004087, + 16.793815966928378, + 16.991395429940894, + 17.189894484938122, + 17.38793403201271, + 17.58281499100849, + 17.775276280008256, + 17.97577103297226, + 18.18232190492563, + 18.38973574398551, + 18.597565570962615, + 18.805014594923705, + 19.012943334993906, + 19.213066756958142, + 19.40928699192591, + 19.603049459983595, + 19.79773353599012, + 19.99346704594791, + 20.186088655958883, + 20.37764043197967, + 20.570686382008716, + 20.764151502982713, + 20.9609741269378, + 21.161373086972162, + 21.359993599005975, + 21.556181401014328, + 21.746029497939162, + 21.937659738003276, + 22.124514063936658, + 22.308257865952328, + 22.485671945963986, + 22.65589664899744 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 198.7280179746449, + "p95": 262.1521280379966, + "max": 342.3539239447564 + } + }, + { + "start": 754412.015254837, + "first": 754412.899971325, + "last": 754435.102112798, + "end": 754435.102269792, + "elapsed_s": 23.087014955002815, + "chunk_offsets_s": [ + 0.884716487955302, + 1.0052157690515742, + 1.3000845280475914, + 1.6143248230218887, + 1.942705612978898, + 2.2854345809901133, + 2.4776846850290895, + 2.6676447850186378, + 2.8629928700393066, + 3.05842393101193, + 3.246258571045473, + 3.433570135035552, + 3.623907726025209, + 3.8206682869931683, + 4.01878097304143, + 4.216518871020526, + 4.4192899820627645, + 4.6237829650053754, + 4.830006298958324, + 5.035852493951097, + 5.239249514997937, + 5.447513491963036, + 5.6503843639511615, + 5.852786927949637, + 6.056403566035442, + 6.260896772961132, + 6.462971749948338, + 6.663834267994389, + 6.8633163729682565, + 7.066220637061633, + 7.269325574976392, + 7.476140993996523, + 7.678196019027382, + 7.88097973796539, + 8.081666509970091, + 8.276845903950743, + 8.471262668026611, + 8.66477836703416, + 8.856004860950634, + 9.04318051901646, + 9.234870105981827, + 9.428170170984231, + 9.622325508040376, + 9.825556398951449, + 10.025116472970694, + 10.2242633580463, + 10.428024878026918, + 10.630115853971802, + 10.832001468981616, + 11.034741875017062, + 11.23225039395038, + 11.430034292046912, + 11.630252738948911, + 11.826561999041587, + 12.02272226603236, + 12.219169302959926, + 12.420379262999631, + 12.618258634000085, + 12.816645639948547, + 13.017546718008816, + 13.216623026994057, + 13.416046475060284, + 13.614603627007455, + 13.81198353599757, + 14.008708895999007, + 14.20904393598903, + 14.409650866989978, + 14.607547885039821, + 14.802464875974692, + 14.997247657971457, + 15.197174662025645, + 15.397186307003722, + 15.601700681028888, + 15.800501498975791, + 15.996778678963892, + 16.195447591948323, + 16.397386342054233, + 16.59658135601785, + 16.79461658303626, + 16.991825225995854, + 17.190690301009454, + 17.38881453697104, + 17.583571542054415, + 17.775940302060917, + 17.976527648046613, + 18.18344671500381, + 18.390567159978673, + 18.598305018036626, + 18.80624143895693, + 19.01380945497658, + 19.21385951701086, + 19.40989128500223, + 19.603882779949345, + 19.79846202302724, + 19.99415454105474, + 20.18718148197513, + 20.378408264019527, + 20.571519157965668, + 20.765461259987205, + 20.961807815008797, + 21.162098997971043, + 21.360705381957814, + 21.557000639964826, + 21.747118563973345, + 21.93832310498692, + 22.12524175096769, + 22.309075264958665, + 22.486631714971736, + 22.656906194984913, + 22.815175815019757, + 22.95809414400719, + 23.08685796102509 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 198.3870059484616, + "p95": 207.93642092030495, + "max": 342.72896801121533 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + } + ] +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/hostwatch-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/hostwatch-rank0.log new file mode 100644 index 0000000000..62053a07f9 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/hostwatch-rank0.log @@ -0,0 +1,376 @@ +10:49:15 114 +10:49:17 114 +10:49:19 112 +10:49:21 112 +10:49:24 112 +10:49:26 110 +10:49:28 110 +10:49:30 109 +10:49:32 109 +10:49:34 108 +10:49:36 107 +10:49:38 107 +10:49:40 106 +10:49:42 106 +10:49:44 105 +10:49:46 104 +10:49:48 104 +10:49:50 104 +10:49:52 103 +10:49:54 102 +10:49:56 102 +10:49:58 102 +10:50:00 100 +10:50:03 100 +10:50:05 100 +10:50:07 98 +10:50:09 98 +10:50:11 98 +10:50:13 97 +10:50:15 97 +10:50:17 97 +10:50:19 95 +10:50:21 95 +10:50:23 94 +10:50:25 94 +10:50:27 94 +10:50:29 92 +10:50:31 92 +10:50:33 92 +10:50:35 90 +10:50:37 91 +10:50:40 89 +10:50:42 89 +10:50:44 89 +10:50:46 87 +10:50:48 87 +10:50:50 86 +10:50:52 86 +10:50:54 85 +10:50:56 85 +10:50:58 85 +10:51:00 84 +10:51:02 84 +10:51:04 83 +10:51:06 82 +10:51:08 82 +10:51:10 82 +10:51:12 81 +10:51:14 81 +10:51:16 79 +10:51:19 79 +10:51:21 79 +10:51:23 78 +10:51:25 78 +10:51:27 77 +10:51:29 76 +10:51:31 77 +10:51:33 76 +10:51:35 75 +10:51:37 75 +10:51:39 74 +10:51:41 74 +10:51:43 73 +10:51:45 72 +10:51:47 72 +10:51:49 72 +10:51:51 71 +10:51:53 71 +10:51:56 69 +10:51:58 69 +10:52:00 69 +10:52:02 68 +10:52:04 68 +10:52:06 68 +10:52:08 66 +10:52:10 66 +10:52:12 65 +10:52:14 65 +10:52:16 64 +10:52:18 64 +10:52:20 64 +10:52:22 63 +10:52:24 62 +10:52:26 62 +10:52:28 61 +10:52:30 61 +10:52:32 61 +10:52:35 59 +10:52:37 59 +10:52:39 59 +10:52:41 58 +10:52:43 58 +10:52:45 57 +10:52:47 57 +10:52:49 57 +10:52:51 55 +10:52:53 55 +10:52:55 54 +10:52:57 53 +10:52:59 52 +10:53:01 52 +10:53:03 50 +10:53:05 50 +10:53:07 50 +10:53:09 50 +10:53:12 50 +10:53:14 40 +10:53:16 23 +10:53:18 23 +10:53:20 23 +10:53:22 23 +10:53:24 23 +10:53:26 23 +10:53:28 23 +10:53:30 23 +10:53:32 23 +10:53:34 23 +10:53:36 23 +10:53:38 23 +10:53:40 23 +10:53:42 23 +10:53:45 23 +10:53:47 23 +10:53:49 23 +10:53:51 23 +10:53:53 23 +10:53:55 23 +10:53:57 23 +10:53:59 23 +10:54:01 23 +10:54:03 23 +10:54:05 24 +10:54:07 24 +10:54:09 24 +10:54:11 24 +10:54:14 24 +10:54:16 24 +10:54:18 24 +10:54:20 24 +10:54:22 24 +10:54:24 24 +10:54:26 24 +10:54:28 24 +10:54:30 24 +10:54:32 24 +10:54:34 24 +10:54:36 24 +10:54:38 24 +10:54:40 24 +10:54:42 24 +10:54:45 24 +10:54:47 24 +10:54:49 24 +10:54:51 24 +10:54:53 24 +10:54:55 24 +10:54:57 24 +10:54:59 24 +10:55:01 24 +10:55:03 24 +10:55:05 24 +10:55:07 24 +10:55:09 24 +10:55:11 24 +10:55:13 24 +10:55:15 24 +10:55:18 24 +10:55:20 24 +10:55:22 24 +10:55:24 24 +10:55:26 24 +10:55:28 24 +10:55:30 24 +10:55:32 24 +10:55:34 24 +10:55:36 24 +10:55:38 24 +10:55:40 24 +10:55:42 24 +10:55:44 24 +10:55:47 24 +10:55:49 24 +10:55:51 24 +10:55:53 24 +10:55:55 24 +10:55:57 24 +10:55:59 24 +10:56:01 24 +10:56:03 24 +10:56:05 24 +10:56:07 24 +10:56:09 24 +10:56:11 24 +10:56:13 24 +10:56:15 24 +10:56:18 24 +10:56:20 24 +10:56:22 24 +10:56:24 24 +10:56:26 24 +10:56:28 24 +10:56:30 24 +10:56:32 24 +10:56:34 24 +10:56:36 24 +10:56:38 24 +10:56:40 24 +10:56:42 24 +10:56:44 24 +10:56:46 24 +10:56:49 24 +10:56:51 24 +10:56:53 24 +10:56:55 24 +10:56:57 24 +10:56:59 24 +10:57:01 24 +10:57:03 24 +10:57:05 24 +10:57:07 24 +10:57:09 24 +10:57:11 24 +10:57:13 24 +10:57:15 24 +10:57:18 24 +10:57:20 24 +10:57:22 24 +10:57:24 24 +10:57:26 24 +10:57:28 24 +10:57:30 24 +10:57:32 24 +10:57:34 24 +10:57:36 24 +10:57:38 24 +10:57:40 24 +10:57:42 24 +10:57:44 24 +10:57:46 24 +10:57:49 24 +10:57:51 24 +10:57:53 24 +10:57:55 24 +10:57:57 24 +10:57:59 24 +10:58:01 24 +10:58:03 24 +10:58:05 24 +10:58:07 24 +10:58:09 24 +10:58:11 24 +10:58:13 24 +10:58:15 24 +10:58:17 24 +10:58:20 24 +10:58:22 24 +10:58:24 24 +10:58:26 24 +10:58:28 24 +10:58:30 24 +10:58:32 24 +10:58:34 24 +10:58:36 24 +10:58:38 24 +10:58:40 24 +10:58:42 24 +10:58:44 24 +10:58:46 24 +10:58:48 24 +10:58:51 24 +10:58:53 24 +10:58:55 24 +10:58:57 24 +10:58:59 24 +10:59:01 24 +10:59:03 24 +10:59:05 24 +10:59:07 24 +10:59:09 24 +10:59:11 24 +10:59:13 24 +10:59:15 24 +10:59:17 24 +10:59:19 24 +10:59:22 24 +10:59:24 24 +10:59:26 24 +10:59:28 24 +10:59:30 24 +10:59:32 24 +10:59:34 24 +10:59:36 24 +10:59:38 24 +10:59:40 24 +10:59:42 24 +10:59:44 24 +10:59:46 24 +10:59:48 24 +10:59:50 24 +10:59:53 24 +10:59:55 24 +10:59:57 24 +10:59:59 24 +11:00:01 24 +11:00:03 24 +11:00:05 24 +11:00:07 24 +11:00:09 24 +11:00:11 24 +11:00:13 24 +11:00:15 24 +11:00:17 24 +11:00:19 24 +11:00:21 24 +11:00:24 24 +11:00:26 24 +11:00:28 24 +11:00:30 24 +11:00:32 24 +11:00:34 24 +11:00:36 24 +11:00:38 24 +11:00:40 24 +11:00:42 24 +11:00:44 24 +11:00:46 24 +11:00:48 24 +11:00:50 24 +11:00:52 24 +11:00:55 24 +11:00:57 24 +11:00:59 24 +11:01:01 24 +11:01:03 24 +11:01:05 24 +11:01:07 24 +11:01:09 24 +11:01:11 24 +11:01:13 24 +11:01:15 24 +11:01:17 24 +11:01:19 24 +11:01:21 24 +11:01:23 24 +11:01:26 24 +11:01:28 24 +11:01:30 24 +11:01:32 24 +11:01:34 24 +11:01:36 24 +11:01:38 24 +11:01:40 24 +11:01:42 24 +11:01:44 24 +11:01:46 24 +11:01:48 24 +11:01:50 24 +11:01:52 24 +11:01:55 24 +11:01:57 24 +11:01:59 24 +11:02:01 24 +11:02:03 24 +11:02:05 24 +11:02:07 54 +11:02:09 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/hostwatch-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/hostwatch-rank1.log new file mode 100644 index 0000000000..e34ea317f9 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/hostwatch-rank1.log @@ -0,0 +1,375 @@ +10:49:16 114 +10:49:18 112 +10:49:20 112 +10:49:22 112 +10:49:24 110 +10:49:26 110 +10:49:28 109 +10:49:30 108 +10:49:32 108 +10:49:34 106 +10:49:36 106 +10:49:38 106 +10:49:40 105 +10:49:42 105 +10:49:44 105 +10:49:47 104 +10:49:49 103 +10:49:51 102 +10:49:53 103 +10:49:55 102 +10:49:57 101 +10:49:59 100 +10:50:01 99 +10:50:03 99 +10:50:05 98 +10:50:07 98 +10:50:09 98 +10:50:11 97 +10:50:13 97 +10:50:15 97 +10:50:17 95 +10:50:19 95 +10:50:21 94 +10:50:23 94 +10:50:26 94 +10:50:28 93 +10:50:30 93 +10:50:32 91 +10:50:34 91 +10:50:36 91 +10:50:38 90 +10:50:40 90 +10:50:42 89 +10:50:44 89 +10:50:46 87 +10:50:48 87 +10:50:50 87 +10:50:52 86 +10:50:54 86 +10:50:56 84 +10:50:58 84 +10:51:00 84 +10:51:03 83 +10:51:05 83 +10:51:07 82 +10:51:09 81 +10:51:11 81 +10:51:13 80 +10:51:15 80 +10:51:17 79 +10:51:19 78 +10:51:21 78 +10:51:23 76 +10:51:25 76 +10:51:27 76 +10:51:29 75 +10:51:31 75 +10:51:33 74 +10:51:35 74 +10:51:37 73 +10:51:40 72 +10:51:42 72 +10:51:44 71 +10:51:46 71 +10:51:48 71 +10:51:50 69 +10:51:52 69 +10:51:54 68 +10:51:56 68 +10:51:58 68 +10:52:00 67 +10:52:02 67 +10:52:04 67 +10:52:06 65 +10:52:08 65 +10:52:10 64 +10:52:12 64 +10:52:14 63 +10:52:17 63 +10:52:19 63 +10:52:21 61 +10:52:23 61 +10:52:25 61 +10:52:27 60 +10:52:29 60 +10:52:31 59 +10:52:33 58 +10:52:35 58 +10:52:37 57 +10:52:39 57 +10:52:41 57 +10:52:43 55 +10:52:45 55 +10:52:47 55 +10:52:49 54 +10:52:51 54 +10:52:54 52 +10:52:56 51 +10:52:58 50 +10:53:00 50 +10:53:02 50 +10:53:04 50 +10:53:06 51 +10:53:08 50 +10:53:10 50 +10:53:12 50 +10:53:14 24 +10:53:16 24 +10:53:18 24 +10:53:20 24 +10:53:22 24 +10:53:24 24 +10:53:26 24 +10:53:29 24 +10:53:31 24 +10:53:33 24 +10:53:35 24 +10:53:37 24 +10:53:39 24 +10:53:41 24 +10:53:43 24 +10:53:45 24 +10:53:47 24 +10:53:49 24 +10:53:51 24 +10:53:53 24 +10:53:55 24 +10:53:57 24 +10:54:00 24 +10:54:02 24 +10:54:04 24 +10:54:06 24 +10:54:08 24 +10:54:10 24 +10:54:12 24 +10:54:14 24 +10:54:16 24 +10:54:18 24 +10:54:20 24 +10:54:22 24 +10:54:24 24 +10:54:26 24 +10:54:28 24 +10:54:31 24 +10:54:33 24 +10:54:35 24 +10:54:37 24 +10:54:39 24 +10:54:41 24 +10:54:43 24 +10:54:45 24 +10:54:47 24 +10:54:49 24 +10:54:51 24 +10:54:53 24 +10:54:55 24 +10:54:57 24 +10:54:59 24 +10:55:01 24 +10:55:04 24 +10:55:06 24 +10:55:08 24 +10:55:10 24 +10:55:12 24 +10:55:14 24 +10:55:16 24 +10:55:18 24 +10:55:20 24 +10:55:22 24 +10:55:24 24 +10:55:26 24 +10:55:28 24 +10:55:30 24 +10:55:32 24 +10:55:35 24 +10:55:37 24 +10:55:39 24 +10:55:41 24 +10:55:43 24 +10:55:45 24 +10:55:47 24 +10:55:49 24 +10:55:51 24 +10:55:53 24 +10:55:55 24 +10:55:57 24 +10:55:59 24 +10:56:01 24 +10:56:04 24 +10:56:06 24 +10:56:08 24 +10:56:10 24 +10:56:12 24 +10:56:14 24 +10:56:16 24 +10:56:18 24 +10:56:20 24 +10:56:22 24 +10:56:24 24 +10:56:26 24 +10:56:28 24 +10:56:30 24 +10:56:32 24 +10:56:35 24 +10:56:37 24 +10:56:39 24 +10:56:41 24 +10:56:43 24 +10:56:45 24 +10:56:47 24 +10:56:49 24 +10:56:51 24 +10:56:53 24 +10:56:55 24 +10:56:57 24 +10:56:59 24 +10:57:01 24 +10:57:03 24 +10:57:06 24 +10:57:08 24 +10:57:10 24 +10:57:12 24 +10:57:14 24 +10:57:16 24 +10:57:18 24 +10:57:20 24 +10:57:22 24 +10:57:24 24 +10:57:26 24 +10:57:28 24 +10:57:30 24 +10:57:32 24 +10:57:35 24 +10:57:37 24 +10:57:39 24 +10:57:41 24 +10:57:43 24 +10:57:45 24 +10:57:47 24 +10:57:49 24 +10:57:51 24 +10:57:53 24 +10:57:55 24 +10:57:57 24 +10:57:59 24 +10:58:01 24 +10:58:03 24 +10:58:05 24 +10:58:08 24 +10:58:10 24 +10:58:12 24 +10:58:14 24 +10:58:16 24 +10:58:18 24 +10:58:20 24 +10:58:22 24 +10:58:24 24 +10:58:26 24 +10:58:28 24 +10:58:30 24 +10:58:32 24 +10:58:34 24 +10:58:36 24 +10:58:39 24 +10:58:41 24 +10:58:43 24 +10:58:45 24 +10:58:47 24 +10:58:49 24 +10:58:51 24 +10:58:53 24 +10:58:55 24 +10:58:57 24 +10:58:59 24 +10:59:01 24 +10:59:03 24 +10:59:05 24 +10:59:07 24 +10:59:10 24 +10:59:12 24 +10:59:14 24 +10:59:16 24 +10:59:18 24 +10:59:20 24 +10:59:22 24 +10:59:24 24 +10:59:26 24 +10:59:28 24 +10:59:30 24 +10:59:32 24 +10:59:34 24 +10:59:36 24 +10:59:38 24 +10:59:41 24 +10:59:43 24 +10:59:45 24 +10:59:47 24 +10:59:49 24 +10:59:51 24 +10:59:53 24 +10:59:55 24 +10:59:57 24 +10:59:59 24 +11:00:01 24 +11:00:03 24 +11:00:05 24 +11:00:07 24 +11:00:09 24 +11:00:12 24 +11:00:14 24 +11:00:16 24 +11:00:18 24 +11:00:20 24 +11:00:22 24 +11:00:24 24 +11:00:26 24 +11:00:28 24 +11:00:30 24 +11:00:32 24 +11:00:34 24 +11:00:36 24 +11:00:38 24 +11:00:41 24 +11:00:43 24 +11:00:45 24 +11:00:47 24 +11:00:49 24 +11:00:51 24 +11:00:53 24 +11:00:55 24 +11:00:57 24 +11:00:59 24 +11:01:01 24 +11:01:03 24 +11:01:05 24 +11:01:07 24 +11:01:09 24 +11:01:12 24 +11:01:14 24 +11:01:16 24 +11:01:18 24 +11:01:20 24 +11:01:22 24 +11:01:24 24 +11:01:26 24 +11:01:28 24 +11:01:30 24 +11:01:32 24 +11:01:34 24 +11:01:36 24 +11:01:38 24 +11:01:40 24 +11:01:43 24 +11:01:45 24 +11:01:47 24 +11:01:49 24 +11:01:51 24 +11:01:53 24 +11:01:55 24 +11:01:57 24 +11:01:59 24 +11:02:01 24 +11:02:03 24 +11:02:05 24 +11:02:07 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/hostwatch-rank2.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/hostwatch-rank2.log new file mode 100644 index 0000000000..79e77cd2c8 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/hostwatch-rank2.log @@ -0,0 +1,375 @@ +10:49:16 114 +10:49:18 112 +10:49:20 112 +10:49:22 111 +10:49:24 110 +10:49:27 111 +10:49:29 109 +10:49:31 109 +10:49:33 108 +10:49:35 108 +10:49:37 108 +10:49:39 106 +10:49:41 106 +10:49:43 105 +10:49:45 105 +10:49:47 104 +10:49:49 104 +10:49:51 104 +10:49:53 103 +10:49:55 102 +10:49:57 102 +10:49:59 101 +10:50:01 101 +10:50:03 100 +10:50:06 100 +10:50:08 99 +10:50:10 99 +10:50:12 99 +10:50:14 97 +10:50:16 97 +10:50:18 97 +10:50:20 96 +10:50:22 96 +10:50:24 94 +10:50:26 95 +10:50:28 94 +10:50:30 93 +10:50:32 93 +10:50:34 92 +10:50:36 92 +10:50:38 91 +10:50:41 90 +10:50:43 90 +10:50:45 89 +10:50:47 89 +10:50:49 88 +10:50:51 88 +10:50:53 87 +10:50:55 86 +10:50:57 86 +10:50:59 85 +10:51:01 85 +10:51:03 84 +10:51:05 83 +10:51:07 83 +10:51:09 82 +10:51:11 82 +10:51:13 81 +10:51:15 81 +10:51:18 80 +10:51:20 79 +10:51:22 79 +10:51:24 79 +10:51:26 78 +10:51:28 77 +10:51:30 77 +10:51:32 77 +10:51:34 75 +10:51:36 75 +10:51:38 75 +10:51:40 74 +10:51:42 74 +10:51:44 73 +10:51:46 73 +10:51:48 72 +10:51:50 72 +10:51:52 71 +10:51:55 70 +10:51:57 70 +10:51:59 69 +10:52:01 69 +10:52:03 68 +10:52:05 68 +10:52:07 68 +10:52:09 67 +10:52:11 67 +10:52:13 65 +10:52:15 65 +10:52:17 65 +10:52:19 63 +10:52:21 64 +10:52:23 62 +10:52:25 63 +10:52:27 63 +10:52:30 62 +10:52:32 62 +10:52:34 61 +10:52:36 60 +10:52:38 60 +10:52:40 59 +10:52:42 59 +10:52:44 59 +10:52:46 57 +10:52:48 55 +10:52:50 55 +10:52:52 55 +10:52:54 55 +10:52:56 55 +10:52:58 57 +10:53:00 55 +10:53:02 55 +10:53:04 55 +10:53:06 55 +10:53:09 55 +10:53:11 55 +10:53:13 56 +10:53:15 29 +10:53:17 29 +10:53:19 29 +10:53:21 29 +10:53:23 29 +10:53:25 29 +10:53:27 29 +10:53:29 29 +10:53:31 29 +10:53:33 29 +10:53:35 29 +10:53:37 29 +10:53:40 29 +10:53:42 29 +10:53:44 29 +10:53:46 29 +10:53:48 29 +10:53:50 29 +10:53:52 29 +10:53:54 29 +10:53:56 29 +10:53:58 29 +10:54:00 29 +10:54:02 29 +10:54:04 29 +10:54:06 29 +10:54:08 29 +10:54:11 29 +10:54:13 29 +10:54:15 29 +10:54:17 29 +10:54:19 29 +10:54:21 29 +10:54:23 29 +10:54:25 29 +10:54:27 29 +10:54:29 29 +10:54:31 29 +10:54:33 29 +10:54:35 29 +10:54:37 29 +10:54:39 29 +10:54:42 29 +10:54:44 29 +10:54:46 29 +10:54:48 29 +10:54:50 29 +10:54:52 29 +10:54:54 29 +10:54:56 29 +10:54:58 29 +10:55:00 29 +10:55:02 29 +10:55:04 29 +10:55:06 29 +10:55:08 29 +10:55:10 29 +10:55:13 29 +10:55:15 29 +10:55:17 29 +10:55:19 29 +10:55:21 29 +10:55:23 29 +10:55:25 29 +10:55:27 29 +10:55:29 29 +10:55:31 29 +10:55:33 29 +10:55:35 29 +10:55:37 29 +10:55:39 29 +10:55:41 29 +10:55:44 29 +10:55:46 29 +10:55:48 29 +10:55:50 29 +10:55:52 29 +10:55:54 29 +10:55:56 29 +10:55:58 29 +10:56:00 29 +10:56:02 29 +10:56:04 29 +10:56:06 29 +10:56:08 29 +10:56:10 29 +10:56:12 29 +10:56:15 29 +10:56:17 29 +10:56:19 29 +10:56:21 29 +10:56:23 29 +10:56:25 29 +10:56:27 29 +10:56:29 29 +10:56:31 29 +10:56:33 29 +10:56:35 29 +10:56:37 29 +10:56:39 29 +10:56:41 29 +10:56:44 29 +10:56:46 29 +10:56:48 29 +10:56:50 29 +10:56:52 29 +10:56:54 29 +10:56:56 29 +10:56:58 29 +10:57:00 29 +10:57:02 29 +10:57:04 29 +10:57:06 29 +10:57:08 29 +10:57:10 29 +10:57:13 29 +10:57:15 29 +10:57:17 29 +10:57:19 29 +10:57:21 29 +10:57:23 29 +10:57:25 29 +10:57:27 29 +10:57:29 29 +10:57:31 29 +10:57:33 29 +10:57:35 29 +10:57:37 29 +10:57:39 29 +10:57:41 29 +10:57:44 29 +10:57:46 29 +10:57:48 29 +10:57:50 29 +10:57:52 29 +10:57:54 29 +10:57:56 29 +10:57:58 29 +10:58:00 29 +10:58:02 29 +10:58:04 29 +10:58:06 29 +10:58:08 29 +10:58:10 29 +10:58:12 29 +10:58:15 29 +10:58:17 29 +10:58:19 29 +10:58:21 29 +10:58:23 29 +10:58:25 29 +10:58:27 29 +10:58:29 29 +10:58:31 29 +10:58:33 29 +10:58:35 29 +10:58:37 29 +10:58:39 29 +10:58:41 29 +10:58:43 29 +10:58:46 29 +10:58:48 29 +10:58:50 29 +10:58:52 29 +10:58:54 29 +10:58:56 29 +10:58:58 29 +10:59:00 29 +10:59:02 29 +10:59:04 29 +10:59:06 29 +10:59:08 29 +10:59:10 29 +10:59:12 29 +10:59:14 29 +10:59:17 29 +10:59:19 29 +10:59:21 29 +10:59:23 29 +10:59:25 29 +10:59:27 29 +10:59:29 29 +10:59:31 29 +10:59:33 29 +10:59:35 29 +10:59:37 29 +10:59:39 29 +10:59:41 29 +10:59:43 29 +10:59:46 29 +10:59:48 29 +10:59:50 29 +10:59:52 29 +10:59:54 29 +10:59:56 29 +10:59:58 29 +11:00:00 29 +11:00:02 29 +11:00:04 29 +11:00:06 29 +11:00:08 29 +11:00:10 29 +11:00:12 29 +11:00:14 29 +11:00:17 29 +11:00:19 29 +11:00:21 29 +11:00:23 29 +11:00:25 29 +11:00:27 29 +11:00:29 29 +11:00:31 29 +11:00:33 29 +11:00:35 29 +11:00:37 29 +11:00:39 29 +11:00:41 29 +11:00:43 29 +11:00:45 29 +11:00:48 29 +11:00:50 29 +11:00:52 29 +11:00:54 29 +11:00:56 29 +11:00:58 29 +11:01:00 29 +11:01:02 29 +11:01:04 29 +11:01:06 29 +11:01:08 29 +11:01:10 29 +11:01:12 29 +11:01:14 29 +11:01:17 29 +11:01:19 29 +11:01:21 29 +11:01:23 29 +11:01:25 29 +11:01:27 29 +11:01:29 29 +11:01:31 29 +11:01:33 29 +11:01:35 29 +11:01:37 29 +11:01:39 29 +11:01:41 29 +11:01:43 29 +11:01:46 29 +11:01:48 29 +11:01:50 29 +11:01:52 29 +11:01:54 29 +11:01:56 29 +11:01:58 29 +11:02:00 29 +11:02:02 29 +11:02:04 29 +11:02:06 29 +11:02:08 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/server-rank0.log new file mode 100644 index 0000000000..da84ebbeb0 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/server-rank0.log @@ -0,0 +1,3324 @@ +[kvp-serve] 2026-10-23T01:49:06+00:00 rank 0 of 3 context 98304 parallel 8 binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=all TF_GLM_MM=unset args +info: rank 0: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-a:4542:4542 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:4542:4542 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:4542:4542 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:4542:4542 [0] NCCL INFO cudaDriverVersion 13030 +host-a:4542:4542 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:4542:4542 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:4542:4542 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:4542:4542 [0] NCCL INFO Count not open libibverbs.so +host-a:4542:4542 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:4542:4542 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:4542:4542 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:4542:4542 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x399af7b0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:4542:4542 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x399f0eb0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:4542:4542 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x39a325b0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:4542:4542 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x39a73cb0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:4542:4542 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4542:4542 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4542:4542 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4542:4542 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:4542:4542 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:4542:4542 [0] NCCL INFO Initialized NET plugin IB +host-a:4542:4542 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:4542:4542 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:4542:4542 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:4542:4542 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:4542:4542 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:4542:4542 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:4542:4542 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:4542:4542 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:4542:4542 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:4542:4542 [0] NCCL INFO Using network IB +host-a:4542:4542 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:4542:4542 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x36d780f0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x21607d5aa2555317 - Init START +host-a:4542:4542 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:4542:4542 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:4542:4542 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:4542:4542 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:4542:4542 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:4542:4542 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:4542:4542 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:4542:4542 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:4542:4542 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:4542:4542 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:4542:4542 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:4542:4542 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:4542:4542 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:4542:4542 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:4542:4542 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:4542:4542 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:4542:4542 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:4542:4542 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:4542:4542 [0] NCCL INFO Rank 0: 6 Net devices +host-a:4542:4542 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:4542:4542 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:4542:4542 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:4542:4542 [0] NCCL INFO comm 0x36d780f0 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:4542:4542 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:4542:4542 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:4542:4542 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:4542:4542 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:4542:4542 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:4542:4542 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:4542:4542 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:4542:4542 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:4542:4542 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:4542:4542 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:4542:4560 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 4 +host-a:4542:4559 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-a:4542:4559 [0] NCCL INFO proxy listening socket at <33667> +host-a:4542:4542 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:4542:4561 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-a:4542:4542 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:4542:4542 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:4542:4542 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:4542:4542 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:4542:4542 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:4542:4542 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:4542:4542 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb24010000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb24054000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb2409c000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6921 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6922 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6921 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1baba8 ctsFifoLkey=0x1baba8 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6921 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6922 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1baba8 ctsFifoLkey=0x1baba8 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6922 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb24145000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb2418d000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb241d1000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb24219000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb2425d000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6953 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6954 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6953 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2c2a ctsFifoLkey=0x1d2c2a +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6953 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6954 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2c2a ctsFifoLkey=0x1d2c2a +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6954 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6921 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6922 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2027 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2028 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2029 pkey=0 pd=0xe2bb242e8ad0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1899 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1900 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1901 pkey=0 pd=0xe2bb24322c00 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2030 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2031 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2032 pkey=0 pd=0xe2bb242e8ad0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1902 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1903 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1904 pkey=0 pd=0xe2bb24322c00 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6953 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6954 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6923 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6924 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6923 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6923 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6924 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6924 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6955 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6956 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6955 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1cd2d6 ctsFifoLkey=0x1cd2d6 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6955 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6956 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1cd2d6 ctsFifoLkey=0x1cd2d6 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6956 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6923 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6924 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6955 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6956 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4542 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:4542:4542 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:4542:4542 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:4542:4542 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:4542:4542 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb245b5000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb245f6000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb24637000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb24678000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1905 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1906 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1905 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1905 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1906 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1906 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2033 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2034 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2033 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19bcbb ctsFifoLkey=0x19bcbb +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2033 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2034 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19bcbb ctsFifoLkey=0x19bcbb +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2034 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1907 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1908 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1907 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1907 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1908 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1908 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2035 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2036 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2035 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x193836 ctsFifoLkey=0x193836 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2035 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2036 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x193836 ctsFifoLkey=0x193836 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2036 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1905 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1906 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2033 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2034 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1907 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1908 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2035 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2036 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4542 [0] NCCL INFO Connected all trees +host-a:4542:4542 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:4542:4542 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:4542:4542 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:4542:4542 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb24825000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb2486a000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb248af000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb248f4000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6925 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6926 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6927 pkey=0 pd=0xe2bb24053590 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6957 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6958 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6959 pkey=0 pd=0xe2bb240f2d40 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6928 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6929 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6930 pkey=0 pd=0xe2bb24053590 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6960 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6961 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6962 pkey=0 pd=0xe2bb240f2d40 +host-a:4542:4542 [0] NCCL INFO Connected binomial trees +host-a:4542:4542 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:4542:4542 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:4542:4542 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:4542:4542 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:4542:4542 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-a:4542:4542 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:4542:4542 [0] NCCL INFO ncclCommInitRank comm 0x36d780f0 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x21607d5aa2555317 - Init COMPLETE +host-a:4542:4542 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 4.57 (kernels 0.23, alloc 0.03, bootstrap 4.02, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.27, rest 0.00) +host-a:4542:4542 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:4542:4542 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:4542:4542 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:4542:4542 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:4542:4542 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:4542:4542 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:4542:4542 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:4542:4542 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb24b17000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb24b58000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb24ba0000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb24be4000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb24c2c000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6931 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6932 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6931 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1afaf7 ctsFifoLkey=0x1afaf7 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6931 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6932 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1afaf7 ctsFifoLkey=0x1afaf7 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6932 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6963 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6964 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6963 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d6058 ctsFifoLkey=0x1d6058 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6963 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6964 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d6058 ctsFifoLkey=0x1d6058 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6964 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2037 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2038 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2037 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19e6e1 ctsFifoLkey=0x19e6e1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2037 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2038 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19e6e1 ctsFifoLkey=0x19e6e1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2038 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb24d81000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe2bb24dc9000) +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe2bb24e0d000) +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2039 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2040 pkey=0 pd=0xe2bb242e8ad0 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2041 pkey=0 pd=0xe2bb242e8ad0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6931 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6932 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1909 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1910 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1911 pkey=0 pd=0xe2bb24322c00 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6963 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6964 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2037 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2038 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6965 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6966 pkey=0 pd=0xe2bb240f2d40 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6967 pkey=0 pd=0xe2bb240f2d40 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1912 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1913 pkey=0 pd=0xe2bb24322c00 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1912 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x179e59 ctsFifoLkey=0x179e59 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1912 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1913 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x179e59 ctsFifoLkey=0x179e59 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1913 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 64.80 GiB; 8 streams at once, the engine's own and 7 more taking 1544 pages of 512 rows (9.61 GiB) as they need them (enough for one at a 98304-token window), 131 MiB fixed each, of 38.1 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 16.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 98304; loaded in 248.3s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-87f589f7cf25a63a10cc5963471bb58f prompt=14 cached=0 thinking=False effort=none tokens=16 sha=e195338a9a9d finish=length rounds=6 accepted=9/13 ttft=0.112 decode=0.352 elapsed=0.464 queued=0.000 tok/s=42.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-64d361da7f370a62a95fc515d36f19f7 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.110 decode=5.254 elapsed=5.364 queued=0.000 tok/s=48.5 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-de9f71427505f8d7cf005d57deaf3392 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=108 accepted=147/202 ttft=0.109 decode=6.188 elapsed=6.297 queued=0.000 tok/s=41.2 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 4, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-481a9749c7d5acb81f9d8c1b45dafb8d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=106 accepted=150/212 ttft=0.110 decode=6.220 elapsed=6.331 queued=0.000 tok/s=41.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 5, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-1d02a4f8607a2f3245e460142b62ea8f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=105 accepted=150/218 ttft=0.110 decode=6.207 elapsed=6.317 queued=0.000 tok/s=41.1 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-70e1233b27a1211b62f42e7fb9429ffb prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=77 accepted=179/227 ttft=0.108 decode=4.940 elapsed=5.048 queued=0.000 tok/s=51.6 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-dd2be51d21fade82820f71916b37d40c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=104 accepted=151/218 ttft=0.108 decode=6.104 elapsed=6.212 queued=0.000 tok/s=41.8 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 8, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-5b6eb6adce5d0244301b6a01af6b4710 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=106 accepted=149/206 ttft=0.110 decode=6.238 elapsed=6.348 queued=0.000 tok/s=40.9 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 9, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-623b0cc3d62212641611a2f9a89341fb prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=119 accepted=136/196 ttft=0.109 decode=6.702 elapsed=6.812 queued=0.000 tok/s=38.0 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 10, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-9cccd1d72780ba369d4a0f7f1cf6b7de prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.213 elapsed=11.315 queued=0.000 tok/s=22.7 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 11, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-3b96d47df8cc2c8cb7768fbe61599784 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.245 elapsed=11.348 queued=0.000 tok/s=22.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-5e51024c1ea9fff3f29178763b796939 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=255 accepted=0/0 ttft=0.101 decode=11.206 elapsed=11.307 queued=0.000 tok/s=22.8 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 13, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-5605276f0c9f7ca48e25902ce3d135fb prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.201 elapsed=11.304 queued=0.000 tok/s=22.8 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 14, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-566c3090238ff34963ffe1860bb3a3c7 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.221 elapsed=11.324 queued=0.000 tok/s=22.7 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 15, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-39b66adb2bed083a0d941ce3d1621357 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.236 elapsed=11.339 queued=0.000 tok/s=22.7 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-b32473a19291db8e5224abb7c672c305 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.248 elapsed=11.350 queued=0.000 tok/s=22.7 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 17, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-542aa0755224e751ef7f4cf64d48aca3 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=255 accepted=0/0 ttft=0.104 decode=11.215 elapsed=11.319 queued=0.000 tok/s=22.7 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 18, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-b94243932578ddf5a62114539bda6c52 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.109 decode=5.274 elapsed=5.384 queued=0.000 tok/s=48.3 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-b73a521d874f4e375acbe185fd1b11ce prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.273 decode=7.362 elapsed=7.635 queued=0.000 tok/s=34.6 seed=1234 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-84ddfb1ae4d5160266feb17a34239825 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=96 accepted=159/273 ttft=0.109 decode=8.610 elapsed=8.719 queued=0.000 tok/s=29.6 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 21, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-bb7c5baab031c57d86ddfff5ac50040d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.276 decode=11.059 elapsed=11.335 queued=0.167 tok/s=23.1 seed=1234 +[tensorfold] done cmpl-a1144548b3673f80ee5aa97fcd6884c1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=89 accepted=167/263 ttft=0.110 decode=12.523 elapsed=12.633 queued=0.000 tok/s=20.4 seed=1237 +[tensorfold] done cmpl-73e5f2d2c7a6aca219689be64c3aad1c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=89 accepted=166/264 ttft=0.682 decode=12.228 elapsed=12.910 queued=0.167 tok/s=20.9 seed=1236 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-205376c734508c3bb8db256536ea1535 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=95 accepted=161/285 ttft=0.469 decode=12.765 elapsed=13.234 queued=0.167 tok/s=20.0 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 27, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-a14cbe2969aad4fee5a62fa8714d9dc8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.923 decode=17.408 elapsed=18.331 queued=0.167 tok/s=14.6 seed=1234 +[tensorfold] done cmpl-6ee712fcb38772db947843944cfb5920 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=79 accepted=176/236 ttft=1.461 decode=17.675 elapsed=19.135 queued=0.166 tok/s=14.4 seed=1238 +[tensorfold] done cmpl-864fb43f553a424d1e70739bcae5bd37 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=89 accepted=166/264 ttft=0.687 decode=19.742 elapsed=20.429 queued=0.167 tok/s=12.9 seed=1236 +[tensorfold] done cmpl-0102b98094c8714cd6ec6e7aa517f818 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=92 accepted=163/271 ttft=0.109 decode=20.320 elapsed=20.429 queued=0.000 tok/s=12.5 seed=1240 +[tensorfold] done cmpl-7cd8dbfd596bfe96d1e88e0838a7b22a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=89 accepted=166/264 ttft=1.185 decode=19.519 elapsed=20.704 queued=0.167 tok/s=13.1 seed=1239 +[tensorfold] done cmpl-d33e85b101a6ec71f79b310b150a5cb0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=89 accepted=167/264 ttft=1.757 decode=19.173 elapsed=20.930 queued=0.166 tok/s=13.3 seed=1237 +[tensorfold] done cmpl-e3073fab112bdfae2f39ea7c7889488d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=95 accepted=160/284 ttft=0.466 decode=20.558 elapsed=21.024 queued=0.000 tok/s=12.4 seed=1235 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-d82ccd7af5648e0d20f17714d0eddc3e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=102 accepted=153/301 ttft=0.276 decode=21.115 elapsed=21.391 queued=0.000 tok/s=12.1 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 33, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-392fc1ae479510119b79806b26b75980 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=3f46b826a241 finish=length rounds=10 accepted=5/13 ttft=0.161 decode=0.543 elapsed=0.704 queued=0.000 tok/s=27.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 34, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-2bf7caf1a8f8c0025a85fe31baa69955 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=135 accepted=120/176 ttft=0.160 decode=7.263 elapsed=7.423 queued=0.000 tok/s=35.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 35, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-ceee011f0fca6b5e9beae46ffab21fd2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=106 accepted=150/234 ttft=0.161 decode=6.420 elapsed=6.581 queued=0.000 tok/s=39.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 36, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-090c29c5d8a3678f6e2e80c765d0486d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=133 accepted=123/191 ttft=0.158 decode=7.297 elapsed=7.455 queued=0.000 tok/s=34.9 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 37, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-4bb9902163b0d337cdc6b6fd7f148bcf prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=141 accepted=115/175 ttft=0.160 decode=7.547 elapsed=7.708 queued=0.000 tok/s=33.8 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 38, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-d671a1bfb5ee1b3f14f7719dc59b1299 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=135 accepted=120/197 ttft=0.161 decode=7.492 elapsed=7.653 queued=0.000 tok/s=34.0 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 39, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-695b56e849c645e04502812bce8f4dbf prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=128 accepted=128/208 ttft=0.162 decode=7.212 elapsed=7.374 queued=0.000 tok/s=35.4 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 40, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-6d02fbc4a3b131fb85311b12f75b3464 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=124 accepted=131/205 ttft=0.161 decode=6.975 elapsed=7.136 queued=0.000 tok/s=36.6 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 41, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-8118e91b627a62a100256673d41dc51d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=123 accepted=132/204 ttft=0.161 decode=6.945 elapsed=7.107 queued=0.000 tok/s=36.7 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-b27aec02e96040a3f9fc85683fb49d2b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.262 elapsed=11.414 queued=0.000 tok/s=22.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 43, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-81276721a20fc88470f105103a92632f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.241 elapsed=11.393 queued=0.000 tok/s=22.7 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 44, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-cfaa80c112d7f8f658c62e6819425923 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.225 elapsed=11.378 queued=0.000 tok/s=22.7 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 45, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-243e13d0a64d7715c73f6a39640a5c47 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.240 elapsed=11.393 queued=0.000 tok/s=22.7 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 46, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-1d1bb3aaa8fc08aba7925d02b7120b0a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=255 accepted=0/0 ttft=0.150 decode=11.235 elapsed=11.385 queued=0.000 tok/s=22.7 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 47, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-493f2dcb7f74b2e6649fb7d29fd62127 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.233 elapsed=11.385 queued=0.000 tok/s=22.7 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-429ec40565b845b8b3eadcc67b65270a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.233 elapsed=11.386 queued=0.000 tok/s=22.7 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 49, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-cd5bfa4806c17e06d1808fc7120c980c prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=255 accepted=0/0 ttft=0.151 decode=11.247 elapsed=11.398 queued=0.000 tok/s=22.7 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 50, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-0b1eac3e58bd6d39431ed58b503de7b8 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=135 accepted=120/176 ttft=0.160 decode=7.280 elapsed=7.440 queued=0.000 tok/s=35.0 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-c50b4955c1ab2cd7ece4eb12928521dd prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=93 accepted=162/278 ttft=0.380 decode=8.838 elapsed=9.218 queued=0.000 tok/s=28.9 seed=1235 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-9281660e55af4e8e8cf150a0e2e83649 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=104 accepted=151/292 ttft=0.160 decode=9.579 elapsed=9.739 queued=0.000 tok/s=26.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-9130b4158b46f616ca89b56741292a1a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=93 accepted=162/278 ttft=0.381 decode=13.305 elapsed=13.686 queued=0.000 tok/s=19.2 seed=1235 +[tensorfold] done chatcmpl-9d58a0c493ce5124c642663151e099e5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=100 accepted=156/297 ttft=0.623 decode=14.001 elapsed=14.624 queued=0.000 tok/s=18.2 seed=1237 +[tensorfold] done chatcmpl-811e7fb25afe5725e320c59a32333a37 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=102 accepted=154/302 ttft=0.161 decode=14.464 elapsed=14.624 queued=0.000 tok/s=17.6 seed=1234 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-7a5b94054615a066589aff00c3a31760 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=107 accepted=149/306 ttft=0.889 decode=14.158 elapsed=15.047 queued=0.000 tok/s=18.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 58, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-2fdedcc44f908f2afcede76f6bb051bf prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=93 accepted=162/278 ttft=1.474 decode=20.549 elapsed=22.024 queued=0.218 tok/s=12.4 seed=1235 +[tensorfold] done chatcmpl-e08f7e4b1d49c6bb37b52e6632cea970 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=101 accepted=154/298 ttft=0.617 decode=22.397 elapsed=23.014 queued=0.219 tok/s=11.4 seed=1241 +[tensorfold] done chatcmpl-240700f4d0982ef6a807c95c09c8502b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=100 accepted=156/297 ttft=0.888 decode=22.126 elapsed=23.014 queued=0.219 tok/s=11.5 seed=1237 +[tensorfold] done chatcmpl-4eab4a3f80feadb76ff16040fe80d4b5 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=105 accepted=151/309 ttft=0.160 decode=23.156 elapsed=23.316 queued=0.000 tok/s=11.0 seed=1236 +[tensorfold] done chatcmpl-cc0dadb4c2bc51a7040cb8c3cfd8c76e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=102 accepted=154/303 ttft=1.175 decode=22.276 elapsed=23.451 queued=0.219 tok/s=11.4 seed=1234 +[tensorfold] done chatcmpl-01b0a83a9b1fc191ec0000f0d515646d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=103 accepted=153/304 ttft=2.146 decode=21.761 elapsed=23.907 queued=0.217 tok/s=11.7 seed=1239 +[tensorfold] done chatcmpl-09abfca0988fe43af6c72ddd16fd8b69 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=106 accepted=149/316 ttft=1.799 decode=22.273 elapsed=24.072 queued=0.217 tok/s=11.4 seed=1240 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-886c047802e7bfdf15cbb63070ecacd2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=112 accepted=143/332 ttft=0.378 decode=23.748 elapsed=24.126 queued=0.000 tok/s=10.7 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 65, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-cf430d122578ed4750663406416d49ac prompt=14 cached=0 thinking=False effort=none tokens=16 sha=3000ccd48624 finish=length rounds=5 accepted=10/11 ttft=0.109 decode=0.301 elapsed=0.410 queued=0.000 tok/s=49.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 66, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-e90fa74eedeb5610af17465dd8c11796 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=51 accepted=204/222 ttft=0.111 decode=3.621 elapsed=3.732 queued=0.000 tok/s=70.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 67, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-3c0987e6924292d86a844bf7d86bfea9 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=255 accepted=0/0 ttft=0.101 decode=11.254 elapsed=11.355 queued=0.000 tok/s=22.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 68, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-e9150695d2876760bbd97b0832adfb50 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=51 accepted=204/222 ttft=0.109 decode=3.621 elapsed=3.729 queued=0.000 tok/s=70.4 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 69, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 70, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-8b83038f42bc843fc907dbd6afc7011c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.108 decode=6.182 elapsed=6.290 queued=0.000 tok/s=41.2 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-a280369e2e6d1e5fcf7a1cda9d841217 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.274 decode=6.073 elapsed=6.347 queued=0.167 tok/s=42.0 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 71, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 74, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 74, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 74, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-ebcd29fb98037365b45aa6daa0a50072 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.111 decode=8.622 elapsed=8.732 queued=0.000 tok/s=29.6 +[tensorfold] done cmpl-4115be6962c43654fc67923c3d469bca prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.278 decode=8.556 elapsed=8.834 queued=0.170 tok/s=29.8 +[tensorfold] done cmpl-918a38544992030d4ca544cff1ceca48 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.467 decode=8.451 elapsed=8.918 queued=0.170 tok/s=30.2 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-4c889cc5b83244800f1606e963a282da prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.679 decode=8.295 elapsed=8.974 queued=0.169 tok/s=30.7 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 75, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-d2828826b5dcf5d1b6cbb6099faac221 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.112 decode=12.684 elapsed=12.796 queued=0.000 tok/s=20.1 +[tensorfold] done cmpl-1758d2ced284ce64141ef9ee9c66fa42 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.275 decode=12.676 elapsed=12.951 queued=0.170 tok/s=20.1 +[tensorfold] done cmpl-a60dc33519c8ab748803b30ef31b2bed prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.466 decode=12.630 elapsed=13.095 queued=0.169 tok/s=20.2 +[tensorfold] done cmpl-7f176a1f849a8de06cb5f75ea86ad33d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.676 decode=12.548 elapsed=13.224 queued=0.169 tok/s=20.3 +[tensorfold] done cmpl-d84e3603d2504d7778372de02ea7bb78 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.903 decode=12.435 elapsed=13.338 queued=0.168 tok/s=20.5 +[tensorfold] done cmpl-df00faa1ffd3a0f0b98b49d528085329 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.151 decode=12.288 elapsed=13.439 queued=0.168 tok/s=20.8 +[tensorfold] done cmpl-80869362c456b503277c195d307d9203 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.413 decode=12.110 elapsed=13.523 queued=0.168 tok/s=21.1 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done cmpl-e827a98bcfe1d735bc45fd4d96d7bbf3 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.691 decode=11.890 elapsed=13.580 queued=0.168 tok/s=21.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 83, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-7cbd1c7a395ac4dc35a5a1a50558a47d prompt=32 cached=0 thinking=False effort=none tokens=16 sha=e3dfbb415b68 finish=length rounds=5 accepted=10/12 ttft=0.158 decode=0.312 elapsed=0.470 queued=0.000 tok/s=48.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 84, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-ed337f003dae5e4363c5b27717869f95 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=136 accepted=119/185 ttft=0.158 decode=7.307 elapsed=7.464 queued=0.000 tok/s=34.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 85, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-7014c6b51dc8a37eabc058a63d18e319 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=255 accepted=0/0 ttft=0.150 decode=11.246 elapsed=11.397 queued=0.000 tok/s=22.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 86, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-40c44e77f0cc3412eeffb3e75b91fc28 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=136 accepted=119/185 ttft=0.159 decode=7.298 elapsed=7.457 queued=0.000 tok/s=34.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 87, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 88, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-21feecc5398f722b60bb2105885041cb prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.159 decode=10.072 elapsed=10.231 queued=0.000 tok/s=25.3 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-b8af9d5f23e3d7dccd26fa7a05b48d7d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.376 decode=9.904 elapsed=10.280 queued=0.219 tok/s=25.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 89, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-1aec1d8d63b0dd9361acd45fa74d3f07 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.160 decode=14.491 elapsed=14.651 queued=0.000 tok/s=17.6 +[tensorfold] done chatcmpl-73da582f473868a5c3050cb79407ec39 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.378 decode=14.364 elapsed=14.741 queued=0.220 tok/s=17.8 +[tensorfold] done chatcmpl-e9866b5afe95c161f84b947c2ee3bcf3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.618 decode=14.190 elapsed=14.808 queued=0.220 tok/s=18.0 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-f0d7f96b314474464d4aacf6608b12e0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.885 decode=13.971 elapsed=14.856 queued=0.220 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 93, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-36d6531239589bc70d80564e26b0e5f9 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.160 decode=22.495 elapsed=22.655 queued=0.000 tok/s=11.3 +[tensorfold] done chatcmpl-2d0964e211d89da3ebc28c354ca3c33d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.378 decode=22.435 elapsed=22.813 queued=0.220 tok/s=11.4 +[tensorfold] done chatcmpl-33383949539498388b8dea391809b620 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.618 decode=22.337 elapsed=22.955 queued=0.220 tok/s=11.4 +[tensorfold] done chatcmpl-7fea157d310fcec4151018102666dcba prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.882 decode=22.202 elapsed=23.084 queued=0.219 tok/s=11.5 +[tensorfold] done chatcmpl-017d76f9f2fda6ac9c9436f1f65c5988 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.160 decode=22.035 elapsed=23.195 queued=0.219 tok/s=11.6 +[tensorfold] done chatcmpl-359c185fda1ca15b1ab61a91dbc8ca03 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.456 decode=21.830 elapsed=23.286 queued=0.219 tok/s=11.7 +[tensorfold] done chatcmpl-f235d1262e98cc8c97442a88d7eb8c9d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.766 decode=21.586 elapsed=23.353 queued=0.219 tok/s=11.8 +info: kv pool idle: promised 0, free 1544 + store 0 == total 1544 +[tensorfold] done chatcmpl-87f069cca2003ae645b6f46b2b7bb1b0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=2.094 decode=21.308 elapsed=23.401 queued=0.218 tok/s=12.0 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1912 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1913 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:4542:4559 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6933 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6934 pkey=0 pd=0xe2bb24053590 oooRq=0 +host-a:4542:4559 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6935 pkey=0 pd=0xe2bb24053590 +host-a:4542:4542 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/server-rank1.log new file mode 100644 index 0000000000..afdd8509bc --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/server-rank1.log @@ -0,0 +1,8084 @@ +[kvp-serve] 2026-10-23T01:49:05+00:00 rank 1 of 3 context 98304 parallel 8 binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=all TF_GLM_MM=unset args +info: rank 1: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-b:4045:4045 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:4045:4045 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:4045:4045 [0] NCCL INFO cudaDriverVersion 13030 +host-b:4045:4045 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:4045:4045 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:4045:4045 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:4045:4045 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:4045:4045 [0] NCCL INFO Count not open libibverbs.so +host-b:4045:4045 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:4045:4045 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:4045:4045 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:4045:4045 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0xcbfb580 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:4045:4045 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0xcc3cc80 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:4045:4045 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0xcc7e380 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:4045:4045 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0xccbfa80 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:4045:4045 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:4045:4045 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:4045:4045 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:4045:4045 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:4045:4045 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:4045:4045 [0] NCCL INFO Initialized NET plugin IB +host-b:4045:4045 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:4045:4045 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:4045:4045 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:4045:4045 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:4045:4045 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:4045:4045 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:4045:4045 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:4045:4045 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:4045:4045 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:4045:4045 [0] NCCL INFO Using network IB +host-b:4045:4045 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:4045:4045 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x9fc3f10 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x21607d5aa2555317 - Init START +host-b:4045:4045 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:4045:4045 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:4045:4045 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:4045:4045 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:4045:4045 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:4045:4045 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:4045:4045 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:4045:4045 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:4045:4045 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:4045:4045 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:4045:4045 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:4045:4045 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:4045:4045 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:4045:4045 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:4045:4045 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:4045:4045 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:4045:4045 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:4045:4045 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:4045:4045 [0] NCCL INFO Rank 1: 6 Net devices +host-b:4045:4045 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:4045:4045 [0] NCCL INFO comm 0x9fc3f10 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:4045:4045 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:4045:4045 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:4045:4045 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:4045:4045 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:4045:4045 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:4045:4045 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:4045:4061 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 4 +host-b:4045:4060 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-b:4045:4060 [0] NCCL INFO proxy listening socket at <52499> +host-b:4045:4045 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:4045:4062 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-b:4045:4045 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:4045:4045 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:4045:4045 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:4045:4045 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:4045:4045 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:4045:4045 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:4045:4045 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b8010000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b8054000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b809c000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2155 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2156 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2155 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2155 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2156 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2156 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2283 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2284 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2283 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d958b ctsFifoLkey=0x1d958b +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2283 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2284 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d958b ctsFifoLkey=0x1d958b +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2284 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b81aa000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6699 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6700 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6701 pkey=0 pd=0xff69b8123a60 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6731 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6732 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6733 pkey=0 pd=0xff69b820cd90 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b82fa000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2155 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2156 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2283 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2284 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2157 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2158 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2157 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b7977 ctsFifoLkey=0x1b7977 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2157 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2158 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b7977 ctsFifoLkey=0x1b7977 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2158 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b8393000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b83db000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b841f000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2157 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2158 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2285 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2286 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2285 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de9e6 ctsFifoLkey=0x1de9e6 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2285 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2286 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de9e6 ctsFifoLkey=0x1de9e6 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2286 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6702 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6703 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6704 pkey=0 pd=0xff69b8123a60 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2285 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2286 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6734 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6735 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6736 pkey=0 pd=0xff69b820cd90 +host-b:4045:4045 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:4045:4045 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:4045:4045 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:4045:4045 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:4045:4045 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b85ac000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b85f1000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b8636000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b867e000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2159 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2160 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2161 pkey=0 pd=0xff69b8053590 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2287 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2288 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2289 pkey=0 pd=0xff69b80f2420 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2162 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2163 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2164 pkey=0 pd=0xff69b8053590 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2290 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2291 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2292 pkey=0 pd=0xff69b80f2420 +host-b:4045:4045 [0] NCCL INFO Connected all trees +host-b:4045:4045 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:4045:4045 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:4045:4045 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:4045:4045 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b88ac000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b88ed000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6705 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6706 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6705 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1726e5 ctsFifoLkey=0x1726e5 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6705 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6706 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1726e5 ctsFifoLkey=0x1726e5 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6706 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6737 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6738 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6737 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x18e0df ctsFifoLkey=0x18e0df +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6737 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6738 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x18e0df ctsFifoLkey=0x18e0df +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6738 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b89e0000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b8a24000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6705 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6706 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6737 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6738 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6707 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6708 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6707 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1826e3 ctsFifoLkey=0x1826e3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6707 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6708 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1826e3 ctsFifoLkey=0x1826e3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6708 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6739 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6740 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6739 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x195c5a ctsFifoLkey=0x195c5a +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6739 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6740 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x195c5a ctsFifoLkey=0x195c5a +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6740 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6707 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6708 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6739 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6740 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4045 [0] NCCL INFO Connected binomial trees +host-b:4045:4045 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:4045:4045 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:4045:4045 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:4045:4045 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:4045:4045 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-b:4045:4045 [0] NCCL INFO ncclCommInitRank comm 0x9fc3f10 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x21607d5aa2555317 - Init COMPLETE +host-b:4045:4045 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 0.66 (kernels 0.25, alloc 0.03, bootstrap 0.09, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.27, rest 0.00) +host-b:4045:4045 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:4045:4045 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:4045:4045 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:4045:4045 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:4045:4045 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:4045:4045 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:4045:4045 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:4045:4045 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b8b1a000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b8b5b000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b8ba0000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b8be4000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2165 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2166 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2165 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8481 ctsFifoLkey=0x1b8481 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2165 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2166 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8481 ctsFifoLkey=0x1b8481 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2166 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2293 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2294 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2293 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1dedeb ctsFifoLkey=0x1dedeb +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2293 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2294 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1dedeb ctsFifoLkey=0x1dedeb +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2294 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b8ce2000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6709 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6710 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6711 pkey=0 pd=0xff69b8123a60 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6741 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6742 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6743 pkey=0 pd=0xff69b820cd90 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6744 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6745 pkey=0 pd=0xff69b820cd90 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6744 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c8c5 ctsFifoLkey=0x19c8c5 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6744 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6745 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19c8c5 ctsFifoLkey=0x19c8c5 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6745 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b8e73000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2165 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2166 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2293 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2294 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2295 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2296 pkey=0 pd=0xff69b80f2420 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2297 pkey=0 pd=0xff69b80f2420 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xff69b8f2b000) +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xff69b8f6f000) +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2167 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2168 pkey=0 pd=0xff69b8053590 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2169 pkey=0 pd=0xff69b8053590 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6744 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6745 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6712 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6713 pkey=0 pd=0xff69b8123a60 oooRq=0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6712 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x164401 ctsFifoLkey=0x164401 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6712 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6713 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x164401 ctsFifoLkey=0x164401 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6713 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 1: rank 0 ended; leaving +FO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6712 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4060 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6713 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:4045:4045 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/server-rank2.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/server-rank2.log new file mode 100644 index 0000000000..0275a12b0e --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6all/server-rank2.log @@ -0,0 +1,4089 @@ +[kvp-serve] 2026-10-23T01:49:05+00:00 rank 2 of 3 context 98304 parallel 8 binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=all TF_GLM_MM=unset args +info: rank 2: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-c:4013:4013 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:4013:4013 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:4013:4013 [0] NCCL INFO cudaDriverVersion 13030 +host-c:4013:4013 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:4013:4013 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:4013:4013 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:4013:4013 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:4013:4013 [0] NCCL INFO Count not open libibverbs.so +host-c:4013:4013 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:4013:4013 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:4013:4013 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:4013:4013 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x7653580 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:4013:4013 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x7694c80 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:4013:4013 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x76d6380 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:4013:4013 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x7717a80 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:4013:4013 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:4013:4013 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:4013:4013 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:4013:4013 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:4013:4013 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:4013:4013 [0] NCCL INFO Initialized NET plugin IB +host-c:4013:4013 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:4013:4013 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:4013:4013 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:4013:4013 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:4013:4013 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:4013:4013 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:4013:4013 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:4013:4013 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:4013:4013 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:4013:4013 [0] NCCL INFO Using network IB +host-c:4013:4013 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:4013:4013 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x4a1bf10 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x21607d5aa2555317 - Init START +host-c:4013:4013 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:4013:4013 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:4013:4013 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:4013:4013 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:4013:4013 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:4013:4013 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:4013:4013 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:4013:4013 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:4013:4013 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:4013:4013 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:4013:4013 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:4013:4013 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:4013:4013 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:4013:4013 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:4013:4013 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:4013:4013 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:4013:4013 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:4013:4013 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:4013:4013 [0] NCCL INFO Rank 2: 6 Net devices +host-c:4013:4013 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:4013:4013 [0] NCCL INFO comm 0x4a1bf10 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:4013:4013 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:4013:4013 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:4013:4013 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:4013:4013 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:4013:4013 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:4013:4013 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:4013:4028 [0] NCCL INFO [Proxy Service] Device 0 CPU core 14 +host-c:4013:4028 [0] NCCL INFO proxy listening socket at <47655> +host-c:4013:4029 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-c:4013:4030 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-c:4013:4013 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:4013:4013 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:4013:4013 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:4013:4013 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:4013:4013 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:4013:4013 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:4013:4013 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:4013:4013 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5c010000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5c054000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5c09c000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5c0e0000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5c128000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5c16c000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5c1b4000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5c1f8000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1899 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1900 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1901 pkey=0 pd=0xfbfb5c053eb0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2283 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2284 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2283 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2283 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2284 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2284 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2027 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2028 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2029 pkey=0 pd=0xfbfb5c28cad0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2155 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2156 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2155 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcfcc ctsFifoLkey=0x1bcfcc +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2155 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2156 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcfcc ctsFifoLkey=0x1bcfcc +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2156 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2285 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2286 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2285 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3f41 ctsFifoLkey=0x1d3f41 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2285 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2286 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3f41 ctsFifoLkey=0x1d3f41 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2286 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2157 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2158 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2157 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2157 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2158 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2158 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2283 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2284 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2155 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2156 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1902 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1903 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1904 pkey=0 pd=0xfbfb5c053eb0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2285 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2286 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2157 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2158 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2030 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2031 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2032 pkey=0 pd=0xfbfb5c28cad0 +host-c:4013:4013 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:4013:4013 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:4013:4013 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:4013:4013 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:4013:4013 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:4013:4013 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:4013:4013 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:4013:4013 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:4013:4013 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5c5ab000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5c5ec000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5c634000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1905 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1906 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1905 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1905 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1906 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1906 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2033 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2034 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2033 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197a79 ctsFifoLkey=0x197a79 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2033 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2034 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197a79 ctsFifoLkey=0x197a79 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2034 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5c72e000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5c776000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5c7ba000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5c802000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5c846000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2159 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2160 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2161 pkey=0 pd=0xfbfb5c2d7de0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2287 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2288 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2289 pkey=0 pd=0xfbfb5c25aa40 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1907 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1908 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1907 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1907 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1908 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1908 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2162 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2163 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2164 pkey=0 pd=0xfbfb5c2d7de0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2035 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2036 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2035 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2035 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2036 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2036 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2290 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2291 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2292 pkey=0 pd=0xfbfb5c25aa40 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1905 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1906 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2033 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2034 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1907 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1908 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2035 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2036 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4013 [0] NCCL INFO Connected all trees +host-c:4013:4013 [0] NCCL INFO Connected binomial trees +host-c:4013:4013 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:4013:4013 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:4013:4013 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:4013:4013 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:4013:4013 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-c:4013:4013 [0] NCCL INFO ncclCommInitRank comm 0x4a1bf10 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x21607d5aa2555317 - Init COMPLETE +host-c:4013:4013 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.51 (kernels 0.23, alloc 0.04, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.22, rest 0.00) +host-c:4013:4013 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:4013:4013 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:4013:4013 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:4013:4013 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:4013:4013 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:4013:4013 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:4013:4013 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:4013:4013 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5cb2a000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5cb6b000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5cbb0000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5cbf1000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5cc36000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5cc77000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfbfb5ccbc000) +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfbfb5ccfd000) +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1909 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1910 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1911 pkey=0 pd=0xfbfb5c053eb0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2293 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2294 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2293 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d9d97 ctsFifoLkey=0x1d9d97 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2293 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2294 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d9d97 ctsFifoLkey=0x1d9d97 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2294 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2037 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2038 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2039 pkey=0 pd=0xfbfb5c28cad0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2295 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2296 pkey=0 pd=0xfbfb5c25aa40 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2297 pkey=0 pd=0xfbfb5c25aa40 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2165 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2166 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2165 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1ba09d ctsFifoLkey=0x1ba09d +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2165 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2166 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1ba09d ctsFifoLkey=0x1ba09d +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2166 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2293 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2294 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2040 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2041 pkey=0 pd=0xfbfb5c28cad0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2040 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x198b88 ctsFifoLkey=0x198b88 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2040 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2041 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x198b88 ctsFifoLkey=0x198b88 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2041 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2165 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2166 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2040 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2041 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1912 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1913 pkey=0 pd=0xfbfb5c053eb0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1912 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17c683 ctsFifoLkey=0x17c683 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1912 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1913 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17c683 ctsFifoLkey=0x17c683 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1913 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0]info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 2: rank 0 ended; leaving + NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2167 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2168 pkey=0 pd=0xfbfb5c2d7de0 oooRq=0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2169 pkey=0 pd=0xfbfb5c2d7de0 +host-c:4013:4028 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1912 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4028 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1913 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:4013:4013 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/client.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/client.log new file mode 100644 index 0000000000..284ee6146b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/client.log @@ -0,0 +1,97 @@ +server up 2026-10-23 10:27:12 +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 48.0, "per_stream_tps": 48.0, "ttft_s_max": 0.11, "timing_valid": true, "end_to_end_tps": 47.257280915012245, "makespan_s": 5.417154669994488, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 58.2, "per_stream_tps": 31.6, "ttft_s_max": 0.28, "timing_valid": true, "end_to_end_tps": 57.66509920199466, "makespan_s": 8.878854057053104, "steady_tps": 60.3, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 76.8, "per_stream_tps": 20.1, "ttft_s_max": 0.7, "timing_valid": true, "end_to_end_tps": 76.51423462307243, "makespan_s": 13.383130669011734, "steady_tps": 80.3, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 1.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 95.9, "per_stream_tps": 13.1, "ttft_s_max": 1.77, "timing_valid": true, "end_to_end_tps": 95.74105439383828, "makespan_s": 21.39103243604768, "steady_tps": 102.7, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 35.0, "per_stream_tps": 35.0, "ttft_s_max": 0.16, "timing_valid": true, "end_to_end_tps": 34.410944851454865, "makespan_s": 7.439493483980186, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 53.0, "per_stream_tps": 27.6, "ttft_s_max": 0.38, "timing_valid": true, "end_to_end_tps": 52.30382685011293, "makespan_s": 9.788958683027886, "steady_tps": 55.1, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 68.8, "per_stream_tps": 17.8, "ttft_s_max": 0.87, "timing_valid": true, "end_to_end_tps": 68.33455152681384, "makespan_s": 14.985098710982129, "steady_tps": 72.4, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 1.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 85.3, "per_stream_tps": 11.4, "ttft_s_max": 2.13, "timing_valid": true, "end_to_end_tps": 85.09953798990507, "makespan_s": 24.06593558995519, "steady_tps": 92.1, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 8, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 70.6, "per_stream_tps": 70.6, "ttft_s_max": 0.11, "timing_valid": true, "end_to_end_tps": 68.74996928956655, "makespan_s": 3.7236380269750953, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 81.8, "per_stream_tps": 41.6, "ttft_s_max": 0.28, "timing_valid": true, "end_to_end_tps": 80.73287904860639, "makespan_s": 6.341901912004687, "steady_tps": 83.6, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 112.0, "per_stream_tps": 29.1, "ttft_s_max": 0.68, "timing_valid": true, "end_to_end_tps": 111.06214999689927, "makespan_s": 9.220062820939347, "steady_tps": 117.4, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "code", "temperature": 0.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 151.8, "per_stream_tps": 20.5, "ttft_s_max": 1.7, "timing_valid": true, "end_to_end_tps": 151.14135774187514, "makespan_s": 13.55022894195281, "steady_tps": 166.0, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 1, "failed": 0, "unmeasured": 0, "aggregate_tps": 34.9, "per_stream_tps": 34.9, "ttft_s_max": 0.16, "timing_valid": true, "end_to_end_tps": 34.24796634969847, "makespan_s": 7.474896389059722, "alone": {"equal": 1, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 2, "failed": 0, "unmeasured": 0, "aggregate_tps": 50.6, "per_stream_tps": 25.6, "ttft_s_max": 0.38, "timing_valid": true, "end_to_end_tps": 50.009440527290046, "makespan_s": 10.23806694499217, "steady_tps": 51.6, "alone": {"equal": 2, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 4, "failed": 0, "unmeasured": 0, "aggregate_tps": 69.8, "per_stream_tps": 18.0, "ttft_s_max": 0.89, "timing_valid": true, "end_to_end_tps": 69.31546214563338, "makespan_s": 14.773038630955853, "steady_tps": 72.3, "alone": {"equal": 4, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +{"label": "", "prompt": "chat", "temperature": 0.0, "streams": 8, "failed": 0, "unmeasured": 0, "aggregate_tps": 88.3, "per_stream_tps": 11.6, "ttft_s_max": 2.1, "timing_valid": true, "end_to_end_tps": 88.00613725206625, "makespan_s": 23.27110431098845, "steady_tps": 93.6, "alone": {"equal": 8, "unequal": 0, "failed": 0}, "serial": {"equal": 1, "unequal": 0, "failed": 0}} +logs/r6auto-conc.json: 16 cells; alone 60/60 equal (0 unequal, 0 failed); serial 72/72 equal (0 unequal, 0 failed) + 0 S1 t1 drafted prompt 7759 history 7757 cached 0 (allowed [0]) ttft 5.13s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok + 1 S1 t1 plain prompt 7759 history 7757 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 1.02s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 tool_calls ok +system cuts [4563, 6099, 6611] + 2 S1 t2 drafted prompt 15550 history 15548 cached 6144 (allowed {'min': 1, 'max': 7757}) ttft 6.15s sha 539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b tool_calls ok + 3 S1 t2 plain prompt 15550 history 15548 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 0.83s sha 539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b tool_calls ok + 4 S1 t3 drafted prompt 19462 history 19460 cached 14336 (allowed {'min': 7356, 'max': 15548}) ttft 3.44s sha 8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa tool_calls ok + 5 S1 t3 plain prompt 19462 history 19460 cached 18432 (allowed {'min': 11268, 'max': 19460}) ttft 0.75s sha 8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa tool_calls ok + 6 S1 t4 drafted prompt 21892 history 21890 cached 18432 (allowed {'min': 11268, 'max': 19460}) ttft 2.32s sha dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af tool_calls ok + 7 S1 t4 plain prompt 21892 history 21890 cached 20480 (allowed {'min': 13698, 'max': 21890}) ttft 0.93s sha dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af tool_calls ok + 8 S1 t5 drafted prompt 24388 history 24386 cached 20480 (allowed {'min': 13698, 'max': 21890}) ttft 2.55s sha 8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa length ok + 9 S1 t5 plain prompt 24388 history 24386 cached 22528 (allowed {'min': 16194, 'max': 24386}) ttft 1.15s sha 8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa length ok + 10 S1 t6 drafted prompt 26895 history 26893 cached 22528 (allowed {'min': 16194, 'max': 24386}) ttft 3.12s sha c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1 tool_calls ok + 11 S1 t6 plain prompt 26895 history 26893 cached 26624 (allowed {'min': 18701, 'max': 26893}) ttft 0.40s sha c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1 tool_calls ok + 12 S1 t7 drafted prompt 29798 history 29796 cached 26624 (allowed {'min': 18701, 'max': 26893}) ttft 2.22s sha 7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb tool_calls ok + 13 S1 t7 plain prompt 29798 history 29796 cached 28672 (allowed {'min': 21604, 'max': 29796}) ttft 0.82s sha 7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb tool_calls ok + 14 S1 t8 drafted prompt 35434 history 35432 cached 28672 (allowed {'min': 21604, 'max': 29796}) ttft 4.66s sha 1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e length ok + 15 S1 t8 plain prompt 35434 history 35432 cached 34816 (allowed {'min': 27240, 'max': 35432}) ttft 0.58s sha 1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e length ok + 16 S1 t9 drafted prompt 38152 history 38150 cached 34816 (allowed {'min': 27240, 'max': 35432}) ttft 2.31s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 length ok + 17 S1 t9 plain prompt 38152 history 38150 cached 36864 (allowed {'min': 29958, 'max': 38150}) ttft 0.91s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 length ok + 18 X t1 drafted prompt 2115 history 2113 cached 0 (allowed [0]) ttft 1.49s sha 7b1461bf71da95a2b74977aec25ecf7215b9f0fa044c82d2e1f4905ed66b798a length ok + 19 A t1 drafted prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 0.90s sha d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666 tool_calls ok + 20 A t1 plain prompt 7412 history 7410 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 0.86s sha d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666 tool_calls ok + 21 B t1 drafted prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 6611}) ttft 0.93s sha 9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb tool_calls ok + 22 B t1 plain prompt 7474 history 7472 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 0.88s sha 9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb tool_calls ok + 23 A t2 drafted prompt 12632 history 12630 cached 6144 (allowed {'min': 1, 'max': 7410}) ttft 4.42s sha e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe length ok + 24 A t2 plain prompt 12632 history 12630 cached 12288 (allowed {'min': 4438, 'max': 12630}) ttft 0.43s sha e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe length ok + 25 B t2 drafted prompt 12721 history 12719 cached 6144 (allowed {'min': 1, 'max': 7472}) ttft 4.44s sha f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2 tool_calls ok + 26 B t2 plain prompt 12721 history 12719 cached 12288 (allowed {'min': 4527, 'max': 12719}) ttft 0.47s sha f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2 tool_calls ok + 27 A t3 drafted prompt 15383 history 15381 cached 12288 (allowed {'min': 4438, 'max': 12630}) ttft 2.13s sha 7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8 tool_calls ok + 28 A t3 plain prompt 15383 history 15381 cached 14336 (allowed {'min': 7189, 'max': 15381}) ttft 0.76s sha 7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8 tool_calls ok + 29 B t3 drafted prompt 17330 history 17328 cached 12288 (allowed {'min': 4527, 'max': 12719}) ttft 3.42s sha 8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49 length ok + 30 B t3 plain prompt 17330 history 17328 cached 16384 (allowed {'min': 9136, 'max': 17328}) ttft 0.72s sha 8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49 length ok + 31 E t1 drafted prompt 40901 history 40899 cached 6144 (allowed {'min': 0, 'max': 6611}) ttft 22.65s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 length ok + 32 E t1 plain prompt 40901 history 40899 cached 38912 (allowed {'min': 32707, 'max': 40899}) ttft 1.26s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 length ok + 33 T t1 drafted prompt 1520 history 1517 cached 0 (allowed [0]) ttft 0.92s sha 033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89 stop ok + 34 T t1 plain prompt 1520 history 1517 cached 0 (allowed [0]) ttft 0.89s sha 033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89 stop ok + 35 T t2 drafted prompt 4202 history 4199 cached 0 (allowed [0]) ttft 3.05s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 36 T t2 plain prompt 4202 history 4199 cached 4096 (allowed {'min': 1, 'max': 4199}) ttft 0.27s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa stop ok + 37 C t1 cancel prompt None history 0 cached None (allowed None) ttft 0.00s sha None None ok + 38 C t1 drafted prompt 9702 history 9700 cached 6144 (allowed {'min': 0, 'max': 9688}) ttft 2.39s sha 88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c tool_calls ok + 39 C t1 plain prompt 9702 history 9700 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 0.97s sha 88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c tool_calls ok + 40 C t2 drafted prompt 15180 history 15178 cached 8192 (allowed {'min': 1508, 'max': 9700}) ttft 4.68s sha 5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8 length ok + 41 C t2 plain prompt 15180 history 15178 cached 14336 (allowed {'min': 6986, 'max': 15178}) ttft 0.68s sha 5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8 length ok + 42 A t4 drafted prompt 18246 history 18244 cached 14336 (allowed {'min': 7189, 'max': 15381}) ttft 2.54s sha 9599ff0fa9b19bbb5cb5218a5b6f538de1146ce12dad36c729686f12f666389e length ok + 43 B t4 drafted prompt 19808 history 19806 cached 16384 (allowed {'min': 9136, 'max': 17328}) ttft 4.97s sha 025c4856c697674513547e9be3c33a84537e8a4fcb74e731ea09263af3e6d80d tool_calls ok +record: 44 requests, 0 not cached as planned +{"length": 1024, "rep": 0, "ttft": 0.7270355030195788, "prompt": 1023, "cached": 0} +{"length": 2048, "rep": 0, "ttft": 1.2436724250437692, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 1, "ttft": 1.24278286495246, "prompt": 2048, "cached": 0} +{"length": 2048, "rep": 2, "ttft": 1.2355036230292171, "prompt": 2047, "cached": 0} +{"length": 8192, "rep": 0, "ttft": 5.396785218035802, "prompt": 8191, "cached": 0} +{"length": 8192, "rep": 1, "ttft": 5.406543482095003, "prompt": 8192, "cached": 0} +{"length": 8192, "rep": 2, "ttft": 5.404332814970985, "prompt": 8191, "cached": 0} +{"length": 16384, "rep": 0, "ttft": 10.687900383025408, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 1, "ttft": 10.693931889953092, "prompt": 16384, "cached": 0} +{"length": 16384, "rep": 2, "ttft": 10.713210720103234, "prompt": 16383, "cached": 0} +{"length": 32768, "rep": 0, "ttft": 21.40478678198997, "prompt": 32768, "cached": 0} +{"length": 32768, "rep": 1, "ttft": 21.408089085947722, "prompt": 32766, "cached": 0} +{"length": 32768, "rep": 2, "ttft": 21.37812329805456, "prompt": 32766, "cached": 0} +{"length": 65536, "rep": 0, "ttft": 43.39012634602841, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 1, "ttft": 43.3315977670718, "prompt": 65536, "cached": 0} +{"length": 65536, "rep": 2, "ttft": 43.32970841496717, "prompt": 65536, "cached": 0} +alone p2048-0-0: prompt 2048 ttft 1.260909931967035 sha 7d62171119f86883ec9ea4e3542846c00509f1ad0baba0a0676ccd3b8e14cd56 +alone p2048-1-0: prompt 2048 ttft 1.2505641130264848 sha c948b4bcbd5833ccd9c3bbd0f64e960f0ed36def137ba411ed3ade9437416854 +alone p2048-2-0: prompt 2047 ttft 1.2464342470047995 sha 6a2fb209181c72c924d2f9c33c76a176bd29daed59cf2e524f9797dc123b2b19 +alone p2048-0-1: prompt 2048 ttft 1.2466398840770125 sha f8aaeecc59d0c280cda17bf00ec2c94ab68bf77c8f76edf8f6198403d239a62b +alone p2048-1-1: prompt 2048 ttft 1.2500573369907215 sha 854ffaa5d75926074fa12f5f072ace31fab67f02428f5cc5cde0f8536b956273 +alone p2048-2-1: prompt 2047 ttft 1.2476839119335636 sha 0effe3e2e261cfefd427f381703c20e16f63a81a352520ec51fc2d517053699c +alone p2048-0-2: prompt 2048 ttft 1.252152565983124 sha 2d30fcc66f9925bb2dbab2bfa2c3b819cfbe1e820d089d4774f26089df2b46b3 + p2048-0-0: prompt 2048 first 5.98 s last 21.30 s equal_alone True started after - + p2048-1-0: prompt 2048 first 7.36 s last 22.24 s equal_alone True started after - + p2048-2-0: prompt 2047 first 3.28 s last 21.56 s equal_alone True started after - + p2048-0-1: prompt 2048 first 11.63 s last 21.56 s equal_alone True started after - + p2048-1-1: prompt 2048 first 8.77 s last 22.79 s equal_alone True started after - + p2048-2-1: prompt 2047 first 4.60 s last 20.38 s equal_alone True started after - + p2048-0-2: prompt 2048 first 10.19 s last 21.14 s equal_alone True started after - + long65536: prompt 65536 first 1.45 s last 19.51 s equal_alone None started after - +together == alone: 7/7; 0 request(s) started only after another ended +=== kvp-client r6auto rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/cold.json b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/cold.json new file mode 100644 index 0000000000..54e3c65a3d --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/cold.json @@ -0,0 +1 @@ +[{"length": 1024, "rep": 0, "ttft": 0.7270355030195788, "prompt": 1023, "cached": 0}, {"length": 2048, "rep": 0, "ttft": 1.2436724250437692, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 1, "ttft": 1.24278286495246, "prompt": 2048, "cached": 0}, {"length": 2048, "rep": 2, "ttft": 1.2355036230292171, "prompt": 2047, "cached": 0}, {"length": 8192, "rep": 0, "ttft": 5.396785218035802, "prompt": 8191, "cached": 0}, {"length": 8192, "rep": 1, "ttft": 5.406543482095003, "prompt": 8192, "cached": 0}, {"length": 8192, "rep": 2, "ttft": 5.404332814970985, "prompt": 8191, "cached": 0}, {"length": 16384, "rep": 0, "ttft": 10.687900383025408, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 1, "ttft": 10.693931889953092, "prompt": 16384, "cached": 0}, {"length": 16384, "rep": 2, "ttft": 10.713210720103234, "prompt": 16383, "cached": 0}, {"length": 32768, "rep": 0, "ttft": 21.40478678198997, "prompt": 32768, "cached": 0}, {"length": 32768, "rep": 1, "ttft": 21.408089085947722, "prompt": 32766, "cached": 0}, {"length": 32768, "rep": 2, "ttft": 21.37812329805456, "prompt": 32766, "cached": 0}, {"length": 65536, "rep": 0, "ttft": 43.39012634602841, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 1, "ttft": 43.3315977670718, "prompt": 65536, "cached": 0}, {"length": 65536, "rep": 2, "ttft": 43.32970841496717, "prompt": 65536, "cached": 0}] diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/conc.json b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/conc.json new file mode 100644 index 0000000000..6a255c567d --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/conc.json @@ -0,0 +1,9510 @@ +{ + "label": "", + "tokens": 256, + "mixed": false, + "cells": [ + { + "prompt": "code", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.0, + "per_stream_tps": 48.0, + "ttft_s_max": 0.11, + "timing_valid": true, + "end_to_end_tps": 47.257280915012245, + "makespan_s": 5.417154669994488, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 48.040566938197784, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 5.417154669994488, + "end_to_end_tps": 47.257280915012245, + "timing_valid": true, + "per_stream_tps": [ + 48.0 + ], + "ttft_s": [ + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + } + ], + "client_intervals": [ + { + "start": 752492.284347998, + "first": 752492.393372679, + "last": 752497.70138664, + "end": 752497.701502668, + "elapsed_s": 5.417154669994488, + "chunk_offsets_s": [ + 0.10902468103449792, + 0.16906518605537713, + 0.2264645490795374, + 0.28623787907417864, + 0.35086008498910815, + 0.4177049830323085, + 0.48160506901331246, + 0.5453930150251836, + 0.6100175720639527, + 0.6770598570583388, + 0.7433835919946432, + 0.8108393370639533, + 0.8782924410188571, + 0.9443408410297707, + 1.0104080729652196, + 1.0718430620618165, + 1.1294976989738643, + 1.1846810430288315, + 1.2356743060518056, + 1.2893356219865382, + 1.348603339982219, + 1.4094148620497435, + 1.4691978270420805, + 1.5284197230357677, + 1.5892781550064683, + 1.6576746370410547, + 1.7271939140046015, + 1.7938404730521142, + 1.8573475390439853, + 1.923628782038577, + 1.9890247760340571, + 2.055033740005456, + 2.1206517110113055, + 2.1866268520243466, + 2.255597396986559, + 2.3226337829837576, + 2.390005821012892, + 2.4574784639989957, + 2.527945582056418, + 2.597058505984023, + 2.66565098403953, + 2.732296795002185, + 2.7993045289767906, + 2.8686446960782632, + 2.9369536180747673, + 3.014803162077442, + 3.090679668006487, + 3.1600033710710704, + 3.2308296960545704, + 3.299386178026907, + 3.367782793007791, + 3.437093921005726, + 3.5058646680554375, + 3.573163800057955, + 3.6399320390773937, + 3.707427822984755, + 3.771898670005612, + 3.8304150940384716, + 3.8901323260506615, + 3.9532949610147625, + 4.019825226045214, + 4.092997538973577, + 4.186425296007656, + 4.257825365057215, + 4.324825807008892, + 4.3935167350573465, + 4.458080155076459, + 4.523584599024616, + 4.613724643015303, + 4.679688050062396, + 4.745336172054522, + 4.811717070057057, + 4.879024191992357, + 4.9447848550044, + 5.015186161035672, + 5.081818970036693, + 5.157724617980421, + 5.235256436048076, + 5.303397338022478, + 5.368058372987434, + 5.417038641986437 + ], + "chunk_gaps_ms": { + "count": 80, + "p50": 66.63930998183787, + "p95": 75.90564794372767, + "max": 93.42775703407824 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 58.2, + "per_stream_tps": 31.6, + "ttft_s_max": 0.28, + "timing_valid": true, + "end_to_end_tps": 57.66509920199466, + "makespan_s": 8.878854057053104, + "steady_tps": 60.3, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 58.171976642854254, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 8.878854057053104, + "end_to_end_tps": 57.66509920199466, + "timing_valid": true, + "steady_tps": 60.327940111798355, + "per_stream_tps": [ + 34.0, + 29.1 + ], + "ttft_s": [ + 0.28, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + } + ], + "client_intervals": [ + { + "start": 752497.703659539, + "first": 752497.982115304, + "last": 752505.474988901, + "end": 752505.475240509, + "elapsed_s": 7.771580970031209, + "chunk_offsets_s": [ + 0.27845576498657465, + 0.3596239440375939, + 0.4504967239918187, + 0.5445880190236494, + 0.6400309019954875, + 0.7440134000498801, + 0.8369068800238892, + 0.9302546010585502, + 1.0211354759521782, + 1.1178041090024635, + 1.2153930310159922, + 1.3114292250247672, + 1.4064615820534527, + 1.5005057309754193, + 1.602842562017031, + 1.701636440004222, + 1.7951381279854104, + 1.8862777990289032, + 1.979098886018619, + 2.0761805650545284, + 2.173163656028919, + 2.27818025497254, + 2.378794157062657, + 2.4749492480186746, + 2.5736091029830277, + 2.6712349290028214, + 2.766832374036312, + 2.8628121269866824, + 2.9563852089922875, + 3.049426452955231, + 3.148327579954639, + 3.254209114005789, + 3.3558160109678283, + 3.4532729670172557, + 3.549483916023746, + 3.6457127849571407, + 3.7430531850550324, + 3.841049515060149, + 3.9356325670378283, + 4.031629267963581, + 4.142302180989645, + 4.239188277046196, + 4.336750993970782, + 4.43625319795683, + 4.536523743998259, + 4.637909100973047, + 4.737616001977585, + 4.837357813958079, + 4.933180044987239, + 5.0401529429946095, + 5.143164069042541, + 5.240748515003361, + 5.337352868053131, + 5.433638544054702, + 5.530357150011696, + 5.626745157991536, + 5.725319109973498, + 5.81908998999279, + 5.922928454005159, + 6.026160952984355, + 6.128947659977712, + 6.2287888820283115, + 6.327567519037984, + 6.4225942339980975, + 6.516275953035802, + 6.615725832059979, + 6.711570631014183, + 6.806071475963108, + 6.901705609983765, + 7.008584551978856, + 7.109045261982828, + 7.2076185459736735, + 7.308295428054407, + 7.410436294041574, + 7.506181612960063, + 7.6010221460601315, + 7.690947370952927, + 7.771329362061806 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 96.88609605655074, + "p95": 105.88153405115008, + "max": 110.67291302606463 + } + }, + { + "start": 752497.703324558, + "first": 752497.814657636, + "last": 752506.581765605, + "end": 752506.582178615, + "elapsed_s": 8.878854057053104, + "chunk_offsets_s": [ + 0.11133307800628245, + 0.17044383706524968, + 0.3597722430713475, + 0.4507348120678216, + 0.5447207989636809, + 0.640467238961719, + 0.7445066480431706, + 0.83715000806842, + 0.9303931719623506, + 1.0216063889674842, + 1.1183413390535861, + 1.2159298290498555, + 1.311590260011144, + 1.4065753379836679, + 1.5006697740172967, + 1.6033957760082558, + 1.7022238280624151, + 1.7952303329948336, + 1.8868839390343055, + 1.9796217479743063, + 2.0763380479766056, + 2.1733062750427052, + 2.278344713966362, + 2.379210975021124, + 2.475394080975093, + 2.574234441970475, + 2.6713871320243925, + 2.7669875690480694, + 2.862935162964277, + 2.9564628390362486, + 3.0494988340651616, + 3.1488794810138643, + 3.254395283991471, + 3.355978484963998, + 3.4538636510260403, + 3.54990662203636, + 3.6458663480589166, + 3.743174300994724, + 3.8411400719778612, + 3.9358743510674685, + 4.031745071988553, + 4.142435121000744, + 4.239682725048624, + 4.337339358055033, + 4.436768829007633, + 4.536681322962977, + 4.638434204040095, + 4.737738845986314, + 4.837442210991867, + 4.933307960978709, + 5.040671678027138, + 5.143711891025305, + 5.241291281068698, + 5.337871139054187, + 5.43416863901075, + 5.530516457045451, + 5.626903809024952, + 5.725428770063445, + 5.819256560993381, + 5.92343350907322, + 6.026722838985734, + 6.129055401077494, + 6.229267554008402, + 6.327729274053127, + 6.4229746530763805, + 6.516695939004421, + 6.615983632043935, + 6.711956490995362, + 6.806466727051884, + 6.901824005995877, + 7.009103863034397, + 7.10957543598488, + 7.208125296980143, + 7.30883181001991, + 7.4108177369926125, + 7.506558464025147, + 7.6012938329949975, + 7.691352733061649, + 7.7717233169823885, + 7.837844408000819, + 7.893904778058641, + 7.944759930018336, + 7.9985806960612535, + 8.060124851064757, + 8.125698584015481, + 8.188573686988093, + 8.24133085703943, + 8.295948990038596, + 8.35512575507164, + 8.414281481062062, + 8.47687312506605, + 8.538231577025726, + 8.595645696972497, + 8.660423940047622, + 8.746000034036115, + 8.820903411018662, + 8.878441047039814 + ], + "chunk_gaps_ms": { + "count": 96, + "p50": 95.96629248699173, + "p95": 105.51580297760665, + "max": 189.3284060060978 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 76.8, + "per_stream_tps": 20.1, + "ttft_s_max": 0.7, + "timing_valid": true, + "end_to_end_tps": 76.51423462307243, + "makespan_s": 13.383130669011734, + "steady_tps": 80.3, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 76.8448717927778, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 13.383130669011734, + "end_to_end_tps": 76.51423462307243, + "timing_valid": true, + "steady_tps": 80.33491315476279, + "per_stream_tps": [ + 22.8, + 19.2, + 20.0, + 20.2 + ], + "ttft_s": [ + 0.7, + 0.11, + 0.29, + 0.48 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0b4d014c8872", + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b82191a9a6ec", + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a" + } + ], + "client_intervals": [ + { + "start": 752506.583983499, + "first": 752507.281474943, + "last": 752518.44299229, + "end": 752518.443157229, + "elapsed_s": 11.859173730015755, + "chunk_offsets_s": [ + 0.6974914440652356, + 0.8274039870593697, + 0.9561756240436807, + 1.0875565990572795, + 1.2217318650800735, + 1.3589954930357635, + 1.4847847550408915, + 1.6166734140133485, + 1.7524080260191113, + 1.8919735030503944, + 2.0336862839758396, + 2.1729138360824436, + 2.316046266001649, + 2.4547179540386423, + 2.595239341026172, + 2.7294561529997736, + 2.8645431289914995, + 2.9974136820528656, + 3.136088025989011, + 3.2808144739829004, + 3.4162201470462605, + 3.5581603250466287, + 3.6977107430575415, + 3.838431042036973, + 3.987148083979264, + 4.131493143038824, + 4.273108116001822, + 4.414907951024361, + 4.554603602038696, + 4.69493332807906, + 4.840486690052785, + 4.986801418010145, + 5.130329246050678, + 5.266053989063948, + 5.4035463239997625, + 5.548941776971333, + 5.689781012013555, + 5.8333092470420524, + 5.98169364198111, + 6.123756292974576, + 6.262517469003797, + 6.41012601996772, + 6.5567730070324615, + 6.7100587820168585, + 6.859795426018536, + 7.009580789017491, + 7.153927067061886, + 7.297100343974307, + 7.445107770035975, + 7.58800600701943, + 7.731232520076446, + 7.868793667992577, + 8.006432877038606, + 8.145782063016668, + 8.290648265974596, + 8.437209713971242, + 8.5832865240518, + 8.723184194066562, + 8.869755227002315, + 9.016809829045087, + 9.16474030597601, + 9.311006246018223, + 9.462766458047554, + 9.60757953999564, + 9.74936424207408, + 9.935518317040987, + 10.124821841018274, + 10.315510872052982, + 10.504367602057755, + 10.692025779979303, + 10.855713193072006, + 11.007685758988373, + 11.15209028404206, + 11.299780510016717, + 11.444628193974495, + 11.586693251039833, + 11.724423057050444, + 11.85900879104156 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 142.89823698345572, + "p95": 187.65817792154849, + "max": 190.68903103470802 + } + }, + { + "start": 752506.583800609, + "first": 752506.693109753, + "last": 752519.966604473, + "end": 752519.966768708, + "elapsed_s": 13.382968098972924, + "chunk_offsets_s": [ + 0.10930914396885782, + 0.16740084101911634, + 0.37081771704833955, + 0.5875429890584201, + 0.8276590830646455, + 0.9564327039988711, + 1.0878147180192173, + 1.221989776007831, + 1.3589537179796025, + 1.4851766619831324, + 1.6166565349558368, + 1.7524264099774882, + 1.8918971690582111, + 2.0342801839578897, + 2.1729951129527763, + 2.3161205999786034, + 2.4548053910257295, + 2.5953148109838367, + 2.729482649010606, + 2.8649007809581235, + 2.9974199059652165, + 3.1366186330560595, + 3.280759884044528, + 3.416222403058782, + 3.5584018690278754, + 3.697723623015918, + 3.8384333619615063, + 3.9872708480106667, + 4.131601506960578, + 4.273253327002749, + 4.415043018991128, + 4.554742270032875, + 4.695023917010985, + 4.8406260929768905, + 4.98717092606239, + 5.130717489053495, + 5.266180112026632, + 5.403951302054338, + 5.548930624965578, + 5.68992163205985, + 5.833533655968495, + 5.981831412995234, + 6.123893455951475, + 6.262485533952713, + 6.410146355978213, + 6.556831820984371, + 6.709987648064271, + 6.860016523045488, + 7.009807948954403, + 7.153985769022256, + 7.296453709015623, + 7.445484110037796, + 7.588273101951927, + 7.731246663024649, + 7.868713223026134, + 8.006576025043614, + 8.146012071054429, + 8.290791684994474, + 8.437440715031698, + 8.583411672036164, + 8.723403897951357, + 8.869708540965803, + 9.0168840669794, + 9.164385357988067, + 9.311021044966765, + 9.462459700065665, + 9.607384491013363, + 9.7489347680239, + 9.935366626014002, + 10.125026682042517, + 10.315549719030969, + 10.504358801990747, + 10.692300650989637, + 10.855456546065398, + 11.007917382987216, + 11.151939696981572, + 11.299703520955518, + 11.444919369067065, + 11.586943194968626, + 11.724705783999525, + 11.858736191992648, + 11.97649722895585, + 12.093250715057366, + 12.212471751961857, + 12.330654614954256, + 12.446342829032801, + 12.56664081197232, + 12.686124422005378, + 12.810550414025784, + 12.9333774340339, + 13.041370988008566, + 13.119737287983298, + 13.188072251039557, + 13.274985471041873, + 13.332421111059375, + 13.382803863962181 + ], + "chunk_gaps_ms": { + "count": 95, + "p50": 141.65182004217058, + "p95": 189.6600560285151, + "max": 240.11609400622547 + } + }, + { + "start": 752506.583884319, + "first": 752506.874604796, + "last": 752519.625120847, + "end": 752519.625297257, + "elapsed_s": 13.04141293792054, + "chunk_offsets_s": [ + 0.2907204768853262, + 0.37057909998111427, + 0.5878091229824349, + 0.8278918738942593, + 0.9566198809770867, + 1.0875514139188454, + 1.2220435489434749, + 1.3592880739597604, + 1.4851981479441747, + 1.616634230944328, + 1.752687903936021, + 1.8921266169054434, + 2.0340191039722413, + 2.173294862965122, + 2.3163429429987445, + 2.4548946189461276, + 2.5954857639735565, + 2.729622818995267, + 2.864459594944492, + 2.9972915089456365, + 3.136414766893722, + 3.281098543899134, + 3.415897004888393, + 3.557958090910688, + 3.697537083993666, + 3.838081196998246, + 3.987045398913324, + 4.131840362912044, + 4.272994486964308, + 4.414856319897808, + 4.554458757978864, + 4.695095033966936, + 4.840444769943133, + 4.98673505091574, + 5.130305533995852, + 5.265879529993981, + 5.404029106954113, + 5.548645977978595, + 5.689502188935876, + 5.8333011829527095, + 5.981634794967249, + 6.123710052925162, + 6.262104649911635, + 6.409849996911362, + 6.556594964000396, + 6.709644506918266, + 6.859661093913019, + 7.009606946958229, + 7.154391994932666, + 7.296533353975974, + 7.445358544937335, + 7.587908856919967, + 7.731495981919579, + 7.868424846907146, + 8.006100039929152, + 8.145768765942194, + 8.290586346993223, + 8.43766048294492, + 8.583165566902608, + 8.723155809915625, + 8.86933965596836, + 9.016522653982975, + 9.164491688949056, + 9.310875480994582, + 9.462726794998161, + 9.607536420924589, + 9.749139031977393, + 9.93545844592154, + 10.12488953396678, + 10.315397786907852, + 10.504228741978295, + 10.691969892941415, + 10.8556614179397, + 11.007618431933224, + 11.151917664916255, + 11.299668768886477, + 11.444761916995049, + 11.586991079966538, + 11.724804403958842, + 11.858691232977435, + 11.976470860885456, + 12.093131085974164, + 12.212425352889113, + 12.330617287894711, + 12.446401466964744, + 12.566521214903332, + 12.686102165956981, + 12.810509342933074, + 12.933331050910056, + 13.04123652796261 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 141.8924870667979, + "p95": 188.83095507044345, + "max": 240.0827509118244 + } + }, + { + "start": 752506.583638039, + "first": 752507.063296551, + "last": 752519.703305776, + "end": 752519.70338131, + "elapsed_s": 13.119743271032348, + "chunk_offsets_s": [ + 0.4796585120493546, + 0.5878950880141929, + 0.8279561600647867, + 0.9566766150528565, + 1.0882005610037595, + 1.2222328550415114, + 1.3592808271059766, + 1.4848892310401425, + 1.617192420992069, + 1.752690096036531, + 1.8922701000701636, + 2.034358357079327, + 2.1737390880007297, + 2.316443052026443, + 2.4552865740843117, + 2.5956733900820836, + 2.729742127005011, + 2.864823919022456, + 2.997703656088561, + 3.1363518410362303, + 3.2806393110658973, + 3.4161879550665617, + 3.5582489140797406, + 3.6978240670869127, + 3.8383658910170197, + 3.9873279330786318, + 4.131401657010429, + 4.273292525089346, + 4.415146918036044, + 4.554743693093769, + 4.6950433240272105, + 4.840482497005723, + 4.987020706059411, + 5.130374795990065, + 5.265953766996972, + 5.403808139031753, + 5.548848899081349, + 5.689906671992503, + 5.833746960037388, + 5.981837924104184, + 6.123913326999173, + 6.262309075100347, + 6.410139059065841, + 6.556891401996836, + 6.709852084051818, + 6.859711140044965, + 7.0096436340827495, + 7.154503783094697, + 7.296820656047203, + 7.445559435058385, + 7.587935528019443, + 7.731292420998216, + 7.868460126104765, + 8.006514074048027, + 8.145829500048421, + 8.291122330003418, + 8.437472649035044, + 8.583236429025419, + 8.723240159102716, + 8.869390711071901, + 9.01681073999498, + 9.164413436083123, + 9.310918392031454, + 9.46278106502723, + 9.607432712102309, + 9.74884984199889, + 9.935335267102346, + 10.124913294101134, + 10.31552930304315, + 10.50428712402936, + 10.691955510061234, + 10.855850308085792, + 11.008258571033366, + 11.151582412072457, + 11.299758542096242, + 11.444906776072457, + 11.587181394104846, + 11.725111978012137, + 11.859117427025922, + 11.976479389006272, + 12.09330738405697, + 12.212518118089065, + 12.330751252011396, + 12.446569717023522, + 12.566603613086045, + 12.686201587086543, + 12.810648570070043, + 12.933417368098162, + 13.041210032999516, + 13.119667737046257 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 141.89086807891726, + "p95": 187.66838603187352, + "max": 240.0610720505938 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 1.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 95.9, + "per_stream_tps": 13.1, + "ttft_s_max": 1.77, + "timing_valid": true, + "end_to_end_tps": 95.74105439383828, + "makespan_s": 21.39103243604768, + "steady_tps": 102.7, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 95.87195243691188, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 21.39103243604768, + "end_to_end_tps": 95.74105439383828, + "timing_valid": true, + "steady_tps": 102.71325986567695, + "per_stream_tps": [ + 14.6, + 12.4, + 13.2, + 13.3, + 14.3, + 13.1, + 12.6, + 12.1 + ], + "ttft_s": [ + 0.69, + 0.47, + 1.47, + 1.77, + 0.93, + 1.19, + 0.11, + 0.28 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c547e852b429", + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "eb0a1898a022", + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0b4d014c8872", + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "b82191a9a6ec", + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "3f4413b23169", + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "51fe79c071ab", + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e9b7a497dd30", + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "097c9fe5896b", + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05" + } + ], + "client_intervals": [ + { + "start": 752519.972554039, + "first": 752520.66308699, + "last": 752538.087619716, + "end": 752538.088199873, + "elapsed_s": 18.115645834011957, + "chunk_offsets_s": [ + 0.6905329510336742, + 0.8220813240623102, + 1.0829331090208143, + 1.3633370760362595, + 1.6609495379962027, + 1.9796109789749607, + 2.189260689076036, + 2.3946270659798756, + 2.6019808989949524, + 2.8117557250661775, + 3.0332933390745893, + 3.2534127490362152, + 3.4759318260475993, + 3.6983884719666094, + 3.9149470700649545, + 4.1324281740235165, + 4.343837701017037, + 4.559955329983495, + 4.781078269006684, + 4.999093135003932, + 5.216494427993894, + 5.4430734070483595, + 5.663686612038873, + 5.890575692988932, + 6.120278034009971, + 6.3465576730668545, + 6.573991290060803, + 6.791239525075071, + 7.014864437049255, + 7.238699523964897, + 7.465004306985065, + 7.688597050029784, + 7.914618417969905, + 8.138886027038097, + 8.36090518196579, + 8.585804259986617, + 8.813898163032718, + 9.040739502059296, + 9.258017332060263, + 9.4784818100743, + 9.707088746014051, + 9.933223858010024, + 10.15541209699586, + 10.380258172051981, + 10.606729506049305, + 10.82951375900302, + 11.057328621973284, + 11.28242314606905, + 11.511445334996097, + 11.738015586975962, + 11.969327992061153, + 12.19703554501757, + 12.424268489005044, + 12.652939178980887, + 12.876026640995406, + 13.098110829014331, + 13.324915316072293, + 13.550126702990383, + 13.777645342983305, + 14.006069028982893, + 14.23565927601885, + 14.466851552017033, + 14.694436712074094, + 14.921169163077138, + 15.14875234698411, + 15.381984421983361, + 15.612217138055712, + 15.844079834059812, + 16.0767553569749, + 16.30559457500931, + 16.536240727058612, + 16.768883327022195, + 17.00272601901088, + 17.22862862003967, + 17.45203822897747, + 17.673719509039074, + 17.897003825986758, + 18.115065676975064 + ], + "chunk_gaps_ms": { + "count": 77, + "p50": 226.0213679401204, + "p95": 260.85178495850414, + "max": 318.66144097875804 + } + }, + { + "start": 752519.970454044, + "first": 752520.440667764, + "last": 752540.995814577, + "end": 752540.995945149, + "elapsed_s": 21.025491104926914, + "chunk_offsets_s": [ + 0.4702137199928984, + 0.5844536658842117, + 0.8241388879250735, + 1.0854073629016057, + 1.3660582669544965, + 1.6631340259918943, + 1.981600976898335, + 2.1909286339068785, + 2.3967986589996144, + 2.6044703209772706, + 2.8143561999313533, + 3.0351436618948355, + 3.2553283498855308, + 3.4779910219367594, + 3.7005196498939767, + 3.9173479669261724, + 4.1343232959043235, + 4.345866145915352, + 4.562314164941199, + 4.782623033970594, + 5.00088201998733, + 5.218626133981161, + 5.445136235910468, + 5.665873563964851, + 5.892710854997858, + 6.122414539917372, + 6.348227473907173, + 6.576047605951317, + 6.793127015000209, + 7.016874593915418, + 7.240294606890529, + 7.466965025989339, + 7.690732002956793, + 7.916516946977936, + 8.140949286986142, + 8.363367756945081, + 8.588480684906244, + 8.816562891937792, + 9.043346968945116, + 9.259920308948494, + 9.481099884957075, + 9.709681700915098, + 9.935444248956628, + 10.157835361897014, + 10.382264233892784, + 10.609032533946447, + 10.832020924892277, + 11.05915456195362, + 11.284483301918954, + 11.513492835918441, + 11.740799639956094, + 11.971372596919537, + 12.199697553995065, + 12.426439505885355, + 12.65498763998039, + 12.87823557597585, + 13.100329299923033, + 13.327262758975849, + 13.55212601297535, + 13.779842966934666, + 14.008335482911207, + 14.237815396976657, + 14.468636228935793, + 14.696960580884479, + 14.923325459938496, + 15.15088896395173, + 15.384140847949311, + 15.614413129980676, + 15.846342111937702, + 16.07908953691367, + 16.307854260900058, + 16.538490668986924, + 16.770476202946156, + 17.004426970961504, + 17.230936560896225, + 17.454220557934605, + 17.675958651932888, + 17.899434274993837, + 18.117532523931004, + 18.3221774129197, + 18.53225740895141, + 18.72878431691788, + 18.909060191945173, + 19.08778108295519, + 19.274602243909612, + 19.45976261096075, + 19.651502895983867, + 19.837613055948168, + 20.019671828951687, + 20.20173814089503, + 20.380563356913626, + 20.543462839908898, + 20.69508858595509, + 20.82299423997756, + 20.931790742906742, + 21.025360532919876 + ], + "chunk_gaps_ms": { + "count": 95, + "p50": 223.74757891520858, + "p95": 239.68522204086185, + "max": 318.4669509064406 + } + }, + { + "start": 752519.971483658, + "first": 752521.44427297, + "last": 752540.793741955, + "end": 752540.79388163, + "elapsed_s": 20.82239797199145, + "chunk_offsets_s": [ + 1.4727893120143563, + 1.6614734570030123, + 1.9805162450065836, + 2.19029593502637, + 2.396236902102828, + 2.6025568960467353, + 2.813432021997869, + 3.0347258530091494, + 3.254581063054502, + 3.4773323560366407, + 3.699987058993429, + 3.9163926710607484, + 4.1335453850915655, + 4.345454464084469, + 4.561084413086064, + 4.781401202082634, + 4.99949675402604, + 5.216742372023873, + 5.443684843019582, + 5.664348751073703, + 5.891284086043015, + 6.121104071033187, + 6.346720275003463, + 6.574717810028233, + 6.791744932066649, + 7.015412002103403, + 7.238849629997276, + 7.465488627087325, + 7.6894546370022, + 7.91535924107302, + 8.13929918105714, + 8.361859775031917, + 8.587280340027064, + 8.815246487036347, + 9.042123921099119, + 9.258256620028988, + 9.480115229031071, + 9.708494316088036, + 9.934965641004965, + 10.15659897099249, + 10.381561937043443, + 10.60744135407731, + 10.83071196009405, + 11.05828331108205, + 11.283941752044484, + 11.512990165036172, + 11.73936778306961, + 11.970628189039417, + 12.198597926064394, + 12.425688075018115, + 12.65385247708764, + 12.877406548010185, + 13.099647675058804, + 13.326385860098526, + 13.551462307106704, + 13.779096720041707, + 14.007377802045085, + 14.237229097052477, + 14.46835651108995, + 14.69559731404297, + 14.922396035050042, + 15.150223627104424, + 15.3837867960101, + 15.613654308021069, + 15.845573786064051, + 16.078148560016416, + 16.30698181001935, + 16.537612443091348, + 16.76973096304573, + 17.003322047996335, + 17.230174122028984, + 17.45352806907613, + 17.675000988063402, + 17.898470275104046, + 18.117114827036858, + 18.321316178073175, + 18.531057159998454, + 18.72830636508297, + 18.90777293802239, + 19.086828330066055, + 19.2738345740363, + 19.458805667003617, + 19.650732153095305, + 19.83670923009049, + 20.019005899084732, + 20.201070116017945, + 20.379748312057927, + 20.5429727450246, + 20.69429754104931, + 20.822258297004737 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 223.5540709225461, + "p95": 232.11851995438337, + "max": 319.04278800357133 + } + }, + { + "start": 752519.973157427, + "first": 752521.739605266, + "last": 752540.902161701, + "end": 752540.902433069, + "elapsed_s": 20.92927564191632, + "chunk_offsets_s": [ + 1.7664478389779106, + 1.9783644279232249, + 2.1880402809474617, + 2.39345097693149, + 2.601677996921353, + 2.8110017819562927, + 3.0323370510013774, + 3.252095896983519, + 3.4749101309571415, + 3.697254652972333, + 3.914415454957634, + 4.131331521901302, + 4.344128218945116, + 4.558767160982825, + 4.7796869069570675, + 4.998588191927411, + 5.215804450912401, + 5.441930196946487, + 5.662610679981299, + 5.889559822971933, + 6.1192466439679265, + 6.345877726911567, + 6.573223954997957, + 6.790931966970675, + 7.013587118010037, + 7.238186212955043, + 7.463767402921803, + 7.6874865419231355, + 7.914363418938592, + 8.13789543590974, + 8.36056610592641, + 8.585146105964668, + 8.813764711958356, + 9.040544916992076, + 9.25669409497641, + 9.47836266295053, + 9.707194838905707, + 9.933339854003862, + 10.155901809921488, + 10.379988516913727, + 10.6059070850024, + 10.829465713002719, + 11.056663875933737, + 11.282220256980509, + 11.511356809991412, + 11.737948037916794, + 11.968927652924322, + 12.196836351999082, + 12.424100222997367, + 12.652549032005481, + 12.875699515920132, + 13.097903731977567, + 13.324678507982753, + 13.549862904008478, + 13.777379687991925, + 14.005766046931967, + 14.23547960293945, + 14.466323905973695, + 14.694307755911723, + 14.920922370976768, + 15.148674605996348, + 15.381971862982027, + 15.611945356009528, + 15.84401426895056, + 16.076675599906594, + 16.3054667559918, + 16.536132250912488, + 16.76766567095183, + 17.00180334597826, + 17.228557758964598, + 17.452023957972415, + 17.67343281500507, + 17.897112606908195, + 18.11497865593992, + 18.319981470005587, + 18.52966470993124, + 18.726512279943563, + 18.906224764999934, + 19.08519307198003, + 19.27229480096139, + 19.457263108924963, + 19.649013089947402, + 19.834987110923976, + 20.017056234995835, + 20.199291469994932, + 20.378355238004588, + 20.541462683002464, + 20.692800885997713, + 20.820499633904546, + 20.929004273959436 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 223.53201697114855, + "p95": 231.53342003934085, + "max": 234.13767502643168 + } + }, + { + "start": 752519.972133253, + "first": 752520.902504093, + "last": 752538.699715082, + "end": 752538.699813895, + "elapsed_s": 18.727680642041378, + "chunk_offsets_s": [ + 0.9303708400111645, + 1.0830233449814841, + 1.3644774230197072, + 1.6610900609521195, + 1.9795092380372807, + 2.189842061023228, + 2.3956663759890944, + 2.602339974953793, + 2.813069937983528, + 3.034285499015823, + 3.2532414910383523, + 3.4765159339876845, + 3.698593824985437, + 3.9151996369473636, + 4.132688213023357, + 4.344534414005466, + 4.559696682030335, + 4.781283525982872, + 4.99935426202137, + 5.216536971041933, + 5.443371061002836, + 5.66415510897059, + 5.890961040044203, + 6.1206599259749055, + 6.34672304301057, + 6.574448314961046, + 6.791695381980389, + 7.01522981596645, + 7.2387342259753495, + 7.465461074956693, + 7.688980396953411, + 7.915489461040124, + 8.138838268001564, + 8.361866815015674, + 8.58697004604619, + 8.815235350979492, + 9.041834970004857, + 9.257984627969563, + 9.479701483040117, + 9.70762450399343, + 9.934092484996654, + 10.156577451038174, + 10.380879367003217, + 10.606842958019115, + 10.830395346041769, + 11.057971608941443, + 11.28302077401895, + 11.512425014982, + 11.73900672304444, + 11.97005204798188, + 12.197908891947009, + 12.425092365941964, + 12.653414555010386, + 12.876802070997655, + 13.098961457028054, + 13.325768967974, + 13.55084890301805, + 13.77860422397498, + 14.006880537956022, + 14.236541182966903, + 14.467384238028899, + 14.695199390989728, + 14.92190641805064, + 15.149623726028949, + 15.383057283004746, + 15.61293282697443, + 15.845006027957425, + 16.077586465980858, + 16.30642081995029, + 16.5371213540202, + 16.76858285604976, + 17.002934140036814, + 17.229615596006624, + 17.45301862899214, + 17.674409630941227, + 17.898099742946215, + 18.11664236197248, + 18.320966684957966, + 18.530647877021693, + 18.72758182894904 + ], + "chunk_gaps_ms": { + "count": 79, + "p50": 225.10323103051633, + "p95": 234.35128398705274, + "max": 318.4191770851612 + } + }, + { + "start": 752519.970789025, + "first": 752521.163841707, + "last": 752540.665878316, + "end": 752540.666182834, + "elapsed_s": 20.69539380900096, + "chunk_offsets_s": [ + 1.193052682094276, + 1.3649960299953818, + 1.6619114900240675, + 1.9813195780152455, + 2.1911146440543234, + 2.3967741960659623, + 2.6038479420822114, + 2.813796442002058, + 3.0349115260178223, + 3.254638437065296, + 3.4773339710664004, + 3.69969815609511, + 3.9163285120157525, + 4.133806160069071, + 4.34638729703147, + 4.561426466098055, + 4.7821874800138175, + 5.000278456020169, + 5.217788801994175, + 5.444539566989988, + 5.665124997030944, + 5.892122250050306, + 6.121731746010482, + 6.347786800004542, + 6.575367307988927, + 6.793113511055708, + 7.016169673064724, + 7.2401441800175235, + 7.4663386470638216, + 7.6900427780346945, + 7.916280619101599, + 8.140307740075514, + 8.36287115002051, + 8.58781917009037, + 8.816402593045495, + 9.042301131063141, + 9.259939093026333, + 9.481260984088294, + 9.709597624023445, + 9.935368363047019, + 10.157336801989004, + 10.3822915290948, + 10.607974569080397, + 10.831794932018965, + 11.05926377500873, + 11.284420184092596, + 11.513441494083963, + 11.740385813987814, + 11.971261191996746, + 12.1991387240123, + 12.426349957007915, + 12.654964601038955, + 12.878318901988678, + 13.100205560098402, + 13.327000240096822, + 13.552098734071478, + 13.779683228000067, + 14.008103730040602, + 14.237716617062688, + 14.468962635030039, + 14.696333625004627, + 14.923125353991054, + 15.151005281019025, + 15.3841979179997, + 15.614381868042983, + 15.846307585015893, + 16.07899339601863, + 16.30759465310257, + 16.538406144012697, + 16.769756386056542, + 17.004316735081375, + 17.230829077074304, + 17.454264861065894, + 17.67585257603787, + 17.899229145026766, + 18.117749717086554, + 18.322276002028957, + 18.53182887902949, + 18.7288013570942, + 18.908395814010873, + 19.087646014988422, + 19.274625188088976, + 19.459554202039726, + 19.651494864025153, + 19.83753971499391, + 20.019664933090098, + 20.2017335339915, + 20.380587053019553, + 20.543774479068816, + 20.695089291082695 + ], + "chunk_gaps_ms": { + "count": 89, + "p50": 223.820362938568, + "p95": 232.68581100273877, + "max": 319.40808799117804 + } + }, + { + "start": 752519.970981834, + "first": 752520.08285957, + "last": 752540.351421004, + "end": 752540.351527945, + "elapsed_s": 20.380546111031435, + "chunk_offsets_s": [ + 0.11187773605342954, + 0.1715094989631325, + 0.36117352405562997, + 0.5836574210552499, + 0.8238109960220754, + 1.0848037040559575, + 1.3659757899586111, + 1.6626421390101314, + 1.980976341990754, + 2.1904943929985166, + 2.3960523799760267, + 2.604148908983916, + 2.813259611953981, + 3.034909783047624, + 3.2546651570592076, + 3.4774184490088373, + 3.699638606980443, + 3.9169636759907007, + 4.133707475964911, + 4.3460113579640165, + 4.561428977991454, + 4.782497151056305, + 5.0003870610380545, + 5.21788315102458, + 5.444461042061448, + 5.665145460981876, + 5.892052412964404, + 6.121753843035549, + 6.347480827011168, + 6.575474409968592, + 6.792542555020191, + 7.0162560869939625, + 7.239720226032659, + 7.4663330799667165, + 7.690067082061432, + 7.916146288043819, + 8.140262349974364, + 8.362802064977586, + 8.587732502026483, + 8.815589644014835, + 9.04265758395195, + 9.258921590051614, + 9.48017079604324, + 9.708950062049553, + 9.935435033985414, + 10.157459471025504, + 10.381978995981626, + 10.6077227370115, + 10.831310164998285, + 11.059031415032223, + 11.284338523983024, + 11.513175806961954, + 11.739781258045696, + 11.971173116005957, + 12.19898041000124, + 12.426105438033119, + 12.654621173045598, + 12.877817878965288, + 13.099956016987562, + 13.326629644958302, + 13.551859591039829, + 13.779790473985486, + 14.007980886963196, + 14.237608253024518, + 14.46832721598912, + 14.69620974198915, + 14.923144347034395, + 15.150595535058528, + 15.38432754704263, + 15.613972121966071, + 15.845925373956561, + 16.07843402295839, + 16.307527423952706, + 16.538349683047272, + 16.76964024605695, + 17.004046551999636, + 17.230806773994118, + 17.453880634042434, + 17.675363120040856, + 17.89879738399759, + 18.118040860979818, + 18.322193350060843, + 18.53187460603658, + 18.728463671985082, + 18.908347896067426, + 19.087415926973335, + 19.274226753041148, + 19.4593992800219, + 19.65088429301977, + 19.837306619039737, + 20.019434093032032, + 20.20111934596207, + 20.38043917005416 + ], + "chunk_gaps_ms": { + "count": 92, + "p50": 223.92503201263025, + "p95": 240.15357496682554, + "max": 318.33420298062265 + } + }, + { + "start": 752519.970355007, + "first": 752520.249958685, + "last": 752541.361241288, + "end": 752541.361387443, + "elapsed_s": 21.39103243604768, + "chunk_offsets_s": [ + 0.2796036780346185, + 0.36160938104148954, + 0.584089949959889, + 0.8239364949986339, + 1.085000753053464, + 1.3654662710614502, + 1.6624882080359384, + 1.981250701006502, + 2.1908934670500457, + 2.3964011360658333, + 2.6037965840660036, + 2.8146277910564095, + 3.035422966000624, + 3.254942107014358, + 3.4778288829838857, + 3.700686556054279, + 3.9168683669995517, + 4.134178964071907, + 4.346726853982545, + 4.561359380022623, + 4.782582188025117, + 5.001077822060324, + 5.217975836014375, + 5.444865092984401, + 5.665517288027331, + 5.892485455027781, + 6.122111686039716, + 6.348262545070611, + 6.575747551978566, + 6.792956221033819, + 7.017223958973773, + 7.240058663068339, + 7.466710539069027, + 7.69040102197323, + 7.917002739966847, + 8.141005206038244, + 8.363539495971054, + 8.58820043806918, + 8.816923632053658, + 9.043391959974542, + 9.260083536035381, + 9.481301766005345, + 9.70982820901554, + 9.935580963967368, + 10.15781283506658, + 10.382424165029079, + 10.608241567970254, + 10.832347537972964, + 11.05958640400786, + 11.284907641005702, + 11.513718029018492, + 11.740686923963949, + 11.97161775699351, + 12.19932734197937, + 12.426584461005405, + 12.65530334203504, + 12.878377332002856, + 13.100465344032273, + 13.327318917959929, + 13.552282872027718, + 13.77990011696238, + 14.008360745967366, + 14.237966401036829, + 14.468603285960853, + 14.696911655017175, + 14.923616874963045, + 15.151033952017315, + 15.384295530966483, + 15.61447108106222, + 15.846371102961712, + 16.07913795101922, + 16.307867077062838, + 16.538521501002833, + 16.770779281039722, + 17.004370572976768, + 17.231082875980064, + 17.45437904901337, + 17.67585511901416, + 17.89982107805554, + 18.118773589027114, + 18.32299519504886, + 18.532097110059112, + 18.729160881019197, + 18.90910430299118, + 19.087755708023906, + 19.274742383975536, + 19.459813377005048, + 19.652123211999424, + 19.837601248989813, + 20.020350111066364, + 20.202350298059173, + 20.381431346060708, + 20.543707648990676, + 20.696165160043165, + 20.823498736019246, + 20.931991040008143, + 21.025217002024874, + 21.095840402995236, + 21.156285392004065, + 21.21396010206081, + 21.27249310305342, + 21.337565609021112, + 21.39088628103491 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 223.18508150056005, + "p95": 233.59129193704575, + "max": 318.76249297056347 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c547e852b4295d0e344e933e7a089f31c19287d1dce64dd2c80ddc185e04c7a3", + "output_sha256": "89194a4cddc5991e9bccb7ae21f485c8e29f59a335ac63740c785bfd44ffd002", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "eb0a1898a022407ea58dca61324b5ac4a66182d9603835554bdd011fdaf1ad50", + "output_sha256": "84fe14a7af678f8074a8349c9ebfd8d91ead82534641e60e6c6a572d11794df2", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0b4d014c8872a303dffd183f0ee01f943e634d9ad2ab4e15a297106f4294f781", + "output_sha256": "d0b5a0e66f9ffd9114f9b14b99584d37ca26d14acf28b41ef4da7290e25a8c06", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "b82191a9a6ec15d25df0cea0037bff1f895795f0f2ecf188803480107898d04f", + "output_sha256": "7c789034ddb8ff8fcc572f77dfe3ea38fbe293005887162cddc6e8634813b43a", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "3f4413b231696b99b2605537f5be946f6ba8a6a56ea27a013ade2884eebc2124", + "output_sha256": "64b9ec22c25ffb7050c0aa520f11ae8e5d78a41508a748aaf6b9951e3e778c51", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "51fe79c071abdc95688f5f3c6a2cadeee2ca340ef43c60368243515a09f02b62", + "output_sha256": "7a7abbd67ed399be40fe877910baea21271909f9c2ecb7df65df8933720e6bf0", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e9b7a497dd300055458dd59766db06dc61f41691adcb4b2ef033e54d02abd8c1", + "output_sha256": "0e4e3d6d8f52294224f21c0b785ae196f7bbc89f26e544034a521a050bb19774", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "097c9fe5896bb08143318fcfdb85937bf304aea79663c6a818560a738df4892d", + "output_sha256": "89f273a171134389f92876bcc72c8c05a4c915df65bc0aebd1a081d700fcdd05", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 35.0, + "per_stream_tps": 35.0, + "ttft_s_max": 0.16, + "timing_valid": true, + "end_to_end_tps": 34.410944851454865, + "makespan_s": 7.439493483980186, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 35.04034472752361, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 7.439493483980186, + "end_to_end_tps": 34.410944851454865, + "timing_valid": true, + "per_stream_tps": [ + 35.0 + ], + "ttft_s": [ + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + } + ], + "client_intervals": [ + { + "start": 752692.906622879, + "first": 752693.068584911, + "last": 752700.345910576, + "end": 752700.346116363, + "elapsed_s": 7.439493483980186, + "chunk_offsets_s": [ + 0.16196203196886927, + 0.22405035607516766, + 0.28187303501181304, + 0.3330370810581371, + 0.3873720630072057, + 0.4459070600569248, + 0.49749483703635633, + 0.5495283630443737, + 0.6011474190745503, + 0.6534367779968306, + 0.7037665579700842, + 0.7553083050297573, + 0.8063610889948905, + 0.8577247289940715, + 0.9091929100686684, + 0.9594972180202603, + 1.0103439759695902, + 1.0618709879927337, + 1.1127441310090944, + 1.1639156810706481, + 1.2145670210011303, + 1.2645924420794472, + 1.3170493330108002, + 1.3704985650256276, + 1.429696153034456, + 1.4848059479845688, + 1.5368324830196798, + 1.5911503310780972, + 1.6492341250414029, + 1.7062034850241616, + 1.7652289969846606, + 1.8235915830591694, + 1.8840681919828057, + 1.9501536879688501, + 2.002705376013182, + 2.060650247032754, + 2.1195246600545943, + 2.1784556009806693, + 2.236696782987565, + 2.2893380069872364, + 2.3412045150762424, + 2.393574304995127, + 2.4451884370064363, + 2.4952387859812006, + 2.5476828310638666, + 2.5983965230407193, + 2.6533822739729658, + 2.7134524630382657, + 2.7715322739677504, + 2.8269015189725906, + 2.878422741079703, + 2.931932740029879, + 2.991027147974819, + 3.0496236180188134, + 3.102675231057219, + 3.1552230180241168, + 3.2078393950359896, + 3.2658523289719597, + 3.3233660609694198, + 3.375573729048483, + 3.427893970045261, + 3.478610575082712, + 3.529194383067079, + 3.5813208060571924, + 3.6325128690805286, + 3.6850423050345853, + 3.736123795970343, + 3.786980652017519, + 3.8377805670024827, + 3.8895075760083273, + 3.9408368360018358, + 3.9941398029914126, + 4.045974906068295, + 4.0967872850596905, + 4.149153653997928, + 4.201638084021397, + 4.259532046038657, + 4.318179556983523, + 4.378821013029665, + 4.43974258902017, + 4.501155592035502, + 4.553802450071089, + 4.605578131042421, + 4.657530847005546, + 4.708954264991917, + 4.760012558079325, + 4.8111391050042585, + 4.861458041006699, + 4.9132351140724495, + 4.9662068750476465, + 5.018267331994139, + 5.070299247046933, + 5.122333978069946, + 5.173097464023158, + 5.224878970067948, + 5.275092437979765, + 5.325459967018105, + 5.376727150985971, + 5.430717797018588, + 5.49068204103969, + 5.551471829996444, + 5.613815800985321, + 5.676668782019988, + 5.741990593029186, + 5.80526378704235, + 5.858533299993724, + 5.919165797997266, + 5.977377081057057, + 6.036794892046601, + 6.086583302007057, + 6.138574691023678, + 6.1906620300142094, + 6.241784627083689, + 6.295395827968605, + 6.3480533270630985, + 6.400623981026001, + 6.452168359071948, + 6.505403490038589, + 6.563297759043053, + 6.614726908039302, + 6.666966067044996, + 6.71887369803153, + 6.770889295032248, + 6.82253186206799, + 6.874226747080684, + 6.926417459035292, + 6.978118313010782, + 7.028495489968918, + 7.0813732569804415, + 7.132693769061007, + 7.184521196060814, + 7.23821255704388, + 7.289732472039759, + 7.341707527055405, + 7.392322506988421, + 7.439287697081454 + ], + "chunk_gaps_ms": { + "count": 135, + "p50": 52.207668079063296, + "p95": 61.413003015331924, + "max": 66.0854959860444 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 53.0, + "per_stream_tps": 27.6, + "ttft_s_max": 0.38, + "timing_valid": true, + "end_to_end_tps": 52.30382685011293, + "makespan_s": 9.788958683027886, + "steady_tps": 55.1, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 52.97415401649217, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 9.788958683027886, + "end_to_end_tps": 52.30382685011293, + "timing_valid": true, + "steady_tps": 55.098363301900505, + "per_stream_tps": [ + 27.1, + 28.2 + ], + "ttft_s": [ + 0.38, + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + } + ], + "client_intervals": [ + { + "start": 752700.348162291, + "first": 752700.727392666, + "last": 752710.136570046, + "end": 752710.136710906, + "elapsed_s": 9.78854861506261, + "chunk_offsets_s": [ + 0.37923037505242974, + 0.46360556001309305, + 0.5593579560518265, + 0.65379072597716, + 0.7478281779913232, + 0.8390698509756476, + 0.9312056270428002, + 1.0258909979602322, + 1.120149084017612, + 1.2132761620450765, + 1.3087451440514997, + 1.4012704290216789, + 1.4930180079536512, + 1.5870071420213208, + 1.6840494909556583, + 1.7807517220498994, + 1.88289694103878, + 1.979251452954486, + 2.076285385992378, + 2.1720292429672554, + 2.2708953509572893, + 2.3663298239698634, + 2.4639603489777073, + 2.558293523034081, + 2.652723144972697, + 2.7508202680619434, + 2.845145093044266, + 2.939349457039498, + 3.033640619018115, + 3.1282398520270362, + 3.222100081969984, + 3.3204941720468923, + 3.416735889040865, + 3.5144011040683836, + 3.609604801982641, + 3.7065032069804147, + 3.8009664620039985, + 3.897675432031974, + 3.994040556019172, + 4.091423331992701, + 4.189161011017859, + 4.284299512044527, + 4.381001090048812, + 4.478120849002153, + 4.575444426969625, + 4.671511944965459, + 4.767687348066829, + 4.862916984013282, + 4.9567823690595105, + 5.050940706976689, + 5.143515150994062, + 5.234188477974385, + 5.32536155206617, + 5.419625118025579, + 5.514118870953098, + 5.608945894986391, + 5.706145863980055, + 5.8005225009983405, + 5.893506968044676, + 5.987596477032639, + 6.084647777955979, + 6.179190410999581, + 6.274238437996246, + 6.3645176980644464, + 6.456745097995736, + 6.550755584961735, + 6.646759891998954, + 6.743496099952608, + 6.840632328996435, + 6.937607362051494, + 7.030549815972336, + 7.124810153967701, + 7.221652599051595, + 7.318313769064844, + 7.416388291982003, + 7.513493609963916, + 7.607925817021169, + 7.701935891993344, + 7.797921738005243, + 7.894587693037465, + 7.987715792027302, + 8.082115696975961, + 8.173956230049953, + 8.269040356972255, + 8.363381286966614, + 8.461306857061572, + 8.558655914035626, + 8.65348548896145, + 8.74353690398857, + 8.838303041062318, + 8.932537768036127, + 9.028529552044347, + 9.122397009050474, + 9.214605789980851, + 9.276424254057929, + 9.3279733939562, + 9.378723131958395, + 9.430988125037402, + 9.483098561991937, + 9.534543529036455, + 9.586730445036665, + 9.638389438972808, + 9.690092848031782, + 9.740933640045114, + 9.78840775496792 + ], + "chunk_gaps_ms": { + "count": 104, + "p50": 94.47850397555158, + "p95": 97.92557009495795, + "max": 102.14521898888052 + } + }, + { + "start": 752700.347752223, + "first": 752700.509233669, + "last": 752709.562399323, + "end": 752709.562505528, + "elapsed_s": 9.214753304957412, + "chunk_offsets_s": [ + 0.16148144600447267, + 0.2229381159413606, + 0.463704644003883, + 0.5593296999577433, + 0.6539088659919798, + 0.747724068001844, + 0.8391549990046769, + 0.9312055140035227, + 1.0260181300109252, + 1.1207685549743474, + 1.213791970978491, + 1.3092828879598528, + 1.4018950830213726, + 1.493642597924918, + 1.5876279879594222, + 1.6842099339701235, + 1.7810864800121635, + 1.8828220299910754, + 1.9794799899682403, + 2.0761699969880283, + 2.1722359879640862, + 2.271032753982581, + 2.3665392729453743, + 2.4640914800111204, + 2.5582742110127583, + 2.6528645009966567, + 2.7509229680290446, + 2.8453148789703846, + 2.9395601379219443, + 3.033852468011901, + 3.1281172949820757, + 3.2219354609260336, + 3.320404861937277, + 3.4166662419447675, + 3.514592745923437, + 3.6098296910058707, + 3.7064990940270945, + 3.80113135301508, + 3.8983401809819043, + 3.994178951019421, + 4.091570638935082, + 4.189077987917699, + 4.284242007997818, + 4.381150156958029, + 4.4782218059990555, + 4.575601317919791, + 4.671611748985015, + 4.767790463985875, + 4.8630678749177605, + 4.956964378943667, + 5.050936113926582, + 5.143755079014227, + 5.234149997937493, + 5.32531967991963, + 5.419620333937928, + 5.514291584957391, + 5.6090011559426785, + 5.706130774924532, + 5.800689648021944, + 5.893679633969441, + 5.9881319819251075, + 6.084626369993202, + 6.179772890987806, + 6.274392321007326, + 6.364616270991974, + 6.456684506032616, + 6.550816030008718, + 6.646726915962063, + 6.743672429933213, + 6.840809011016972, + 6.9378108909586444, + 7.030721633927897, + 7.124871207983233, + 7.221675332984887, + 7.31840210093651, + 7.416413314989768, + 7.5135639909422025, + 7.6080596359679475, + 7.702005120925605, + 7.797947207931429, + 7.8948023749981076, + 7.988319327007048, + 8.082600802998058, + 8.174473415943794, + 8.269271389930509, + 8.363504274981096, + 8.461459395010024, + 8.558802852989174, + 8.653459487948567, + 8.743472249014303, + 8.83855861693155, + 8.93273161794059, + 9.02874634496402, + 9.12230956996791, + 9.214647099957801 + ], + "chunk_gaps_ms": { + "count": 94, + "p50": 94.80529499705881, + "p95": 98.05846703238785, + "max": 240.7665280625224 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 68.8, + "per_stream_tps": 17.8, + "ttft_s_max": 0.87, + "timing_valid": true, + "end_to_end_tps": 68.33455152681384, + "makespan_s": 14.985098710982129, + "steady_tps": 72.4, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 68.80622423555167, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 14.985098710982129, + "end_to_end_tps": 68.33455152681384, + "timing_valid": true, + "steady_tps": 72.4223733676012, + "per_stream_tps": [ + 17.8, + 19.3, + 17.2, + 17.8 + ], + "ttft_s": [ + 0.61, + 0.87, + 0.16, + 0.38 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0a24c104609f", + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "28cedcb814fc", + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767" + } + ], + "client_intervals": [ + { + "start": 752710.138988333, + "first": 752710.750093046, + "last": 752725.072160767, + "end": 752725.072270746, + "elapsed_s": 14.933282412937842, + "chunk_offsets_s": [ + 0.6111047129379585, + 0.7123126939404756, + 0.9940923819085583, + 1.1229349349159747, + 1.25433730694931, + 1.3923906359123066, + 1.5247519098920748, + 1.656574381981045, + 1.793366752914153, + 1.9296987879788503, + 2.0689617509488016, + 2.207049977965653, + 2.3423188079614192, + 2.4802947979187593, + 2.616979766986333, + 2.7582240849733353, + 2.899930600891821, + 3.0422653089044616, + 3.1803082599071786, + 3.3135322199668735, + 3.4454293199814856, + 3.5816012169234455, + 3.719837106997147, + 3.8594808009220287, + 3.998090762994252, + 4.1350335699971765, + 4.274232812924311, + 4.413704095990397, + 4.555409895954654, + 4.693595293909311, + 4.827683251001872, + 4.970608891919255, + 5.111638744943775, + 5.2518756738863885, + 5.396681601996534, + 5.53911218396388, + 5.678922773920931, + 5.820835163001902, + 5.989345930982381, + 6.172469618963078, + 6.364947967929766, + 6.54782091092784, + 6.739408131921664, + 6.918887040927075, + 7.0652344999834895, + 7.211857040994801, + 7.356897288002074, + 7.500162253971212, + 7.64563184895087, + 7.786146456957795, + 7.923788376967423, + 8.064306218991987, + 8.204270636895671, + 8.347521861898713, + 8.490868012886494, + 8.63018315192312, + 8.776693937950768, + 8.920730233890936, + 9.060482146916911, + 9.199000061955303, + 9.337334445910528, + 9.479239357984625, + 9.615841099992394, + 9.759825172950514, + 9.895604938967153, + 10.031912003993057, + 10.174294889904559, + 10.318844374967739, + 10.45471260289196, + 10.596235592965968, + 10.735514721949585, + 10.875859984895214, + 11.01221956196241, + 11.15527360199485, + 11.300228901905939, + 11.440454381983727, + 11.579324398888275, + 11.716109495959245, + 11.858592265984043, + 12.000812301994301, + 12.138593464973383, + 12.273587577976286, + 12.408111844910309, + 12.54771852097474, + 12.688698325888254, + 12.832419293001294, + 12.977305410895497, + 13.109090547892265, + 13.24648954893928, + 13.381692994968034, + 13.521923454944044, + 13.666833146940917, + 13.808303888887167, + 13.946715780999511, + 14.081859899917617, + 14.203811026993208, + 14.325134956976399, + 14.4488711518934, + 14.567619225941598, + 14.677593437954783, + 14.773104661959223, + 14.861057863919996, + 14.933172433986329 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 139.39321105135605, + "p95": 179.47890900541097, + "max": 281.7796879680827 + } + }, + { + "start": 752710.139409282, + "first": 752711.008412786, + "last": 752724.220754338, + "end": 752724.220975625, + "elapsed_s": 14.081566342967562, + "chunk_offsets_s": [ + 0.8690035039326176, + 0.9940697109559551, + 1.1230041009839624, + 1.2546313459752128, + 1.3923243649769574, + 1.5244091509375721, + 1.6566902350168675, + 1.7934687819797546, + 1.9297901450190693, + 2.0691049470333382, + 2.20685825496912, + 2.341567212017253, + 2.480265022953972, + 2.6170859079575166, + 2.758309970027767, + 2.8998390829656273, + 3.042487654951401, + 3.180271124932915, + 3.313515739981085, + 3.445551572018303, + 3.581442820955999, + 3.719680950976908, + 3.8596801549429074, + 3.9980487319407985, + 4.135091056930833, + 4.274189101997763, + 4.413094721036032, + 4.555132398963906, + 4.693305621040054, + 4.8275474139954895, + 4.970440287957899, + 5.111710053985007, + 5.251848826999776, + 5.396387128974311, + 5.538985659950413, + 5.6787860409822315, + 5.820543842972256, + 5.9893604420358315, + 6.172331750043668, + 6.36499663896393, + 6.54747527197469, + 6.739205401041545, + 6.918965182034299, + 7.0648959330283105, + 7.211166883003898, + 7.356897862977348, + 7.500256250961684, + 7.645560426986776, + 7.785998716950417, + 7.923959539970383, + 8.064271723967977, + 8.20443175197579, + 8.347268637036905, + 8.491037735948339, + 8.629559535998851, + 8.776589956949465, + 8.920788679970428, + 9.060356374946423, + 9.198864481993951, + 9.337149858940393, + 9.479373467038386, + 9.615645184996538, + 9.759776614024304, + 9.895479967002757, + 10.03192894696258, + 10.174276347039267, + 10.318766313022934, + 10.454744345974177, + 10.595861169975251, + 10.734993760008365, + 10.875478410045616, + 11.011859172023833, + 11.154778286931105, + 11.299643493955955, + 11.440071768942289, + 11.578847720986232, + 11.715653458959423, + 11.858131675980985, + 12.000443477998488, + 12.138097792980261, + 12.2732011169428, + 12.40776280197315, + 12.54723795701284, + 12.688058889936656, + 12.831962942960672, + 12.976819021976553, + 13.108629732974805, + 13.245931509998627, + 13.381143181002699, + 13.521465025027283, + 13.666668149991892, + 13.807946795015596, + 13.94622625503689, + 14.081345055950806 + ], + "chunk_gaps_ms": { + "count": 93, + "p50": 139.8003810318187, + "p95": 179.7597809927538, + "max": 192.66488892026246 + } + }, + { + "start": 752710.138554761, + "first": 752710.298750883, + "last": 752725.122991151, + "end": 752725.123467733, + "elapsed_s": 14.984912972082384, + "chunk_offsets_s": [ + 0.16019612201489508, + 0.21901126008015126, + 0.45227602508384734, + 0.7126992270350456, + 0.9945670410525054, + 1.1233260600129142, + 1.2548116620164365, + 1.3927792970789596, + 1.52549681707751, + 1.6570557610830292, + 1.7937580870930105, + 1.930174118024297, + 2.0693520600907505, + 2.207539756083861, + 2.3426659350516275, + 2.480976428021677, + 2.6176566120702773, + 2.7588746750261635, + 2.900175666087307, + 3.042925034998916, + 3.1806749700335786, + 3.314094861038029, + 3.4459132270421833, + 3.5820818920619786, + 3.7202219440368935, + 3.8599811390740797, + 3.9986379160545766, + 4.135518709081225, + 4.274489062023349, + 4.414191299001686, + 4.55563569709193, + 4.69384768710006, + 4.827916044043377, + 4.970781319076195, + 5.11184734699782, + 5.252100756042637, + 5.396916539059021, + 5.5396038830513135, + 5.6790898410836235, + 5.821058613015339, + 5.989651762065478, + 6.172865319997072, + 6.36532869306393, + 6.548189797089435, + 6.739766906015575, + 6.919378435006365, + 7.065794869093224, + 7.212428385042585, + 7.357555142021738, + 7.500808876007795, + 7.646282455069013, + 7.786533134058118, + 7.924452887033112, + 8.065008680103347, + 8.204924363060854, + 8.348022968042642, + 8.491434110095724, + 8.630677714012563, + 8.777079576044343, + 8.921232460066676, + 9.061050196061842, + 9.19949440099299, + 9.337725971010514, + 9.47973985702265, + 9.616230754065327, + 9.760330855031498, + 9.895994369057007, + 10.032393127097748, + 10.174836107995361, + 10.319325593998656, + 10.455290427082218, + 10.596625710022636, + 10.736122593050823, + 10.876212263014168, + 11.012604656047188, + 11.155579881044105, + 11.300897172070108, + 11.440813858062029, + 11.57996001909487, + 11.71643169107847, + 11.859230674104765, + 12.00119799003005, + 12.138906511012465, + 12.273982484010048, + 12.408500846009701, + 12.548038721084595, + 12.689093520049937, + 12.832646189024672, + 12.977798006031662, + 13.109598473994993, + 13.246972974040546, + 13.382065479061566, + 13.52210838906467, + 13.66735453309957, + 13.808900237083435, + 13.947005829075351, + 14.082407164038159, + 14.20418501610402, + 14.32551057904493, + 14.449244693038054, + 14.567985917092301, + 14.677893137093633, + 14.773723432095721, + 14.861396916094236, + 14.933515375014395, + 14.984436389990151 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 139.49688302818686, + "p95": 183.21355793159455, + "max": 281.86781401745975 + } + }, + { + "start": 752710.138369022, + "first": 752710.515867971, + "last": 752724.816710971, + "end": 752724.816816806, + "elapsed_s": 14.67844778392464, + "chunk_offsets_s": [ + 0.3774989489465952, + 0.452690141974017, + 0.7131460789823905, + 0.9953212769469246, + 1.123841501888819, + 1.2553744938923046, + 1.3933055870002136, + 1.5256082699634135, + 1.6575659388909116, + 1.794404100975953, + 1.9307277359766886, + 2.0700320579344407, + 2.208097052993253, + 2.343046356923878, + 2.481236884952523, + 2.618371232994832, + 2.759530017967336, + 2.9007165959337726, + 3.04325860994868, + 3.1815443059895188, + 3.3147650508908555, + 3.4468318260041997, + 3.5826283409260213, + 3.7209130940027535, + 3.8605265798978508, + 3.999291049898602, + 4.136361293960363, + 4.275421132915653, + 4.414191762916744, + 4.556369069963694, + 4.694417190970853, + 4.828407166991383, + 4.971544338972308, + 5.112478561932221, + 5.252695811912417, + 5.397482987958938, + 5.54008281393908, + 5.679873019922525, + 5.821675523999147, + 5.990325903985649, + 6.173420120961964, + 6.366216461989097, + 6.548569609993137, + 6.7404217849252746, + 6.920120782917365, + 7.065751061891206, + 7.212115161935799, + 7.358136677998118, + 7.501537432894111, + 7.646808504941873, + 7.786994881927967, + 7.925263904966414, + 8.065515578957275, + 8.205729316920042, + 8.348508474999107, + 8.492022621911019, + 8.630724176997319, + 8.777857602923177, + 8.922037333948538, + 9.061592757003382, + 9.200085120974109, + 9.338409184943885, + 9.4801880849991, + 9.616872111917473, + 9.760746219893917, + 9.89672585297376, + 10.033215904957615, + 10.175276031950489, + 10.31976580596529, + 10.455974744982086, + 10.597134911920875, + 10.736509654903784, + 10.876711864955723, + 11.013130097999237, + 11.156146505963989, + 11.301450356957503, + 11.441451534978114, + 11.580344478948973, + 11.717138871899806, + 11.859536426956765, + 12.00175604596734, + 12.139382866909727, + 12.274538283934817, + 12.408960092929192, + 12.548730010981672, + 12.689201623899862, + 12.833245102898218, + 12.978128356975503, + 13.110133787966333, + 13.247490154928528, + 13.382679836940952, + 13.522700452944264, + 13.668231996940449, + 13.809415641939268, + 13.947706351988018, + 14.08295345690567, + 14.204724586918019, + 14.326055782963522, + 14.449880447937176, + 14.5685211509699, + 14.678341948892921 + ], + "chunk_gaps_ms": { + "count": 100, + "p50": 139.5844544749707, + "p95": 182.35314800404012, + "max": 282.17519796453416 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 1.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 85.3, + "per_stream_tps": 11.4, + "ttft_s_max": 2.13, + "timing_valid": true, + "end_to_end_tps": 85.09953798990507, + "makespan_s": 24.06593558995519, + "steady_tps": 92.1, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 85.34456961804969, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 24.06593558995519, + "end_to_end_tps": 85.09953798990507, + "timing_valid": true, + "steady_tps": 92.1168659708904, + "per_stream_tps": [ + 11.3, + 12.5, + 11.6, + 11.9, + 11.0, + 11.2, + 11.0, + 11.5 + ], + "ttft_s": [ + 0.61, + 1.47, + 1.79, + 2.13, + 0.88, + 0.38, + 0.16, + 1.17 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "864744d80fdf", + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "5743eea16ab2", + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "0a24c104609f", + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "28cedcb814fc", + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "22e8f3f09dfa", + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c28174794bf0", + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "c6d6ec2a35a8", + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "1a4e0d70467b", + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66" + } + ], + "client_intervals": [ + { + "start": 752725.127249038, + "first": 752725.74014675, + "last": 752748.218893762, + "end": 752748.218989656, + "elapsed_s": 23.09174061799422, + "chunk_offsets_s": [ + 0.6128977120388299, + 0.7250831050332636, + 1.0085238880710676, + 1.3098842810140923, + 1.6331603879807517, + 1.9761550590628758, + 2.327380906092003, + 2.5300072070676833, + 2.7322434920351952, + 2.9376785220811144, + 3.142762070056051, + 3.3479675670387223, + 3.5555231750477105, + 3.768201767001301, + 3.9805734989931807, + 4.197237987071276, + 4.415246610995382, + 4.62954052106943, + 4.848104538046755, + 5.065835069981404, + 5.280990671017207, + 5.498886316083372, + 5.718217685003765, + 5.9337880230741575, + 6.149907437036745, + 6.370704255998135, + 6.591159168048762, + 6.815658637089655, + 7.03463857609313, + 7.2509215930476785, + 7.466307433089241, + 7.686404249048792, + 7.903426093049347, + 8.119352683075704, + 8.338170705013908, + 8.555518405046314, + 8.768885708064772, + 8.982676474028267, + 9.199902684078552, + 9.424343917984515, + 9.64300501497928, + 9.866534761036746, + 10.088156176032498, + 10.306565736071207, + 10.528131074039266, + 10.751223303028382, + 10.980058812070638, + 11.201956526027061, + 11.424214069033042, + 11.640998454997316, + 11.861538710072637, + 12.080997743061744, + 12.302599636022933, + 12.524857492069714, + 12.743315345025621, + 12.957970098010264, + 13.179194617085159, + 13.394221064983867, + 13.606450406019576, + 13.828566667041741, + 14.047227827017196, + 14.262285820092075, + 14.483209111029282, + 14.69847252801992, + 14.91458144201897, + 15.13347527908627, + 15.349366444046609, + 15.567251117085107, + 15.78608436405193, + 16.00540617504157, + 16.219786462024786, + 16.4415630659787, + 16.66402831103187, + 16.891650286037475, + 17.11597669601906, + 17.329049260006286, + 17.546823187032714, + 17.768286660080776, + 17.985955815995112, + 18.204058989067562, + 18.418994694016874, + 18.635327298077755, + 18.853260888019577, + 19.070237548090518, + 19.29141063208226, + 19.513390238047577, + 19.734687238000333, + 19.950554303009994, + 20.160603535012342, + 20.383022553054616, + 20.60186972701922, + 20.819063664064743, + 21.03395971201826, + 21.2505687919911, + 21.46634325000923, + 21.68429547001142, + 21.90269390202593, + 22.10921463998966, + 22.31329840607941, + 22.51818870799616, + 22.724077365011908, + 22.911640939069912, + 23.091644724016078 + ], + "chunk_gaps_ms": { + "count": 102, + "p50": 217.890159052331, + "p95": 228.83550904225558, + "max": 351.2258470291272 + } + }, + { + "start": 752725.129015508, + "first": 752726.594510465, + "last": 752747.029847702, + "end": 752747.030082283, + "elapsed_s": 21.901066774968058, + "chunk_offsets_s": [ + 1.4654949570540339, + 1.630776294041425, + 1.9739400149555877, + 2.3256484280573204, + 2.5283725450281054, + 2.7308135050116107, + 2.936231762985699, + 3.1407770339865237, + 3.3459437059937045, + 3.553882032050751, + 3.766347130993381, + 3.9789290269836783, + 4.194673739024438, + 4.412979874992743, + 4.6280183179769665, + 4.8467475429642946, + 5.063872743980028, + 5.2789096130291, + 5.496848372044042, + 5.715973402955569, + 5.931476940982975, + 6.147475318051875, + 6.368467448046431, + 6.588990233023651, + 6.813422629027627, + 7.032500816043466, + 7.248676143004559, + 7.464901225990616, + 7.68425498297438, + 7.901934953057207, + 8.11766260000877, + 8.335606346954592, + 8.553298193961382, + 8.766506243031472, + 8.980485278065316, + 9.19770276604686, + 9.422247865004465, + 9.640718644950539, + 9.864840275957249, + 10.08620422205422, + 10.304725024965592, + 10.5263202090282, + 10.749280279036611, + 10.978216147981584, + 11.20010525197722, + 11.422334996052086, + 11.639001097995788, + 11.859697101986967, + 12.079195696976967, + 12.300697869970463, + 12.522952973959036, + 12.741585668059997, + 12.95613281102851, + 13.17738275998272, + 13.392299694009125, + 13.604599499958567, + 13.82665102602914, + 14.045499878004193, + 14.260712243034504, + 14.480911363963969, + 14.6966271760175, + 14.912749564973637, + 15.131030041957274, + 15.347028561052866, + 15.565414119046181, + 15.783981575048529, + 16.003270850982517, + 16.217967579956166, + 16.439966524019837, + 16.66187760804314, + 16.8893837480573, + 17.113926960038953, + 17.327140531968325, + 17.54499569500331, + 17.76614587905351, + 17.984058659058064, + 18.202244124026038, + 18.41735033201985, + 18.63362940296065, + 18.851454696967266, + 19.06831168103963, + 19.28954101796262, + 19.51169984904118, + 19.73218008503318, + 19.94860471901484, + 20.158709308016114, + 20.381072825053707, + 20.600151236052625, + 20.816784997005016, + 21.03238341398537, + 21.248595762997866, + 21.464681523968466, + 21.682288544951007, + 21.90083219402004 + ], + "chunk_gaps_ms": { + "count": 93, + "p50": 217.93875901494175, + "p95": 224.54509895760566, + "max": 351.70841310173273 + } + }, + { + "start": 752725.129974673, + "first": 752726.915347666, + "last": 752748.976471673, + "end": 752748.976631278, + "elapsed_s": 23.846656605019234, + "chunk_offsets_s": [ + 1.7853729929775, + 1.9733680969802663, + 2.324738378985785, + 2.5276042269542813, + 2.7298844429897144, + 2.935332212015055, + 3.1397728339070454, + 3.344919756986201, + 3.5528834969736636, + 3.7653308960143477, + 3.978186138905585, + 4.193901450955309, + 4.412266609957442, + 4.626499368925579, + 4.844780804938637, + 5.062752772006206, + 5.2777868720004335, + 5.495940370950848, + 5.7149674739921466, + 5.930563242989592, + 6.146464715944603, + 6.3675681540044025, + 6.587964316015132, + 6.812409995007329, + 7.031251052976586, + 7.2478016059612855, + 7.463501137914136, + 7.683348438935354, + 7.900922350934707, + 8.116216741967946, + 8.334191040019505, + 8.552179245976731, + 8.765717471018434, + 8.979446077952161, + 9.196581225958653, + 9.42119427700527, + 9.639589182916097, + 9.86372348992154, + 10.084999845945276, + 10.30401007796172, + 10.525226659956388, + 10.748371526016854, + 10.977294351905584, + 11.19918284798041, + 11.421441030921414, + 11.638100762967952, + 11.858850875985809, + 12.078139581019059, + 12.299823812907562, + 12.522100089932792, + 12.740706953918561, + 12.954888698994182, + 13.176515889004804, + 13.391032904968597, + 13.603510111919604, + 13.825276226969436, + 14.043981636990793, + 14.259797008009627, + 14.480349396006204, + 14.695621519931592, + 14.911479030968621, + 15.130451606004499, + 15.346483444911428, + 15.564740379923023, + 15.783619461930357, + 16.00250198598951, + 16.217415805906057, + 16.439350478933193, + 16.661077471915632, + 16.888865164946765, + 17.11362041090615, + 17.326563983922824, + 17.54460370191373, + 17.765794007922523, + 17.983342718915083, + 18.201786818914115, + 18.41633502591867, + 18.6329717959743, + 18.850724673015065, + 19.06777843798045, + 19.28918784891721, + 19.511072424938902, + 19.731894147000276, + 19.948230664012954, + 20.158442797954194, + 20.380878379917704, + 20.599614023929462, + 20.81649545696564, + 21.03148208593484, + 21.248230621917173, + 21.463995940983295, + 21.681829509907402, + 21.900376552017406, + 22.106863218010403, + 22.3109339249786, + 22.515373211936094, + 22.72072969994042, + 22.908591181971133, + 23.08780717290938, + 23.245888522011228, + 23.380890676984563, + 23.489562994916923, + 23.586871322942898, + 23.67985711793881, + 23.767114443005994, + 23.846496999962255 + ], + "chunk_gaps_ms": { + "count": 105, + "p50": 217.5487109925598, + "p95": 224.44567899219692, + "max": 351.3702820055187 + } + }, + { + "start": 752725.128421105, + "first": 752727.259826974, + "last": 752748.619780299, + "end": 752748.619954851, + "elapsed_s": 23.491533746011555, + "chunk_offsets_s": [ + 2.1314058690331876, + 2.3256279120687395, + 2.528406122000888, + 2.7308444409864023, + 2.9361592499772087, + 3.1407584230182692, + 3.3460668890038505, + 3.5537370700621977, + 3.766335146035999, + 3.979797321022488, + 4.195416786009446, + 4.413638837984763, + 4.628123754053377, + 4.846201188978739, + 5.064362594042905, + 5.279450771049596, + 5.497335869004019, + 5.716640606988221, + 5.932170183979906, + 6.148331352043897, + 6.369621427031234, + 6.590030568069778, + 6.81444215902593, + 7.03330954303965, + 7.249852734035812, + 7.465703070978634, + 7.685284547042102, + 7.902388010057621, + 8.117827860056423, + 8.33584581699688, + 8.554145233007148, + 8.76786138897296, + 8.981731901061721, + 9.198806843021885, + 9.42302548000589, + 9.641796619980596, + 9.865510713076219, + 10.086915228050202, + 10.305603896034881, + 10.527003352995962, + 10.750084172002971, + 10.978925043018535, + 11.200816162046976, + 11.423078811028972, + 11.639765765983611, + 11.860328537994064, + 12.079747786046937, + 12.301467690966092, + 12.523621201980859, + 12.742217167047784, + 12.956763511057943, + 13.177933632978238, + 13.39301451004576, + 13.605233047972433, + 13.827312292996794, + 14.046017047017813, + 14.261371748987585, + 14.481941694044508, + 14.697351688984782, + 14.913529195007868, + 15.132243853993714, + 15.34814382204786, + 15.566260772990063, + 15.785019602044486, + 16.004175694077276, + 16.219002822064795, + 16.440942920045927, + 16.662931462051347, + 16.890604328014888, + 17.115141891059466, + 17.32827768602874, + 17.546084517030977, + 17.767189154983498, + 17.9850171309663, + 18.203435497009195, + 18.41804971906822, + 18.634453444043174, + 18.852347744978033, + 19.069285148056224, + 19.29082938900683, + 19.512590880971402, + 19.73298796103336, + 19.949332655989565, + 20.159595458069816, + 20.381996551994234, + 20.600812182063237, + 20.817846979014575, + 21.032921868027188, + 21.249572798027657, + 21.46534314204473, + 21.683078913018107, + 21.901881357072853, + 22.108221349073574, + 22.312526494031772, + 22.51757055905182, + 22.722204867051914, + 22.91034151497297, + 23.089919460006058, + 23.247299322043546, + 23.382490976015106, + 23.49135919404216 + ], + "chunk_gaps_ms": { + "count": 100, + "p50": 217.77130098780617, + "p95": 223.71409309562296, + "max": 228.8408710155636 + } + }, + { + "start": 752725.127420105, + "first": 752726.012017486, + "last": 752749.192405804, + "end": 752749.192821194, + "elapsed_s": 24.06540108891204, + "chunk_offsets_s": [ + 0.8845973809948191, + 1.0079735749168321, + 1.3091282059904188, + 1.6318072049180046, + 1.9753797159064561, + 2.3266048579243943, + 2.530219865962863, + 2.732176707009785, + 2.9372085019713268, + 3.1418092109961435, + 3.34715349401813, + 3.554989603930153, + 3.7673907509306446, + 3.980308152968064, + 4.196720787906088, + 4.415110384928994, + 4.629405190004036, + 4.8481446099467576, + 5.065727457986213, + 5.28073435195256, + 5.498780480003916, + 5.7180826580151916, + 5.93372523796279, + 6.1496105319820344, + 6.370816714013927, + 6.59110396890901, + 6.815532651962712, + 7.034043135005049, + 7.250151644926518, + 7.466210126993246, + 7.685520386905409, + 7.903337605996057, + 8.118699228973128, + 8.336641758913174, + 8.554785843007267, + 8.768196163931862, + 8.982165852910839, + 9.199217285960913, + 9.423918645945378, + 9.642258370993659, + 9.866623756010085, + 10.087720245937817, + 10.305955604999326, + 10.527660030988045, + 10.750473505933769, + 10.9796127040172, + 11.201430237968452, + 11.423714139964432, + 11.640453202999197, + 11.861122272908688, + 12.080515434965491, + 12.302190768998116, + 12.524207527982071, + 12.74287190893665, + 12.957606522017159, + 13.178663736907765, + 13.393757960991934, + 13.605977154918946, + 13.828004179988056, + 14.046727993991226, + 14.262395491008647, + 14.482609062921256, + 14.69758998195175, + 14.914612261927687, + 15.13271909893956, + 15.348709392012097, + 15.567059820983559, + 15.786086728912778, + 16.004968068911694, + 16.219919689930975, + 16.441622692975216, + 16.663993195979856, + 16.891356230014935, + 17.115889632957987, + 17.329177077976055, + 17.546720008016564, + 17.768289217958227, + 17.98574028990697, + 18.20400206092745, + 18.41914736595936, + 18.63549003598746, + 18.853428858914413, + 19.070427539991215, + 19.29179537296295, + 19.513502579997294, + 19.734371208935045, + 19.950945765012875, + 20.160952011938207, + 20.383337965933606, + 20.601883950992487, + 20.81901009590365, + 21.033862885902636, + 21.250969392014667, + 21.466669416986406, + 21.684197090915404, + 21.902623898000456, + 22.109284362988546, + 22.313449956011027, + 22.518116622930393, + 22.72410468000453, + 22.911757248919457, + 23.090963060967624, + 23.24838344496675, + 23.383656924939714, + 23.492507732007653, + 23.58932036301121, + 23.682291305973195, + 23.769563355017453, + 23.849625505972654, + 23.912377242930233, + 23.965107767959125, + 24.016141877975315, + 24.06498569890391 + ], + "chunk_gaps_ms": { + "count": 112, + "p50": 217.28860842995346, + "p95": 227.36303403507918, + "max": 351.2251420179382 + } + }, + { + "start": 752725.126981804, + "first": 752725.504246612, + "last": 752748.21818024, + "end": 752748.218532704, + "elapsed_s": 23.091550900018774, + "chunk_offsets_s": [ + 0.37726480793207884, + 0.45570136001333594, + 0.7253254849929363, + 1.0087314429692924, + 1.3101871719118208, + 1.6327026109211147, + 1.975896404008381, + 2.3274905330035836, + 2.5303242289228365, + 2.732543869991787, + 2.937999081914313, + 3.1425298049580306, + 3.3480158189777285, + 3.5559546899748966, + 3.7684226859128103, + 3.9809040930122137, + 4.19705536798574, + 4.415324374916963, + 4.62987767602317, + 4.847985854023136, + 5.066199894994497, + 5.281218423973769, + 5.499078554916196, + 5.718312028911896, + 5.934123464976437, + 6.149839581921697, + 6.370893438928761, + 6.591379286954179, + 6.81570048793219, + 7.034595303004608, + 7.2509837369434536, + 7.4668727070093155, + 7.687126665026881, + 7.903505377937108, + 8.119352621026337, + 8.338391431956552, + 8.555945644970052, + 8.768957694992423, + 8.982689086929895, + 9.20021157595329, + 9.42452212201897, + 9.643317875918001, + 9.867194456979632, + 10.088486384949647, + 10.306622839998454, + 10.528279638965614, + 10.750999507959932, + 10.980115643935278, + 11.201939483988099, + 11.423985761008225, + 11.640737083973363, + 11.861484667984769, + 12.08102661697194, + 12.302438816986978, + 12.524698384921066, + 12.743252692976967, + 12.957982567953877, + 13.179030005005188, + 13.394083114922978, + 13.606282064924017, + 13.828393669915386, + 14.047333086957224, + 14.262478251941502, + 14.483306000940502, + 14.698517133947462, + 14.915201029973105, + 15.133798877941445, + 15.349538647918962, + 15.567775978939608, + 15.786579970968887, + 16.005533279967494, + 16.220284533919767, + 16.442306604003534, + 16.66449229500722, + 16.89197991800029, + 17.116447416949086, + 17.329680306022055, + 17.547559842001647, + 17.768667793017812, + 17.98642245598603, + 18.20490987796802, + 18.419137018965557, + 18.635833386913873, + 18.853603850933723, + 19.070817654021084, + 19.292117070988752, + 19.513976456946693, + 19.734855056973174, + 19.95105995994527, + 20.161358864977956, + 20.38355706492439, + 20.60252448997926, + 20.81952396593988, + 21.03451056301128, + 21.251141903921962, + 21.46692130691372, + 21.684739336022176, + 21.90346843493171, + 22.109905713936314, + 22.313854136969894, + 22.518847806961276, + 22.724012315971777, + 22.911989536019973, + 23.091198435984552 + ], + "chunk_gaps_ms": { + "count": 103, + "p50": 217.8795359795913, + "p95": 269.6241249796003, + "max": 351.5941289952025 + } + }, + { + "start": 752725.127775857, + "first": 752725.28930325, + "last": 752748.51099626, + "end": 752748.511089481, + "elapsed_s": 23.383313623955473, + "chunk_offsets_s": [ + 0.16152739303652197, + 0.21973906201310456, + 0.454630741965957, + 0.7243264690041542, + 1.0077980590285733, + 1.3089381110621616, + 1.6316205030307174, + 1.9748716220492497, + 2.326215593959205, + 2.5299940820550546, + 2.731198113062419, + 2.9367537980433553, + 3.1413555790204555, + 3.3466482979711145, + 3.5545499590225518, + 3.766930782003328, + 3.980681298999116, + 4.1958039390156046, + 4.413974986993708, + 4.628358182031661, + 4.847510294057429, + 5.0649022640427575, + 5.2802592270309106, + 5.4978220539633185, + 5.717022783006541, + 5.932660722057335, + 6.148604798014276, + 6.3696511980379, + 6.590106589952484, + 6.814457462984137, + 7.033403377979994, + 7.249750058050267, + 7.465615486027673, + 7.686207302031107, + 7.902840572060086, + 8.118181454017758, + 8.337132403044961, + 8.554705788963474, + 8.768035914981738, + 8.98209892201703, + 9.199074801057577, + 9.423636095016263, + 9.642178221023642, + 9.866354168974794, + 10.087286010966636, + 10.305904533015564, + 10.52738661901094, + 10.75079217099119, + 10.979381893994287, + 11.20127160497941, + 11.423607966979034, + 11.640346757951193, + 11.860866592032835, + 12.080304212984629, + 12.301981259020977, + 12.52413172996603, + 12.742731407983229, + 12.957315840991214, + 13.178443216951564, + 13.39361958799418, + 13.605805511004291, + 13.827801008010283, + 14.046600728062913, + 14.26189924206119, + 14.482323999982327, + 14.697777605964802, + 14.91362540202681, + 15.132509796996601, + 15.34887292899657, + 15.56682411301881, + 15.785616790992208, + 16.004782613017596, + 16.219396475004032, + 16.441461115027778, + 16.663851669989526, + 16.891399882966653, + 17.115727653959766, + 17.328718342003413, + 17.54660589597188, + 17.767664922983386, + 17.98570005095098, + 18.203879787004553, + 18.418741010013036, + 18.635004510055296, + 18.852850144030526, + 19.069832836976275, + 19.29120297404006, + 19.513110948028043, + 19.733698680065572, + 19.950180975021794, + 20.160319012007676, + 20.38271823397372, + 20.601980271050707, + 20.818616462987848, + 21.033342872047797, + 21.250509681063704, + 21.466349769034423, + 21.683876979979686, + 21.902632324956357, + 22.10878433904145, + 22.312995285959914, + 22.517504508956335, + 22.72301245597191, + 22.91116414696444, + 23.0902703050524, + 23.24819577904418, + 23.383220402989537 + ], + "chunk_gaps_ms": { + "count": 106, + "p50": 217.8665939718485, + "p95": 269.6957270381972, + "max": 351.34397190995514 + } + }, + { + "start": 752725.126885604, + "first": 752726.294587194, + "last": 752748.375931963, + "end": 752748.376108051, + "elapsed_s": 23.24922244704794, + "chunk_offsets_s": [ + 1.1677015899913386, + 1.310140953049995, + 1.6330628920113668, + 1.97642405400984, + 2.327534159994684, + 2.5300343659473583, + 2.7329106490360573, + 2.938469612970948, + 3.142575160949491, + 3.3479323589708656, + 3.5557334680343047, + 3.768271657056175, + 3.9811485529644415, + 4.19749894796405, + 4.415457624010742, + 4.629860023967922, + 4.848509725998156, + 5.066327958949842, + 5.281231164000928, + 5.499283822020516, + 5.7185288820182905, + 5.934189529973082, + 6.150226652040146, + 6.371111092972569, + 6.591692163026892, + 6.8159377300180495, + 7.034743372001685, + 7.251016914029606, + 7.467317359056324, + 7.685986215015873, + 7.903685998055153, + 8.119668634026311, + 8.338431618059985, + 8.555998219060712, + 8.769599369959906, + 8.983079229015857, + 9.200232782051899, + 9.424670270993374, + 9.64312320703175, + 9.867322040023282, + 10.087986827013083, + 10.307027689996175, + 10.528049026033841, + 10.750940183061175, + 10.980001313961111, + 11.201866251998581, + 11.424304701969959, + 11.640881870989688, + 11.861523382016458, + 12.08097314904444, + 12.302678443025798, + 12.524904115009122, + 12.74345714203082, + 12.958034790004604, + 13.1790760490112, + 13.394217196037062, + 13.606455610948615, + 13.82857812999282, + 14.047372921020724, + 14.262725176056847, + 14.483243235968985, + 14.69816886098124, + 14.915003338013776, + 15.133214695029892, + 15.34907774196472, + 15.567471398040652, + 15.786070205038413, + 16.00532294495497, + 16.22022061701864, + 16.442049041041173, + 16.664345943019725, + 16.89158707705792, + 17.116269008954987, + 17.329458127962425, + 17.547301672049798, + 17.768419448984787, + 17.986113953986205, + 18.204484198009595, + 18.41953364002984, + 18.635601527057588, + 18.85353716602549, + 19.070525701041333, + 19.291830254951492, + 19.513907450949773, + 19.734743705019355, + 19.95093700499274, + 20.16101187595632, + 20.383465221966617, + 20.60218489402905, + 20.819311807048507, + 21.034121579956263, + 21.25102306297049, + 21.467146976967342, + 21.684557072003372, + 21.902832483989187, + 22.109880493953824, + 22.313710074988194, + 22.51872213196475, + 22.724316691048443, + 22.91213416296523, + 23.0913683649851, + 23.249046358978376 + ], + "chunk_gaps_ms": { + "count": 101, + "p50": 217.8182329516858, + "p95": 224.68193189706653, + "max": 351.11010598484427 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "864744d80fdfd16cd80fb2b19937ec5647362842dc58810e5c3e2714eb5e7030", + "output_sha256": "fd7ce16d9ba2f0764277874a1659965a178e331e7f3f5da41cae2cb3506d6812", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "5743eea16ab2718c6e1c4b80ffa448455b9fee704de97adf831204acd245cbfe", + "output_sha256": "ad4f59f9723f269152a4242ffecd6c21ffbd01d9193db7483c9e2d5f4c467122", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "0a24c104609f5940af40c0eee5c0303dcd0c3aeea6fe1fbb63056ac045d692f1", + "output_sha256": "2df4b0f05272c1f2a414954a83fa5d36ad9111847a1b945007a98321cd0337be", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "28cedcb814fcf4bf1fafd4003c0016058814e095cb5a6df86afe09c14a0b5cf6", + "output_sha256": "c90800517599819d355f2b1dfe65c8122085005971e4042fc1f59302acf98767", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "22e8f3f09dfa97d2ac6a112cda9761e738c922ece421b624959408571bd13dbf", + "output_sha256": "5a1661a3656bd3f8b5e797a067276e69036e3e5096d94f155ea986f9645828f9", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c28174794bf0eb08aea0372f51c548b4637e199a52e841953b55274ac8ebac9c", + "output_sha256": "aa304ac93bd7e2677db60e600a2de43c0d3298d7776922eaac0a3accfa31c720", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "c6d6ec2a35a8c901a2a6225efcfdeb9c58033c231f047797428c701bc00e39d6", + "output_sha256": "e89b80b6b283ce6f8f85a7498de9ca2ceddc5343ccd67cc678a53cd69dc0d054", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "1a4e0d70467b76a0bda3ef0a3dfc1ad6f3e2bd43c294c3126496f45de2957390", + "output_sha256": "b46cab69420f33686401190ec67144d03476813ce65bbc28ca0348073e34af66", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 70.6, + "per_stream_tps": 70.6, + "ttft_s_max": 0.11, + "timing_valid": true, + "end_to_end_tps": 68.74996928956655, + "makespan_s": 3.7236380269750953, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 70.58038353073701, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 3.7236380269750953, + "end_to_end_tps": 68.74996928956655, + "timing_valid": true, + "per_stream_tps": [ + 70.6 + ], + "ttft_s": [ + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 752764.90164512, + "first": 752765.012263023, + "last": 752768.625164898, + "end": 752768.625283147, + "elapsed_s": 3.7236380269750953, + "chunk_offsets_s": [ + 0.11061790294479579, + 0.16909199499059469, + 0.22758359496947378, + 0.2899355540284887, + 0.3549572649644688, + 0.4228496690047905, + 0.48987672000657767, + 0.5554271680302918, + 0.6207007890334353, + 0.6867856869939715, + 0.7526790410047397, + 0.8187316280091181, + 0.8833818519487977, + 0.9477436150191352, + 1.0122241870267317, + 1.0771138849668205, + 1.1428479680325836, + 1.2077804879518226, + 1.2653934709960595, + 1.3162786740576848, + 1.3666480709798634, + 1.4200780059909448, + 1.4804535589646548, + 1.5399974159663543, + 1.59796554595232, + 1.6575052100233734, + 1.7234909939579666, + 1.788461187039502, + 1.8505802620202303, + 1.911960909026675, + 1.9937732849502936, + 2.0754582099616528, + 2.158618406043388, + 2.2441548330243677, + 2.3242475809529424, + 2.4097677799873054, + 2.4922679810551926, + 2.5776864299550653, + 2.6613934299675748, + 2.746739096008241, + 2.8305053090443835, + 2.9144647140055895, + 2.997611018945463, + 3.082972666947171, + 3.1666572759859264, + 3.25084935198538, + 3.334760503959842, + 3.4185820859856904, + 3.5023046230198815, + 3.5835857490310445, + 3.6665163110010326, + 3.7235197779955342 + ], + "chunk_gaps_ms": { + "count": 51, + "p50": 65.9857839345932, + "p95": 85.4184488998726, + "max": 85.53642698097974 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 81.8, + "per_stream_tps": 41.6, + "ttft_s_max": 0.28, + "timing_valid": true, + "end_to_end_tps": 80.73287904860639, + "makespan_s": 6.341901912004687, + "steady_tps": 83.6, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 81.84234478218544, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 6.341901912004687, + "end_to_end_tps": 80.73287904860639, + "timing_valid": true, + "steady_tps": 83.61268628262641, + "per_stream_tps": [ + 42.0, + 41.3 + ], + "ttft_s": [ + 0.28, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 752768.62748033, + "first": 752768.904849813, + "last": 752774.969099239, + "end": 752774.969268986, + "elapsed_s": 6.341788656078279, + "chunk_offsets_s": [ + 0.27736948302481323, + 0.360168393002823, + 0.4471048020059243, + 0.5325900590978563, + 0.6227960030082613, + 0.7122525570448488, + 0.8001291480613872, + 0.8862300220644102, + 0.9692661720328033, + 1.0513437050394714, + 1.138451447011903, + 1.2247000799980015, + 1.3163245230680332, + 1.4060680910479277, + 1.4962480220710859, + 1.5867646960541606, + 1.6738690880592912, + 1.765982224023901, + 1.855848745093681, + 1.9467556290328503, + 2.040414784103632, + 2.12846017209813, + 2.2147368390578777, + 2.302050737082027, + 2.3917536820517853, + 2.478545557009056, + 2.5610019681043923, + 2.6457012310856953, + 2.730499779107049, + 2.815225239028223, + 2.899132497026585, + 2.986232115072198, + 3.071747932001017, + 3.15732699399814, + 3.2452570101013407, + 3.3333737030625343, + 3.416722027002834, + 3.503634499036707, + 3.5919051410164684, + 3.67997500509955, + 3.765512488083914, + 3.8525406640255824, + 3.940398292033933, + 4.027530154096894, + 4.114065860048868, + 4.2022747290320694, + 4.290111104026437, + 4.378242417005822, + 4.466327656060457, + 4.553468491998501, + 4.6402708570240065, + 4.72756713302806, + 4.813118905061856, + 4.901412731036544, + 4.989331695018336, + 5.076765952049755, + 5.164437744999304, + 5.2516166371060535, + 5.338628607103601, + 5.42507545708213, + 5.5123779700370505, + 5.6006758731091395, + 5.687469674041495, + 5.773502374067903, + 5.860940387006849, + 5.947246599011123, + 6.031987225054763, + 6.115383414085954, + 6.20239842904266, + 6.28468847007025, + 6.341618909034878 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 87.1060669887811, + "p95": 90.90688393916935, + "max": 93.65915507078171 + } + }, + { + "start": 752768.627367074, + "first": 752768.737606222, + "last": 752774.912280006, + "end": 752774.912435334, + "elapsed_s": 6.285068259923719, + "chunk_offsets_s": [ + 0.11023914802353829, + 0.17021560901775956, + 0.36039818602148443, + 0.4474485069513321, + 0.5328122499631718, + 0.6230576420202851, + 0.712460489012301, + 0.8004223610041663, + 0.8864733210066333, + 0.9695619299309328, + 1.0516907869605348, + 1.1388390959473327, + 1.2250748629448935, + 1.3165597569895908, + 1.4064218129497021, + 1.4966174439759925, + 1.587043875013478, + 1.6740918869618326, + 1.765993122011423, + 1.8562578710261732, + 1.947024549008347, + 2.0407904300373048, + 2.128964373958297, + 2.215097154956311, + 2.3019894280005246, + 2.3919958770275116, + 2.47880505095236, + 2.5613457040162757, + 2.646051752031781, + 2.731164556927979, + 2.8154739789897576, + 2.8995226259576157, + 2.9866750719957054, + 3.072058476973325, + 3.1579839639598504, + 3.2455428140237927, + 3.333677557995543, + 3.4173179109347984, + 3.503947829012759, + 3.5922065479680896, + 3.6801894339732826, + 3.765424949931912, + 3.8524000589968637, + 3.940697122947313, + 4.027826632023789, + 4.114511296967976, + 4.202588474960066, + 4.290444965939969, + 4.378598026931286, + 4.466777989990078, + 4.553954288945533, + 4.640918000019155, + 4.727609543013386, + 4.813518635928631, + 4.901715114014223, + 4.989705980988219, + 5.077145822928287, + 5.1647550439229235, + 5.252162221935578, + 5.338918603956699, + 5.42572354292497, + 5.5126831700326875, + 5.6010330190183595, + 5.687782266992144, + 5.7737968190340325, + 5.860916686942801, + 5.947523216949776, + 6.03266134893056, + 6.115624857018702, + 6.2026805189670995, + 6.284912931965664 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 87.0529914391227, + "p95": 91.48489404469728, + "max": 190.18257700372487 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 112.0, + "per_stream_tps": 29.1, + "ttft_s_max": 0.68, + "timing_valid": true, + "end_to_end_tps": 111.06214999689927, + "makespan_s": 9.220062820939347, + "steady_tps": 117.4, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 112.01382630243745, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 9.220062820939347, + "end_to_end_tps": 111.06214999689927, + "timing_valid": true, + "steady_tps": 117.41678542901006, + "per_stream_tps": [ + 29.0, + 29.9, + 29.3, + 28.8 + ], + "ttft_s": [ + 0.28, + 0.68, + 0.47, + 0.11 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 752774.973754915, + "first": 752775.253848466, + "last": 752784.052752892, + "end": 752784.052944834, + "elapsed_s": 9.079189919051714, + "chunk_offsets_s": [ + 0.28009355103131384, + 0.3613076281035319, + 0.5731941880658269, + 0.8006390620721504, + 0.9224204090423882, + 1.0424046700354666, + 1.1634752820245922, + 1.2831847510533407, + 1.3993161290418357, + 1.513492580037564, + 1.625959561089985, + 1.7398557300912216, + 1.8628453420242295, + 1.9899447991047055, + 2.1219399120891467, + 2.248264590045437, + 2.3731101820012555, + 2.4980922820977867, + 2.6253890140214935, + 2.7549872220261022, + 2.8758069711038843, + 3.0013768740464, + 3.129462037002668, + 3.252208484103903, + 3.374912155093625, + 3.4957835170207545, + 3.615006466046907, + 3.732649028999731, + 3.8465501450700685, + 3.9585196090629324, + 4.074372681090608, + 4.194172007031739, + 4.30970619909931, + 4.427585969096981, + 4.546383525012061, + 4.665173031040467, + 4.78134636010509, + 4.897814324009232, + 5.018803817103617, + 5.140478263027035, + 5.26226300210692, + 5.383417807053775, + 5.503279631026089, + 5.621530475094914, + 5.740097958012484, + 5.863991907099262, + 5.985087771085091, + 6.107752067036927, + 6.228065668023191, + 6.348262813058682, + 6.46663439203985, + 6.583302477025427, + 6.703478146111593, + 6.822587633039802, + 6.9431425461079925, + 7.064490343094803, + 7.185753995086998, + 7.3060940550640225, + 7.422899104072712, + 7.541562332073227, + 7.670547928079031, + 7.829069685074501, + 7.988372013089247, + 8.150982930092141, + 8.311803523101844, + 8.46639799606055, + 8.625235361047089, + 8.747111525037326, + 8.863634734065272, + 8.976573156076483, + 9.078997977077961 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 120.68733107298613, + "p95": 160.82059300970286, + "max": 227.44487400632352 + } + }, + { + "start": 752774.975027208, + "first": 752775.653503293, + "last": 752784.193321891, + "end": 752784.19342788, + "elapsed_s": 9.218400671961717, + "chunk_offsets_s": [ + 0.678476085071452, + 0.7995478140655905, + 0.9214006160618737, + 1.041401167982258, + 1.1628298530122265, + 1.2823543730191886, + 1.3981212920043617, + 1.5124062770046294, + 1.6247619229834527, + 1.7389387090224773, + 1.8621293070027605, + 1.9890791970537975, + 2.120885518961586, + 2.2470809230580926, + 2.3720286800526083, + 2.496956625021994, + 2.6246823880355805, + 2.7540658650686964, + 2.87507294700481, + 3.0002813370665535, + 3.128458647057414, + 3.251290999003686, + 3.3738678919617087, + 3.4947862239787355, + 3.613844195031561, + 3.73114942596294, + 3.845593244070187, + 3.957458046032116, + 4.073397424072027, + 4.193341467995197, + 4.308906913967803, + 4.426610792055726, + 4.545383590972051, + 4.664207024034113, + 4.780597373028286, + 4.896667032968253, + 5.017880714964122, + 5.139400616986677, + 5.260729360976256, + 5.381839804002084, + 5.50184124102816, + 5.620542063028552, + 5.738584576989524, + 5.863003542996012, + 5.984302632045001, + 6.106747203972191, + 6.227177259977907, + 6.347235240973532, + 6.465647852048278, + 6.582243827055208, + 6.702442668029107, + 6.8214096470037475, + 6.941969376988709, + 7.063344154972583, + 7.184745691018179, + 7.30508959107101, + 7.421889886958525, + 7.540629195049405, + 7.669572222046554, + 7.827728930977173, + 7.98720273305662, + 8.150203488068655, + 8.310686365002766, + 8.465218937024474, + 8.62424562103115, + 8.74609752302058, + 8.862593543017283, + 8.975599450059235, + 9.078285683062859, + 9.161397536052391, + 9.218294683028944 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 120.49489299533889, + "p95": 159.02668400667608, + "max": 163.00075501203537 + } + }, + { + "start": 752774.974164167, + "first": 752775.442435508, + "last": 752784.136371629, + "end": 752784.13675753, + "elapsed_s": 9.162593363085762, + "chunk_offsets_s": [ + 0.4682713410584256, + 0.5731902830302715, + 0.8005134999984875, + 0.9224765640683472, + 1.0424637700198218, + 1.1635221390752122, + 1.2830386570421979, + 1.3990822091000155, + 1.5133087181020528, + 1.6254034150624648, + 1.7396267380099744, + 1.8630634180735797, + 1.9896085220389068, + 2.121792009100318, + 2.2479912530397996, + 2.373120872071013, + 2.497946428018622, + 2.6253487650537863, + 2.754987269989215, + 2.8754689811030403, + 3.0012456791009754, + 3.1295363070676103, + 3.2521944961044937, + 3.3748976869974285, + 3.4956872570328414, + 3.614810697035864, + 3.7323490630369633, + 3.846777054015547, + 3.958599032019265, + 4.074367927038111, + 4.193924228078686, + 4.309602433000691, + 4.427520371042192, + 4.546323863090947, + 4.665115434094332, + 4.781234112102538, + 4.897582676028833, + 5.01857148099225, + 5.140387299004942, + 5.261514145997353, + 5.382637408096343, + 5.50247223500628, + 5.621203815098852, + 5.739491466083564, + 5.863652573083527, + 5.98480251210276, + 6.107686324045062, + 6.227955644018948, + 6.348176682018675, + 6.466441935044713, + 6.583061147015542, + 6.703260964015499, + 6.822361329104751, + 6.943070897017606, + 7.06430235109292, + 7.185568724060431, + 7.306217005010694, + 7.422634921036661, + 7.541809227084741, + 7.670401481096633, + 7.82848060503602, + 7.9882789680268615, + 8.15102327207569, + 8.311709966044873, + 8.4661479430506, + 8.62524049601052, + 8.747212311020121, + 8.86342027306091, + 8.977083095000125, + 9.079348492086865, + 9.162207462009974 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 120.56499603204429, + "p95": 159.79836299084127, + "max": 227.323216968216 + } + }, + { + "start": 752774.973365059, + "first": 752775.087303164, + "last": 752783.950808919, + "end": 752783.950923661, + "elapsed_s": 8.977558601996861, + "chunk_offsets_s": [ + 0.11393810494337231, + 0.17342509794980288, + 0.36176965897902846, + 0.5737901340471581, + 0.8014291119761765, + 0.9230089610209689, + 1.0426286519505084, + 1.1641256710281596, + 1.2836587369674817, + 1.3998415649402887, + 1.513997946982272, + 1.626275849994272, + 1.7405290839960799, + 1.8635744670173153, + 1.9905694390181452, + 2.1223973659798503, + 2.2489425039384514, + 2.3738895249553025, + 2.498901134938933, + 2.626055981963873, + 2.755556443007663, + 2.876474531018175, + 3.0021353169577196, + 3.130536095937714, + 3.2527617489686236, + 3.3756337580271065, + 3.4962963099824265, + 3.6156598470406607, + 3.7327336230082437, + 3.847074363962747, + 3.959171182010323, + 4.075135492021218, + 4.1947725779609755, + 4.310205372981727, + 4.428084071027115, + 4.547165284049697, + 4.665696498006582, + 4.781826041988097, + 4.898424208979122, + 5.01960409199819, + 5.141153104021214, + 5.262483992031775, + 5.38356238103006, + 5.503067173995078, + 5.6224566149758175, + 5.740103257005103, + 5.864986652042717, + 5.985665744985454, + 6.108447088976391, + 6.228800585027784, + 6.348942567012273, + 6.467276458977722, + 6.583941758959554, + 6.704146794043481, + 6.823106699041091, + 6.943822763976641, + 7.065051928977482, + 7.1862610270036384, + 7.307062922976911, + 7.423503834987059, + 7.542354653007351, + 7.671046221978031, + 7.829346606973559, + 7.988826105953194, + 8.1516231240239, + 8.31231333897449, + 8.466980802011676, + 8.625736278947443, + 8.748060083948076, + 8.864289266988635, + 8.97744386002887 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 120.75898045441136, + "p95": 162.79701807070524, + "max": 227.63897792901844 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "code", + "temperature": 0.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 151.8, + "per_stream_tps": 20.5, + "ttft_s_max": 1.7, + "timing_valid": true, + "end_to_end_tps": 151.14135774187514, + "makespan_s": 13.55022894195281, + "steady_tps": 166.0, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 151.7909432918096, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 13.55022894195281, + "end_to_end_tps": 151.14135774187514, + "timing_valid": true, + "steady_tps": 165.9979834429901, + "per_stream_tps": [ + 20.6, + 21.5, + 20.8, + 20.2, + 21.1, + 20.4, + 20.2, + 20.2 + ], + "ttft_s": [ + 0.91, + 1.7, + 1.16, + 0.46, + 1.42, + 0.68, + 0.11, + 0.28 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + }, + { + "prompt_tokens": 14, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "f1f736f680e3", + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553" + } + ], + "client_intervals": [ + { + "start": 752784.197458583, + "first": 752785.103418382, + "last": 752797.505440777, + "end": 752797.505562048, + "elapsed_s": 13.30810346500948, + "chunk_offsets_s": [ + 0.9059597989544272, + 1.0459781679091975, + 1.3094744369154796, + 1.5884958539390936, + 1.8780976299894974, + 2.0600631489651278, + 2.236377107910812, + 2.413722245953977, + 2.5888234049780294, + 2.7672996589681134, + 2.953899633954279, + 3.143722128937952, + 3.33545258291997, + 3.532566348905675, + 3.728878160007298, + 3.9258033119840547, + 4.12136823090259, + 4.313902077963576, + 4.5051424079574645, + 4.697721270960756, + 4.891087153926492, + 5.083139293943532, + 5.273789800936356, + 5.459127833950333, + 5.637539620976895, + 5.8153201749082655, + 5.988351188949309, + 6.159962406964041, + 6.330254985950887, + 6.497497339965776, + 6.664773204945959, + 6.832213339977898, + 7.003589219995774, + 7.170907179941423, + 7.337268571951427, + 7.506035195896402, + 7.675823536934331, + 7.844821300008334, + 8.01296556997113, + 8.181657289969735, + 8.350637657917105, + 8.519753442960791, + 8.690317619941197, + 8.859649217920378, + 9.030925248982385, + 9.20102657401003, + 9.37239873490762, + 9.54184122895822, + 9.711139487917535, + 9.880379108944908, + 10.04926262400113, + 10.218233498977497, + 10.386439267895184, + 10.556400796980597, + 10.724987257970497, + 10.894156371010467, + 11.065646948991343, + 11.236263745930046, + 11.4066495699808, + 11.574354275944643, + 11.742218674975447, + 11.913124156999402, + 12.081179353990592, + 12.255082139978185, + 12.425160648999736, + 12.593054263968952, + 12.759104123921134, + 12.916835394920781, + 13.063185377977788, + 13.193267022958025, + 13.30798219400458 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 170.3392015188001, + "p95": 197.11376598570496, + "max": 289.6017760504037 + } + }, + { + "start": 752784.196095113, + "first": 752785.893659586, + "last": 752797.746078327, + "end": 752797.746206832, + "elapsed_s": 13.550111719057895, + "chunk_offsets_s": [ + 1.697564473026432, + 1.8795264090877026, + 2.0609212880954146, + 2.2378976810723543, + 2.4151737010106444, + 2.589930793037638, + 2.769002027111128, + 2.9546438780380413, + 3.14582216006238, + 3.337334971060045, + 3.533586792065762, + 3.72981895506382, + 3.9276254000142217, + 4.12276710011065, + 4.3154483360704035, + 4.5067036440595984, + 4.699223775998689, + 4.8921982070896775, + 5.084703459055163, + 5.275114560034126, + 5.46056600601878, + 5.639105707057752, + 5.816407080041245, + 5.990099294111133, + 6.161271739052609, + 6.331941286101937, + 6.499139824067242, + 6.66642133903224, + 6.833371122018434, + 7.004212163039483, + 7.171642529079691, + 7.337897164048627, + 7.506804119097069, + 7.676516846055165, + 7.845496173016727, + 8.014011667110026, + 8.183556367061101, + 8.351239237003028, + 8.52151303703431, + 8.691114308079705, + 8.860119301010855, + 9.031947853043675, + 9.201912432094105, + 9.37305907800328, + 9.542935688048601, + 9.712178816087544, + 9.881427479092963, + 10.050400611013174, + 10.219171752105467, + 10.387309956015088, + 10.55731896602083, + 10.725836310070008, + 10.895142297027633, + 11.06718453206122, + 11.23731579608284, + 11.407501998008229, + 11.57552862202283, + 11.744041688041762, + 11.914896632079035, + 12.082962599000894, + 12.25620833307039, + 12.426440231036395, + 12.59411898907274, + 12.76020271203015, + 12.918247450026684, + 13.064098955015652, + 13.194806142011657, + 13.309269618010148, + 13.409679110045545, + 13.49281503201928, + 13.549983214004897 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 170.07013701368123, + "p95": 195.1417000964284, + "max": 197.80644495040178 + } + }, + { + "start": 752784.19699017, + "first": 752785.352068003, + "last": 752797.606060549, + "end": 752797.60616332, + "elapsed_s": 13.409173149964772, + "chunk_offsets_s": [ + 1.1550778329838067, + 1.3098512470023707, + 1.5887523209676147, + 1.8789408049779013, + 2.060346549958922, + 2.2372215149225667, + 2.414515442913398, + 2.588665534975007, + 2.7681561479112133, + 2.9541256949305534, + 3.1443374349037185, + 3.3363294439623132, + 3.5327435529325157, + 3.7291201599873602, + 3.9265256309881806, + 4.1217471389099956, + 4.314589062007144, + 4.505858324933797, + 4.698239085962996, + 4.89167651894968, + 5.083887410000898, + 5.27437524497509, + 5.459745171945542, + 5.637974011944607, + 5.815752756898291, + 5.9889310479629785, + 6.160488254972734, + 6.330957219935954, + 6.498164767981507, + 6.6652924759546295, + 6.83297840191517, + 7.003988676005974, + 7.1714091199683025, + 7.3376666510012, + 7.506318100960925, + 7.676191642996855, + 7.845208208891563, + 8.013711620937102, + 8.182338847895153, + 8.351137019926682, + 8.520444042980671, + 8.690856973989867, + 8.859799777972512, + 9.031350084929727, + 9.201579194981605, + 9.372577843954787, + 9.542370933922939, + 9.711639395914972, + 9.8808835769305, + 10.049688100931235, + 10.218651838949881, + 10.386788857984357, + 10.556764965993352, + 10.725423936964944, + 10.894698653952219, + 11.065955859958194, + 11.23709881992545, + 11.407151700928807, + 11.574775495915674, + 11.743070744909346, + 11.914342648000456, + 12.081585834966972, + 12.25490743899718, + 12.42547591298353, + 12.593804985983297, + 12.759404520969838, + 12.917657602927648, + 13.063562781899236, + 13.193659437005408, + 13.308294641901739, + 13.409070378984325 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 170.32102053053677, + "p95": 196.4141089702025, + "max": 290.18848401028663 + } + }, + { + "start": 752784.197292086, + "first": 752784.661722071, + "last": 752797.259803851, + "end": 752797.260214296, + "elapsed_s": 13.062922210083343, + "chunk_offsets_s": [ + 0.4644299850333482, + 0.5678542690584436, + 0.7964765860233456, + 1.045861569000408, + 1.309021690976806, + 1.5880893900757656, + 1.8780687600374222, + 2.0590508540626615, + 2.236125427065417, + 2.4134515370242298, + 2.5878139110282063, + 2.7675835069967434, + 2.9531397570390254, + 3.143693180056289, + 3.3351699360646307, + 3.5319561660289764, + 3.7281800559721887, + 3.925395329017192, + 4.120887626078911, + 4.3136302010389045, + 4.50484375806991, + 4.697477048030123, + 4.890829088049941, + 5.0828206250444055, + 5.2734118240186945, + 5.458768852055073, + 5.6372714100871235, + 5.8147290530614555, + 5.988137964042835, + 6.159932498005219, + 6.330050979042426, + 6.4975487110204995, + 6.66450358601287, + 6.832984177977778, + 7.003362009068951, + 7.17078190902248, + 7.337108941981569, + 7.505711614037864, + 7.675467469030991, + 7.844456686056219, + 8.012906520045362, + 8.181546373991296, + 8.35024609498214, + 8.519788283039816, + 8.689966257079504, + 8.859010040061548, + 9.030630856985226, + 9.200762012042105, + 9.372456202981994, + 9.541798501973972, + 9.710905301035382, + 9.880186242051423, + 10.049244157038629, + 10.218113637063652, + 10.386154287029058, + 10.55602069303859, + 10.724713879055344, + 10.894263144000433, + 11.066045682062395, + 11.236597826005891, + 11.406360252061859, + 11.573941110051237, + 11.742898005060852, + 11.913856329047121, + 12.081493735080585, + 12.255176127073355, + 12.424770176992752, + 12.592831919086166, + 12.758696416974999, + 12.9174143060809, + 13.06251176504884 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 170.27790256543085, + "p95": 249.38498297706246, + "max": 289.97936996165663 + } + }, + { + "start": 752784.198255174, + "first": 752785.616175721, + "last": 752797.68898584, + "end": 752797.689069983, + "elapsed_s": 13.490814808988944, + "chunk_offsets_s": [ + 1.4179205470718443, + 1.587514347047545, + 1.8775224749697372, + 2.0591558649903163, + 2.236040111980401, + 2.4134387640515342, + 2.5880941720679402, + 2.766782707069069, + 2.9525426130276173, + 3.143420964013785, + 3.335022288025357, + 3.5315542119788006, + 3.7281833320157602, + 3.9250821439782158, + 4.12033338600304, + 4.3132421700283885, + 4.5044447960099205, + 4.697017017984763, + 4.890377699979581, + 5.082582734059542, + 5.273149289074354, + 5.458441999973729, + 5.637143812025897, + 5.814789540017955, + 5.988273650058545, + 6.159520540968515, + 6.32984059699811, + 6.4973061149939895, + 6.664506589062512, + 6.831674703047611, + 7.002586510032415, + 7.170181627036072, + 7.336308541009203, + 7.505185057991184, + 7.675064153037965, + 7.843989262008108, + 8.012485487968661, + 8.181130222976208, + 8.349766380037181, + 8.519270800985396, + 8.68943782907445, + 8.858497551060282, + 9.029615679057315, + 9.199677524971776, + 9.370844269986264, + 9.54068311303854, + 9.709981195977889, + 9.879309986019507, + 10.04818082600832, + 10.217086089076474, + 10.385062854038551, + 10.555002299020998, + 10.723603963037021, + 10.89293390698731, + 11.064973133965395, + 11.235279839020222, + 11.4052724520443, + 11.573093131999485, + 11.741058462997898, + 11.91204135096632, + 12.08058454503771, + 12.254013400990516, + 12.424037083983421, + 12.591827719006687, + 12.757549294037744, + 12.916135909967124, + 13.06145977997221, + 13.19269372196868, + 13.306829296052456, + 13.407568098045886, + 13.490730665973388 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 170.00814800849184, + "p95": 196.53192395344377, + "max": 290.0081279221922 + } + }, + { + "start": 752784.196445646, + "first": 752784.873887197, + "last": 752797.390762669, + "end": 752797.390971316, + "elapsed_s": 13.19452567002736, + "chunk_offsets_s": [ + 0.6774415510008112, + 0.7972848110366613, + 1.0468848920427263, + 1.3101206129649654, + 1.5892534289741889, + 1.879295627004467, + 2.0603165769716725, + 2.2375086760148406, + 2.414859174983576, + 2.589417811948806, + 2.7685327349463478, + 2.954625971033238, + 3.1450875679729506, + 3.336602331022732, + 3.53332133195363, + 3.729716998990625, + 3.926698434050195, + 4.12234455300495, + 4.3150096570607275, + 4.506303437054157, + 4.698697800980881, + 4.891899251961149, + 5.0842755190096796, + 5.274884433951229, + 5.4600891610607505, + 5.6384822559775785, + 5.8160208920016885, + 5.989399477024563, + 6.160868635983206, + 6.331303801969625, + 6.498754356987774, + 6.666031486005522, + 6.833153526997194, + 7.004569415003061, + 7.171798862051219, + 7.338168526999652, + 7.5069190999493, + 7.6767855520593, + 7.845499356044456, + 8.014016145956703, + 8.182523046969436, + 8.351333540980704, + 8.521035216050223, + 8.691108513972722, + 8.860066568944603, + 9.031842918950133, + 9.202195876976475, + 9.373163272044621, + 9.542809549951926, + 9.7120554150315, + 9.88146623596549, + 10.050382461049594, + 10.219285964034498, + 10.387286974000745, + 10.557098921970464, + 10.725790075026453, + 10.895208914997056, + 11.06671023205854, + 11.237679254962131, + 11.407627588021569, + 11.57526410499122, + 11.743156717973761, + 11.914506523986347, + 12.082813433022238, + 12.256141518009827, + 12.425932788057253, + 12.593999651959166, + 12.759432323975489, + 12.91817208903376, + 13.063693084055558, + 13.194317023037001 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 170.3940620063804, + "p95": 249.600081006065, + "max": 290.0421980302781 + } + }, + { + "start": 752784.196518541, + "first": 752784.306541318, + "last": 752796.955744393, + "end": 752796.95659945, + "elapsed_s": 12.760080909007229, + "chunk_offsets_s": [ + 0.11002277699299157, + 0.16919120599050075, + 0.3577440300723538, + 0.5685813560849056, + 0.7975036380812526, + 1.0467739100567997, + 1.3099731750553474, + 1.5889491280540824, + 1.8789467579917982, + 2.0599981140112504, + 2.237028900999576, + 2.4143899460323155, + 2.5890003690728918, + 2.7684121830388904, + 2.9540919930441305, + 3.14509680098854, + 3.3361417279811576, + 3.5328323390567675, + 3.7290029540890828, + 3.92633895506151, + 4.121702715056017, + 4.314439737005159, + 4.505580767057836, + 4.698309148079716, + 4.891648450051434, + 5.083830535993911, + 5.274306913022883, + 5.459711614064872, + 5.637881995062344, + 5.815665878006257, + 5.989007511991076, + 6.160353079088964, + 6.3309800900751725, + 6.497989327064715, + 6.6654187419917434, + 6.83388253103476, + 7.003899140981957, + 7.171320673078299, + 7.337659595068544, + 7.506434573093429, + 7.676129920990206, + 7.845184687059373, + 8.013640552992001, + 8.182371673057787, + 8.350965038058348, + 8.520469186012633, + 8.690791620058008, + 8.859739337000065, + 9.032261223066598, + 9.201931457035244, + 9.373173273052089, + 9.543096308014356, + 9.712019726983272, + 9.8812730780337, + 10.050260085030459, + 10.219026121078059, + 10.387346568051726, + 10.557287277071737, + 10.725660928990692, + 10.894604223081842, + 11.066872070077807, + 11.237132142065093, + 11.407323305029422, + 11.574580310028978, + 11.743898489046842, + 11.913877923041582, + 12.082186112063937, + 12.255681638023816, + 12.425473420065828, + 12.593772039050236, + 12.759225852089003 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 170.47472251579165, + "p95": 249.27027197554708, + "max": 289.9976299377158 + } + }, + { + "start": 752784.19597789, + "first": 752784.473299801, + "last": 752797.114184229, + "end": 752797.114428883, + "elapsed_s": 12.918450992903672, + "chunk_offsets_s": [ + 0.27732191095128655, + 0.3582530589774251, + 0.5691533739445731, + 0.7979591839248314, + 1.0476002648938447, + 1.3107260440010577, + 1.5895328599726781, + 1.8794392239069566, + 2.060613626963459, + 2.2376134489895776, + 2.41498454392422, + 2.589316639932804, + 2.7692228709347546, + 2.9546816298970953, + 3.1450693398946896, + 3.3366370419971645, + 3.533435193938203, + 3.7295853729592636, + 3.9267602069303393, + 4.122295375913382, + 4.315037071006373, + 4.506205194978975, + 4.698900320916437, + 4.892485942924395, + 5.084302404895425, + 5.274887267965823, + 5.460322742932476, + 5.638732881983742, + 5.816272813943215, + 5.9895059309201315, + 6.160999189945869, + 6.331585457897745, + 6.498637406970374, + 6.666000920929946, + 6.834133573924191, + 7.0053445579251274, + 7.172226273920387, + 7.338716889964417, + 7.507292212918401, + 7.677150520961732, + 7.846174511942081, + 8.014679603977129, + 8.182765862904489, + 8.352003639913164, + 8.520939006935805, + 8.69161490094848, + 8.860977273900062, + 9.0324881169945, + 9.202292312984355, + 9.373667003004812, + 9.543236513971351, + 9.712470344966277, + 9.881768664927222, + 10.050658996915445, + 10.219528749003075, + 10.387853643973358, + 10.557604781002738, + 10.726135039934888, + 10.89537040493451, + 11.067039096960798, + 11.23796491289977, + 11.407924559898674, + 11.575571941910312, + 11.743424851913005, + 11.915321770007722, + 12.083029188914225, + 12.255820531980135, + 12.426864231936634, + 12.594501557992771, + 12.760367271956056, + 12.918206339003518 + ], + "chunk_gaps_ms": { + "count": 70, + "p50": 171.1273419787176, + "p95": 249.64108096901327, + "max": 289.9063639342785 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "f1f736f680e36fb36a46727385bb591461eeaf78e49e2712ab886e85ac497ba3", + "output_sha256": "5bb6f2535458db4bcfe375c6786a1734ecf4505d923cfdbe2ddc3d93d4e29553", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 1, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 34.9, + "per_stream_tps": 34.9, + "ttft_s_max": 0.16, + "timing_valid": true, + "end_to_end_tps": 34.24796634969847, + "makespan_s": 7.474896389059722, + "alone": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 34.87848044571409, + "requested_requests": 1, + "timed_requests": 1, + "makespan_s": 7.474896389059722, + "end_to_end_tps": 34.24796634969847, + "timing_valid": true, + "per_stream_tps": [ + 34.9 + ], + "ttft_s": [ + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 752817.328824702, + "first": 752817.492224217, + "last": 752824.803322543, + "end": 752824.803721091, + "elapsed_s": 7.474896389059722, + "chunk_offsets_s": [ + 0.16339951509144157, + 0.22566347802057862, + 0.2865038480376825, + 0.34856952109839767, + 0.4128629770129919, + 0.4785487400367856, + 0.5407829710748047, + 0.5931387920863926, + 0.6514646030263975, + 0.709281237097457, + 0.7663074440788478, + 0.8178160151001066, + 0.8686109409900382, + 0.9195085420506075, + 0.9704320990713313, + 1.0238158610882238, + 1.074228257057257, + 1.127095193020068, + 1.1791248311055824, + 1.2296838930342346, + 1.2803730240557343, + 1.3317764090606943, + 1.392050400027074, + 1.450964954099618, + 1.5074050590628758, + 1.558140070992522, + 1.6110886390088126, + 1.6706174850696698, + 1.7285035010427237, + 1.7874886370263994, + 1.8468158640898764, + 1.9118031010730192, + 1.9738547060405836, + 2.0259734940482304, + 2.0829659150913358, + 2.141586971003562, + 2.1944865030236542, + 2.245895871077664, + 2.2976780410390347, + 2.3505474870325997, + 2.401849461020902, + 2.454046583035961, + 2.5090414070291445, + 2.5612236780580133, + 2.6114938410464674, + 2.6625374970026314, + 2.7133579140063375, + 2.764762576087378, + 2.815622088033706, + 2.8661113590933383, + 2.9169061559950933, + 2.9669538150774315, + 3.018189680064097, + 3.070432551088743, + 3.121085691032931, + 3.1727801800007, + 3.225520194042474, + 3.28258098999504, + 3.334627923090011, + 3.3852118990616873, + 3.436838328023441, + 3.4864829090656713, + 3.5381268929922953, + 3.5890006630215794, + 3.6414280120516196, + 3.692832497996278, + 3.743583381990902, + 3.795463277027011, + 3.8478103569941595, + 3.8981106840074062, + 3.94894255802501, + 4.002191762090661, + 4.053054377087392, + 4.103574886103161, + 4.155414886074141, + 4.207549690036103, + 4.2577901179902256, + 4.308629145030864, + 4.360090824076906, + 4.411782063078135, + 4.463140044012107, + 4.513595805037767, + 4.566381857031956, + 4.6193400780903175, + 4.676325539010577, + 4.727797603001818, + 4.780013411073014, + 4.833086807047948, + 4.890830975025892, + 4.94789817801211, + 4.997432116069831, + 5.049291511066258, + 5.101660049054772, + 5.1583828090224415, + 5.207993816002272, + 5.258540361071937, + 5.309046738082543, + 5.359790677088313, + 5.411068355082534, + 5.461965135997161, + 5.514609161997214, + 5.57207036903128, + 5.62241858500056, + 5.673000959097408, + 5.725898274104111, + 5.783767003100365, + 5.842050906037912, + 5.89789872802794, + 5.949982224032283, + 6.0029636500403285, + 6.063457229058258, + 6.120352462050505, + 6.1759602430975065, + 6.235004965099506, + 6.300146621069871, + 6.36278371908702, + 6.414893572102301, + 6.472323457011953, + 6.531474373070523, + 6.588662600028329, + 6.639366451068781, + 6.691654573078267, + 6.742531813099049, + 6.794191315071657, + 6.8445521160028875, + 6.898484758101404, + 6.9545281460741535, + 7.011345282080583, + 7.068531921016984, + 7.118296651053242, + 7.170778806088492, + 7.2220981700811535, + 7.274556496995501, + 7.3243328290991485, + 7.374975405051373, + 7.425472213071771, + 7.474497841089033 + ], + "chunk_gaps_ms": { + "count": 136, + "p50": 52.12679598480463, + "p95": 62.23423103801906, + "max": 65.6857630237937 + } + } + ], + "equal_alone": [ + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 2, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.6, + "per_stream_tps": 25.6, + "ttft_s_max": 0.38, + "timing_valid": true, + "end_to_end_tps": 50.009440527290046, + "makespan_s": 10.23806694499217, + "steady_tps": 51.6, + "alone": { + "equal": 2, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 50.60841327759216, + "requested_requests": 2, + "timed_requests": 2, + "makespan_s": 10.23806694499217, + "end_to_end_tps": 50.009440527290046, + "timing_valid": true, + "steady_tps": 51.64049078347103, + "per_stream_tps": [ + 25.9, + 25.4 + ], + "ttft_s": [ + 0.38, + 0.16 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 752824.805053831, + "first": 752825.184738545, + "last": 752835.042653867, + "end": 752835.043025224, + "elapsed_s": 10.237971393042244, + "chunk_offsets_s": [ + 0.37968471401836723, + 0.4649587880121544, + 0.5564021210884675, + 0.6370910779805854, + 0.7226129540940747, + 0.8154603010043502, + 0.899097952991724, + 0.98613919201307, + 1.0692735090851784, + 1.1561357780592516, + 1.2447048210306093, + 1.3319075040053576, + 1.4186967890709639, + 1.5019163810648024, + 1.5962328630266711, + 1.6914205490611494, + 1.7852510610828176, + 1.8741527360398322, + 1.9640763450879604, + 2.057359852013178, + 2.1463553140638396, + 2.232747982023284, + 2.3247245689854026, + 2.418742126086727, + 2.507698298082687, + 2.596045264042914, + 2.6864355040015653, + 2.7772986710770056, + 2.8615022799931467, + 2.9488900280557573, + 3.0431132670491934, + 3.1343665180029348, + 3.2275408330606297, + 3.3229452200466767, + 3.4123141589807346, + 3.4973050160333514, + 3.5793512649834156, + 3.6664470370160416, + 3.7479279910912737, + 3.837545596063137, + 3.924318417091854, + 4.011986716999672, + 4.102033258066513, + 4.191883187042549, + 4.286689190077595, + 4.378915337030776, + 4.471816876088269, + 4.563446570071392, + 4.653570860042237, + 4.743163284030743, + 4.834417763049714, + 4.922109373030253, + 5.013766288058832, + 5.10222335299477, + 5.19470024202019, + 5.282904330058955, + 5.376508953049779, + 5.468667702982202, + 5.559990985086188, + 5.653218792984262, + 5.743002218077891, + 5.833912103087641, + 5.9220298250438645, + 6.011351716006175, + 6.104830767028034, + 6.1959072990575805, + 6.282935874070972, + 6.378313310095109, + 6.474635992082767, + 6.56696082407143, + 6.6560100230854005, + 6.7404342910740525, + 6.833612402086146, + 6.927034169086255, + 7.014097757055424, + 7.10355788806919, + 7.191322183003649, + 7.285773424082436, + 7.3780789960874245, + 7.467117098043673, + 7.554249554988928, + 7.641878894995898, + 7.732439890038222, + 7.821405851980671, + 7.910061730071902, + 8.000939193065278, + 8.09503232606221, + 8.186327732983045, + 8.271055211080238, + 8.359129243064672, + 8.449996793060564, + 8.538756186026148, + 8.621518298983574, + 8.707095022080466, + 8.798508565989323, + 8.88717765698675, + 8.976843135082163, + 9.067480455036275, + 9.159402613062412, + 9.246974092093296, + 9.33588662405964, + 9.423485771985725, + 9.510001317015849, + 9.598964674049057, + 9.684770758030936, + 9.771555628045462, + 9.86217666498851, + 9.951458618044853, + 10.040284736081958, + 10.122189403045923, + 10.188541890005581, + 10.237600036081858 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 89.2819530563429, + "p95": 94.45124107878655, + "max": 96.32268198765814 + } + }, + { + "start": 752824.804958279, + "first": 752824.965278052, + "last": 752834.993790601, + "end": 752834.99388068, + "elapsed_s": 10.188922400935553, + "chunk_offsets_s": [ + 0.16031977301463485, + 0.2233006739988923, + 0.4648147310363129, + 0.5563750120345503, + 0.6373852390097454, + 0.7225368130020797, + 0.8158291479339823, + 0.8991361350053921, + 0.9860413030255586, + 1.069112568977289, + 1.1564879489596933, + 1.2449290349613875, + 1.331827073940076, + 1.4186098460340872, + 1.5021947480272502, + 1.5962636999320239, + 1.6917896990198642, + 1.7850906329695135, + 1.8744296949589625, + 1.964123600977473, + 2.057677185977809, + 2.1467221119673923, + 2.2328808039892465, + 2.324857775005512, + 2.418800006969832, + 2.5079666959354654, + 2.596202331013046, + 2.686580855981447, + 2.7776234289631248, + 2.8615451030200347, + 2.9489317149855196, + 3.0434782890370116, + 3.134402218973264, + 3.2276001709979028, + 3.3230021740309894, + 3.4123686800012365, + 3.4973001029575244, + 3.579909006948583, + 3.666716697975062, + 3.748557968996465, + 3.837806263938546, + 3.9248773109866306, + 4.012229606043547, + 4.1023611050331965, + 4.192343648988754, + 4.286954818991944, + 4.379140062024817, + 4.472123343963176, + 4.5637529420200735, + 4.653922023018822, + 4.74351935996674, + 4.8348160539753735, + 4.9223323540063575, + 5.014159825979732, + 5.10248749400489, + 5.194966381997801, + 5.283170870970935, + 5.376860563992523, + 5.469115618034266, + 5.5602184949675575, + 5.653379588038661, + 5.743205004022457, + 5.834216137998737, + 5.922257766942494, + 6.011547524947673, + 6.105331265949644, + 6.196115238009952, + 6.283129233983345, + 6.378511791001074, + 6.474795299000107, + 6.567277004942298, + 6.656451600953005, + 6.740929430001415, + 6.833913587965071, + 6.927299684961326, + 7.014385869028047, + 7.103797400020994, + 7.191613576025702, + 7.286021464969963, + 7.378376740962267, + 7.4673718919511884, + 7.554084951989353, + 7.641875582979992, + 7.732727825990878, + 7.821700492990203, + 7.910339136025868, + 8.001341179944575, + 8.095375102944672, + 8.186627798946574, + 8.271276335930452, + 8.359386901953258, + 8.450277335010469, + 8.538947417982854, + 8.621947490959428, + 8.707468427950516, + 8.79887688299641, + 8.887441891944036, + 8.97723199997563, + 9.067731073009782, + 9.15967725100927, + 9.24722106102854, + 9.336116262944415, + 9.424090896034613, + 9.510320554021746, + 9.599461795995012, + 9.68476787803229, + 9.771843194961548, + 9.862447542021982, + 9.95168343896512, + 10.040501939947717, + 10.122601054026745, + 10.188832322019152 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 89.28975800517946, + "p95": 94.61117000319064, + "max": 241.51405703742057 + } + } + ], + "equal_alone": [ + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 4, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.8, + "per_stream_tps": 18.0, + "ttft_s_max": 0.89, + "timing_valid": true, + "end_to_end_tps": 69.31546214563338, + "makespan_s": 14.773038630955853, + "steady_tps": 72.3, + "alone": { + "equal": 4, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 69.80380845600295, + "requested_requests": 4, + "timed_requests": 4, + "makespan_s": 14.773038630955853, + "end_to_end_tps": 69.31546214563338, + "timing_valid": true, + "steady_tps": 72.26560841884174, + "per_stream_tps": [ + 17.9, + 17.7, + 18.1, + 18.4 + ], + "ttft_s": [ + 0.38, + 0.16, + 0.62, + 0.89 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 752835.046721175, + "first": 752835.426142219, + "last": 752849.704039503, + "end": 752849.704188758, + "elapsed_s": 14.65746758307796, + "chunk_offsets_s": [ + 0.37942104402463883, + 0.46458535408601165, + 0.7260861520189792, + 1.005181738990359, + 1.128647092031315, + 1.2490896460367367, + 1.3740709110861644, + 1.502530551981181, + 1.6229134640889242, + 1.74337469402235, + 1.8631265910807997, + 1.985917252022773, + 2.1078217470785603, + 2.2270900059957057, + 2.353737706085667, + 2.484629634069279, + 2.6191144180484116, + 2.7518000460695475, + 2.8866910149808973, + 3.0182450510328636, + 3.1468079619808123, + 3.274451284087263, + 3.402545926044695, + 3.5318543179892004, + 3.6661075920565054, + 3.7984289050800726, + 3.9230566810583696, + 4.049485890078358, + 4.17444610898383, + 4.296625639079139, + 4.4250728860497475, + 4.558216780074872, + 4.6914195159915835, + 4.821323701995425, + 4.949573973077349, + 5.078152940026484, + 5.20002025004942, + 5.317475090036169, + 5.433178798994049, + 5.552462573046796, + 5.672791476012208, + 5.797881209058687, + 5.922914453083649, + 6.047425779048353, + 6.176229186006822, + 6.308083578012884, + 6.439528257003985, + 6.5687515389872715, + 6.699291464989074, + 6.829494968056679, + 6.956656538997777, + 7.084496884024702, + 7.210773186059669, + 7.336742079001851, + 7.4674229100346565, + 7.595043459092267, + 7.721216049045324, + 7.850222019013017, + 7.982084317016415, + 8.112804852076806, + 8.241560243070126, + 8.373116943985224, + 8.499889618018642, + 8.628713180078194, + 8.757427475997247, + 8.886522095068358, + 9.013490870012902, + 9.143242055084556, + 9.27126591000706, + 9.403225605026819, + 9.535317769041285, + 9.65566547808703, + 9.780797914019786, + 9.909754373016767, + 10.041958190035075, + 10.172848679008894, + 10.297934522037394, + 10.426952395006083, + 10.560353472013958, + 10.691709698992781, + 10.821922193048522, + 10.945816219085827, + 11.072193248081021, + 11.199909048038535, + 11.32670732808765, + 11.454494195990264, + 11.588364245020784, + 11.725583623046987, + 11.858655947027728, + 11.987701424048282, + 12.112769022001885, + 12.236837904085405, + 12.358918569982052, + 12.478744145017117, + 12.600856431992725, + 12.724617792991921, + 12.850816881051287, + 12.977363944984972, + 13.104931846028194, + 13.230598266003653, + 13.357230868074112, + 13.482557379058562, + 13.608809826080687, + 13.73205410700757, + 13.850996594992466, + 13.968055788078345, + 14.08988428907469, + 14.214234479004517, + 14.336839101044461, + 14.457107887021266, + 14.567162953084335, + 14.657318328041583 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 127.16157094109803, + "p95": 134.25327406730503, + "max": 279.0955869713798 + } + }, + { + "start": 752835.046808389, + "first": 752835.207069311, + "last": 752849.613808597, + "end": 752849.614106307, + "elapsed_s": 14.567297918023542, + "chunk_offsets_s": [ + 0.16026092204265296, + 0.22233826003503054, + 0.4647628719685599, + 0.726315165986307, + 1.005223601940088, + 1.1286644709762186, + 1.2492793089477345, + 1.3743114619283006, + 1.5023383769439533, + 1.6232224110281095, + 1.7436464990023524, + 1.8632329929387197, + 1.9860546659911051, + 2.1082437520381063, + 2.2271213870262727, + 2.3538764329859987, + 2.484790203976445, + 2.6192611310398206, + 2.751906816032715, + 2.886781221954152, + 3.018330264952965, + 3.147297082003206, + 3.274563829996623, + 3.4028670829720795, + 3.5315078620333225, + 3.666087636956945, + 3.7986714399885386, + 3.9231740449322388, + 4.049669791944325, + 4.174567073001526, + 4.2963265179423615, + 4.425183014944196, + 4.558406122028828, + 4.691559826023877, + 4.82137940695975, + 4.949628317961469, + 5.078360861982219, + 5.200264865998179, + 5.317700517945923, + 5.433227750938386, + 5.552490993984975, + 5.67256043094676, + 5.798148419940844, + 5.923009812948294, + 6.0474963509477675, + 6.176432034932077, + 6.308122384943999, + 6.43966759101022, + 6.568787785014138, + 6.699468837003224, + 6.829513770993799, + 6.956837384030223, + 7.084605413023382, + 7.210494917933829, + 7.33695105696097, + 7.467467988957651, + 7.595042499015108, + 7.7212861879961565, + 7.850259096943773, + 7.981538597960025, + 8.112380960956216, + 8.241827292949893, + 8.373496716027148, + 8.499908564961515, + 8.62889452103991, + 8.75752849993296, + 8.88656384497881, + 9.01371021696832, + 9.14321997098159, + 9.271600106963888, + 9.403235862962902, + 9.535418392973952, + 9.65579682693351, + 9.780894408933818, + 9.90992888098117, + 10.041995170991868, + 10.173143220948987, + 10.298032360966317, + 10.427134486963041, + 10.560436108964495, + 10.691518116975203, + 10.821975257014856, + 10.94627616298385, + 11.072374108014628, + 11.199997446034104, + 11.326684859930538, + 11.45464999601245, + 11.588356739957817, + 11.725559826940298, + 11.858175246976316, + 11.987767562037334, + 12.112895922036842, + 12.23705200094264, + 12.358894021017477, + 12.47894710500259, + 12.60091920895502, + 12.724818576010875, + 12.850917040021159, + 12.97753323498182, + 13.10502632393036, + 13.230789478984661, + 13.357349046040326, + 13.482621068018489, + 13.60901965002995, + 13.732188606983982, + 13.850435593980365, + 13.968010835931636, + 14.089867934002541, + 14.214265811024234, + 14.33686190401204, + 14.456907054991461, + 14.56700020795688 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 127.32361303642392, + "p95": 134.5797749236226, + "max": 278.908435953781 + } + }, + { + "start": 752835.047476419, + "first": 752835.667401039, + "last": 752849.770642136, + "end": 752849.770721492, + "elapsed_s": 14.72324507299345, + "chunk_offsets_s": [ + 0.6199246200267226, + 0.7258603710215539, + 1.004709020955488, + 1.1282022499945015, + 1.2485582309309393, + 1.3736913120374084, + 1.5015755789354444, + 1.6224518680246547, + 1.7429298849310726, + 1.8625102499499917, + 1.9854503750102594, + 2.1073740100255236, + 2.2264939720043913, + 2.3532395280199125, + 2.4842439700150862, + 2.6187860439531505, + 2.7512057239655405, + 2.8861969659337774, + 3.0176142760319635, + 3.1466771230334416, + 3.2741124749882147, + 3.4022536060074344, + 3.530967677012086, + 3.6654656949685887, + 3.798057451029308, + 3.922450180980377, + 4.048967435956001, + 4.173994145938195, + 4.2959297159686685, + 4.424742414033972, + 4.557786867953837, + 4.691173697938211, + 4.820660313009284, + 4.949066336965188, + 5.07773455104325, + 5.199364703032188, + 5.317124263034202, + 5.432674171985127, + 5.551892734947614, + 5.6719598670024425, + 5.7975387839833274, + 5.9223089129664, + 6.046791402972303, + 6.175824590027332, + 6.307510026963428, + 6.438881941954605, + 6.568085861043073, + 6.6988792839692906, + 6.828894564998336, + 6.9562180179636925, + 7.0838885109405965, + 7.209880832000636, + 7.336239452008158, + 7.466839164961129, + 7.594434141996317, + 7.7207028430420905, + 7.849639810970984, + 7.980801244964823, + 8.111775996978395, + 8.241232426953502, + 8.373081926023588, + 8.499195695971139, + 8.628144718008116, + 8.756824224023148, + 8.88617805193644, + 9.013082160963677, + 9.142768856021576, + 9.271105624036863, + 9.402533091022633, + 9.534713877015747, + 9.655078724958003, + 9.780185507959686, + 9.909300265018828, + 10.041372893028893, + 10.172436416964047, + 10.297411601990461, + 10.426506543997675, + 10.559804821037687, + 10.691095020971261, + 10.82126590795815, + 10.945565485977568, + 11.071776040946133, + 11.19937846402172, + 11.326054563978687, + 11.453944184002466, + 11.58772513200529, + 11.724932843935676, + 11.857408465933986, + 11.987174664041959, + 12.112176219932735, + 12.236345340963453, + 12.358262188965455, + 12.478241789038293, + 12.600213925004937, + 12.724108811933547, + 12.850326751009561, + 12.976830160012469, + 13.104410607018508, + 13.230044492986053, + 13.356718478025869, + 13.482001445954666, + 13.608388649998233, + 13.73147356195841, + 13.849997504032217, + 13.967251015943475, + 14.089237286010757, + 14.213659983011894, + 14.336245273938403, + 14.456277398974635, + 14.566617997945286, + 14.656682173954323, + 14.723165716975927 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 126.9041090272367, + "p95": 133.7809480028227, + "max": 278.8486499339342 + } + }, + { + "start": 752835.046537481, + "first": 752835.931677323, + "last": 752849.81945254, + "end": 752849.819576112, + "elapsed_s": 14.773038630955853, + "chunk_offsets_s": [ + 0.8851398419355974, + 1.0054534629452974, + 1.1293891969835386, + 1.2494469679659232, + 1.3743706879904494, + 1.5026515949284658, + 1.623771733022295, + 1.7438222869532183, + 1.8634046849329025, + 1.9864351919386536, + 2.108417332987301, + 2.227348271990195, + 2.354104358004406, + 2.4852478619432077, + 2.61997929494828, + 2.7521077119745314, + 2.887268965016119, + 3.018848925945349, + 3.1478605170268565, + 3.2750129270134494, + 3.403078236966394, + 3.5322531670099124, + 3.666452703997493, + 3.798862735973671, + 3.923333751037717, + 4.049869008013047, + 4.174789045006037, + 4.296559643931687, + 4.425647586933337, + 4.558605163008906, + 4.692055427003652, + 4.821872927015647, + 4.950152578996494, + 5.078590146964416, + 5.200666834949516, + 5.317889812984504, + 5.433457965031266, + 5.552725767949596, + 5.673082594992593, + 5.798245475976728, + 5.922942762030289, + 6.047567578032613, + 6.176458423025906, + 6.308323904988356, + 6.43976170301903, + 6.568870791001245, + 6.699519117944874, + 6.829503241926432, + 6.957038376946002, + 7.084948509000242, + 7.2111048239748925, + 7.33698362193536, + 7.467563317040913, + 7.595127296983264, + 7.721486267982982, + 7.850694895954803, + 7.982034805929288, + 8.112921639927663, + 8.241999609977938, + 8.37351783993654, + 8.499858221970499, + 8.629011627985165, + 8.757395311957225, + 8.886893018963747, + 9.013890711008571, + 9.143213442992419, + 9.271670198999345, + 9.403178605949506, + 9.535275812027976, + 9.655903836945072, + 9.7807626449503, + 9.91009546793066, + 10.042087506968528, + 10.173471465939656, + 10.298185345949605, + 10.427202322985977, + 10.560341214993969, + 10.691963820019737, + 10.821931474958546, + 10.94618378195446, + 11.07244564103894, + 11.200165009940974, + 11.326742199016735, + 11.454722297028638, + 11.588347075972706, + 11.725557524012402, + 11.85870566801168, + 11.987928788992576, + 12.112738794996403, + 12.237106187036261, + 12.358951871981844, + 12.479080150951631, + 12.601246493984945, + 12.7248383249389, + 12.851037780987099, + 12.977582396939397, + 13.105157546931878, + 13.230746657936834, + 13.357474907999858, + 13.482706059003249, + 13.609181212959811, + 13.732274317997508, + 13.850872287992388, + 13.96809201897122, + 14.090024312026799, + 14.214383478974923, + 14.336984900990501, + 14.457452566013671, + 14.567922433023341, + 14.657458686968312, + 14.723887661006302, + 14.772915059002116 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 126.75608601421118, + "p95": 133.4502639947459, + "max": 137.21044803969562 + } + } + ], + "equal_alone": [ + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + }, + { + "prompt": "chat", + "temperature": 0.0, + "streams": 8, + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 88.3, + "per_stream_tps": 11.6, + "ttft_s_max": 2.1, + "timing_valid": true, + "end_to_end_tps": 88.00613725206625, + "makespan_s": 23.27110431098845, + "steady_tps": 93.6, + "alone": { + "equal": 8, + "unequal": 0, + "failed": 0 + }, + "serial": { + "equal": 1, + "unequal": 0, + "failed": 0 + }, + "reps": [ + { + "failed": 0, + "unmeasured": 0, + "aggregate_tps": 88.28227403744755, + "requested_requests": 8, + "timed_requests": 8, + "makespan_s": 23.27110431098845, + "end_to_end_tps": 88.00613725206625, + "timing_valid": true, + "steady_tps": 93.58379025005469, + "per_stream_tps": [ + 11.8, + 11.6, + 11.5, + 11.9, + 12.0, + 11.4, + 11.4, + 11.6 + ], + "ttft_s": [ + 1.45, + 1.16, + 0.62, + 1.77, + 2.1, + 0.38, + 0.16, + 0.88 + ], + "errors": [], + "evidence": [ + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + }, + { + "prompt_tokens": 32, + "cached_tokens": 0, + "tokens": 256, + "cache_state": "cold", + "complete": true, + "token_sha": "e7870fdfe932", + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb" + } + ], + "client_intervals": [ + { + "start": 752849.829924092, + "first": 752851.282089683, + "last": 752872.982721952, + "end": 752872.982850643, + "elapsed_s": 23.152926550945267, + "chunk_offsets_s": [ + 1.4521655909484252, + 1.6086418069899082, + 1.9370057579362765, + 2.2813175519695505, + 2.471366642974317, + 2.661105142906308, + 2.8543692579260096, + 3.047647214960307, + 3.2351691279327497, + 3.419232935993932, + 3.606029371963814, + 3.802151559968479, + 3.9965520749101415, + 4.19461633998435, + 4.394945684005506, + 4.5969771929085255, + 4.799951404915191, + 5.004303812980652, + 5.205739834927954, + 5.406394123914652, + 5.608264309936203, + 5.810165363945998, + 6.0103179609868675, + 6.211619941983372, + 6.412885842961259, + 6.614707459928468, + 6.8130517499521375, + 7.012533603934571, + 7.216127001913264, + 7.420954369008541, + 7.621030761976726, + 7.82481795491185, + 8.021827963995747, + 8.21668008598499, + 8.407769427984022, + 8.599992397008464, + 8.791339042945765, + 8.976141901919618, + 9.16656509891618, + 9.358089056913741, + 9.551665498991497, + 9.753271849942394, + 9.949606945971027, + 10.150451419991441, + 10.351066791918129, + 10.553195541957393, + 10.755761221982539, + 10.954610873945057, + 11.152101709973067, + 11.349058858002536, + 11.548570642946288, + 11.747191686998121, + 11.942210034001619, + 12.137219343916513, + 12.336914054933004, + 12.53625847294461, + 12.735249207005836, + 12.934788150945678, + 13.135535774985328, + 13.333292937953956, + 13.529598605935462, + 13.725901074009016, + 13.91985493793618, + 14.121694365981966, + 14.320424075936899, + 14.520511810900643, + 14.716798697947524, + 14.910693358979188, + 15.108178554917686, + 15.306959156994708, + 15.509786434005946, + 15.707059758948162, + 15.904619205975905, + 16.102087062899955, + 16.302360815927386, + 16.501176788005978, + 16.698921434930526, + 16.895248530898243, + 17.090722811990418, + 17.28796580794733, + 17.482532781898044, + 17.674176656990312, + 17.873870137962513, + 18.07963109598495, + 18.28526733594481, + 18.4917059619911, + 18.69803522794973, + 18.902809771941975, + 19.101147101959214, + 19.29379366990179, + 19.486777613987215, + 19.679299171897583, + 19.873666387982666, + 20.06745500897523, + 20.255428795004264, + 20.444192732917145, + 20.63738510000985, + 20.833549592993222, + 21.031968825962394, + 21.23055106389802, + 21.42442374699749, + 21.612283950904384, + 21.802110340911895, + 21.988901334931143, + 22.17179932491854, + 22.348984732991084, + 22.520881544915028, + 22.678314109914936, + 22.821742255007848, + 22.950907015008852, + 23.062559267971665, + 23.152797859977 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.46785692404956, + "p95": 205.63623995985836, + "max": 344.31179403327405 + } + }, + { + "start": 752849.827178096, + "first": 752850.987401885, + "last": 752872.892433704, + "end": 752872.892557786, + "elapsed_s": 23.065379689913243, + "chunk_offsets_s": [ + 1.1602237889310345, + 1.2986246969085187, + 1.6111340839415789, + 1.9391372109530494, + 2.2837232869351283, + 2.4737752749351785, + 2.6630069609964266, + 2.856703525991179, + 3.0503421309404075, + 3.2377200059127063, + 3.421897111926228, + 3.6088145419489592, + 3.804294903995469, + 3.999335292959586, + 4.197041710955091, + 4.397760938969441, + 4.599683470907621, + 4.802733263000846, + 5.0067299999063835, + 5.208266148925759, + 5.40939854294993, + 5.610416038893163, + 5.812557402998209, + 6.013219579937868, + 6.213604622986168, + 6.415454035974108, + 6.617621945915744, + 6.815954025951214, + 7.015074192895554, + 7.21853624295909, + 7.423472299007699, + 7.623446131008677, + 7.827003497979604, + 8.024507277994417, + 8.218682306003757, + 8.410047945915721, + 8.602361616911367, + 8.793700484908186, + 8.979036559932865, + 9.169503459939733, + 9.360915272962302, + 9.554211081005633, + 9.755448000971228, + 9.951701772981323, + 10.152219829964451, + 10.352936337003484, + 10.555392895941623, + 10.757989299949259, + 10.956859130994417, + 11.154344718903303, + 11.351107885944657, + 11.550810386892408, + 11.749340089969337, + 11.94404771795962, + 12.139179975958541, + 12.339338450925425, + 12.53869205492083, + 12.737569411983714, + 12.93705628393218, + 13.13794636190869, + 13.335628777975217, + 13.532252572942525, + 13.72793221194297, + 13.922161972965114, + 14.124157759943046, + 14.32282130094245, + 14.522716190898791, + 14.719054893008433, + 14.913016988895833, + 15.110329999937676, + 15.30934538890142, + 15.512072041979991, + 15.70946909696795, + 15.907319114892744, + 16.104869399918243, + 16.305010846001096, + 16.50381934491452, + 16.701632481999695, + 16.898025491973385, + 17.093429729924537, + 17.290639168932103, + 17.485329248942435, + 17.67687077296432, + 17.876320122973993, + 18.08234070194885, + 18.28841284592636, + 18.494323011953384, + 18.700986549956724, + 18.905595949967392, + 19.103931903955527, + 19.296324866940267, + 19.48959318792913, + 19.682208710932173, + 19.876453765900806, + 20.070113136898726, + 20.258328812895343, + 20.44704208697658, + 20.64028860698454, + 20.83621352899354, + 21.03463225089945, + 21.233687771949917, + 21.42727246193681, + 21.61517649597954, + 21.805068350979127, + 21.991542227915488, + 22.174872646923177, + 22.3515777469147, + 22.523487728904, + 22.681812677998096, + 22.824520415975712, + 22.953613900928758, + 23.065255607943982 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.50378001481295, + "p95": 206.02057897485793, + "max": 344.5860759820789 + } + }, + { + "start": 752849.827648745, + "first": 752850.448035433, + "last": 752872.651607843, + "end": 752872.651712866, + "elapsed_s": 22.824064120999537, + "chunk_offsets_s": [ + 0.6203866879222915, + 0.7278667959617451, + 1.0027148999506608, + 1.2984565430087969, + 1.6107198909157887, + 1.9388523820089176, + 2.283180019003339, + 2.4730544119374827, + 2.662341689923778, + 2.856375698931515, + 3.0493470809888095, + 3.237028458970599, + 3.4210849379887804, + 3.608034549979493, + 3.803571432013996, + 3.998377352952957, + 4.196387625997886, + 4.396556016989052, + 4.598817512975074, + 4.8018561999779195, + 5.006040885928087, + 5.207516946014948, + 5.408257503993809, + 5.60962182097137, + 5.811706903972663, + 6.012129524955526, + 6.2132011039648205, + 6.414550504996441, + 6.616637890925631, + 6.814874387928285, + 7.014130927971564, + 7.218172953929752, + 7.423084173933603, + 7.622920545982197, + 7.826706187915988, + 8.02387681195978, + 8.218682351987809, + 8.409627735964023, + 8.601842224015854, + 8.793293636990711, + 8.979133324930444, + 9.168912232969888, + 9.359939332003705, + 9.55362913501449, + 9.75550144596491, + 9.951682230923325, + 10.152436563977972, + 10.353442266001366, + 10.555574905010872, + 10.75770576694049, + 10.957111101946793, + 11.154519318020903, + 11.351381827960722, + 11.55074016598519, + 11.7489303619368, + 11.944572433945723, + 12.139202455990016, + 12.339326445944607, + 12.538128559943289, + 12.73741997894831, + 12.937014690949582, + 13.13790380791761, + 13.33536374301184, + 13.53191128699109, + 13.727758726920001, + 13.921914268983528, + 14.123568677925505, + 14.322581293992698, + 14.522758177947253, + 14.719001489924267, + 14.912790839909576, + 15.110301975975744, + 15.309086547000334, + 15.511748774908483, + 15.709142325911671, + 15.907164335949346, + 16.104433252010494, + 16.3047082539415, + 16.50319281697739, + 16.701241348986514, + 16.897582015022635, + 17.092838151962496, + 17.29020279692486, + 17.484641943941824, + 17.676542512956075, + 17.87650902592577, + 18.081953026005067, + 18.28763837693259, + 18.49394592794124, + 18.700347316917032, + 18.905052202986553, + 19.103390651987866, + 19.296030770987272, + 19.489087413996458, + 19.681537376949564, + 19.876143332920037, + 20.06984826095868, + 20.25738947291393, + 20.44617843790911, + 20.639372869976796, + 20.835863329004496, + 21.03395421197638, + 21.233033943921328, + 21.42634944396559, + 21.614063450018875, + 21.80443993501831, + 21.991459409939125, + 22.1744359069271, + 22.35085738298949, + 22.522194674005732, + 22.681565484963357, + 22.82395909796469 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.51113606616855, + "p95": 206.40138897579163, + "max": 344.3276369944215 + } + }, + { + "start": 752849.828773048, + "first": 752851.594099601, + "last": 752873.049105755, + "end": 752873.049205049, + "elapsed_s": 23.22043200104963, + "chunk_offsets_s": [ + 1.7653265530243516, + 1.9384529589442536, + 2.2831760810222477, + 2.473179101012647, + 2.6618188549764454, + 2.8548938690219074, + 3.0484867229824886, + 3.236225388944149, + 3.4205343070207164, + 3.6073644290445372, + 3.803371798945591, + 3.9976705219596624, + 4.195805229945108, + 4.396046336041763, + 4.598400278016925, + 4.8011750529985875, + 5.005496334983036, + 5.2069281650474295, + 5.407056324998848, + 5.609007467981428, + 5.810632816050202, + 6.01087145798374, + 6.212280175997876, + 6.413551148958504, + 6.6154010749887675, + 6.8135819640010595, + 7.012855881941505, + 7.216856622020714, + 7.4216976390453056, + 7.621729993028566, + 7.8256408859742805, + 8.02264247694984, + 8.217226542998105, + 8.408555425005034, + 8.600971751962788, + 8.792330862954259, + 8.97737470199354, + 9.168016892042942, + 9.35829359095078, + 9.552258571027778, + 9.75392515596468, + 9.950833345996216, + 10.151386784040369, + 10.352009580936283, + 10.55411508004181, + 10.756823276984505, + 10.956029301974922, + 11.153303817030974, + 11.349925954942591, + 11.54950781504158, + 11.748104406986386, + 11.943152950960211, + 12.138274517958052, + 12.338239892967977, + 12.537553266971372, + 12.736136691994034, + 12.935550017049536, + 13.136742411996238, + 13.334280533948913, + 13.53082380502019, + 13.726827317033894, + 13.921039427048527, + 14.122889111982659, + 14.321495397016406, + 14.521601406042464, + 14.717911959975027, + 14.91187848802656, + 15.109109231038019, + 15.30814198195003, + 15.510841559967957, + 15.7080647329567, + 15.905999275040813, + 16.103342378046364, + 16.30354581703432, + 16.50227023998741, + 16.700151931028813, + 16.89649610198103, + 17.09179971704725, + 17.289152920944616, + 17.483635514974594, + 17.675456119934097, + 17.874813087983057, + 18.080134044983424, + 18.285595797002316, + 18.49235906195827, + 18.698605675948784, + 18.903430419042706, + 19.101599356974475, + 19.29445200203918, + 19.487321609980427, + 19.679818507982418, + 19.87476993794553, + 20.06821894098539, + 20.256380305974744, + 20.444772206014022, + 20.63805666693952, + 20.8344508089358, + 21.03259038599208, + 21.231312516028993, + 21.42494297202211, + 21.613010582048446, + 21.802752607967705, + 21.98934442095924, + 22.172452121041715, + 22.34912023996003, + 22.520874663023278, + 22.67974937101826, + 22.822455637040548, + 22.951939897960983, + 23.06352468498517, + 23.153873582021333, + 23.2203327070456 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.2307430114597, + "p95": 204.84101702459157, + "max": 344.72312207799405 + } + }, + { + "start": 752849.827896128, + "first": 752851.923573751, + "last": 752873.097625858, + "end": 752873.097736994, + "elapsed_s": 23.269840866094455, + "chunk_offsets_s": [ + 2.0956776230596006, + 2.283427817048505, + 2.473597326083109, + 2.6633537330199033, + 2.8566046140622348, + 3.0497137600323185, + 3.2371590060647577, + 3.421477782074362, + 3.6078141150064766, + 3.8039603860815987, + 3.998668485088274, + 4.196717755054124, + 4.397100660018623, + 4.599052684032358, + 4.801886795088649, + 5.006298268097453, + 5.207840226008557, + 5.408537897048518, + 5.6101382341003045, + 5.812320836004801, + 6.012539538089186, + 6.213458229089156, + 6.4145274290349334, + 6.616975845070556, + 6.814918697113171, + 7.014594709035009, + 7.218202493037097, + 7.422929557040334, + 7.623011630028486, + 7.826919962069951, + 8.024085899000056, + 8.21876164211426, + 8.409760891110636, + 8.602058911114, + 8.793405365082435, + 8.978689224109985, + 9.169294304098003, + 9.359898828086443, + 9.553424158017151, + 9.754990216111764, + 9.95134249900002, + 10.151873085065745, + 10.352704874007031, + 10.555065979016945, + 10.757668208098039, + 10.95669620600529, + 11.1540463170968, + 11.350979494047351, + 11.550031034043059, + 11.749096416053362, + 11.943974471068941, + 12.13879631308373, + 12.338408964104019, + 12.53809450508561, + 12.736581564066, + 12.93610394501593, + 13.137145405053161, + 13.334716957062483, + 13.531215021037497, + 13.726806936087087, + 13.921487996005453, + 14.122970971046016, + 14.321931596030481, + 14.521887567010708, + 14.718154627014883, + 14.912129396107048, + 15.109409090015106, + 15.308284410042688, + 15.511062176083215, + 15.708840439096093, + 15.906516223098151, + 16.103669521049596, + 16.303941418998875, + 16.502697510062717, + 16.70071426103823, + 16.896944591077045, + 17.092327610007487, + 17.289371105027385, + 17.484098934102803, + 17.675920516019687, + 17.876459416002035, + 18.080945371999405, + 18.287480143015273, + 18.493485586019233, + 18.699772862019017, + 18.90450280799996, + 19.10275433305651, + 19.29546159808524, + 19.488531250040978, + 19.680853563011624, + 19.875846614013426, + 20.068950928049162, + 20.25687803607434, + 20.44570017408114, + 20.63876554707531, + 20.83514488709625, + 21.03358458704315, + 21.232237571035512, + 21.4259516550228, + 21.613714053062722, + 21.803548092022538, + 21.990074568078853, + 22.17325882310979, + 22.349898970103823, + 22.52230255806353, + 22.680859908112325, + 22.823474097065628, + 22.952759786043316, + 23.064305959036574, + 23.154689549002796, + 23.221167893032543, + 23.269729730091058 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.15329795144498, + "p95": 204.48595599737018, + "max": 206.5347710158676 + } + }, + { + "start": 752849.826991187, + "first": 752850.208667093, + "last": 752872.50760816, + "end": 752872.50769358, + "elapsed_s": 22.68070239305962, + "chunk_offsets_s": [ + 0.3816759060136974, + 0.46528824500273913, + 0.7282417500391603, + 1.0031421181047335, + 1.2990004990715533, + 1.6118404970038682, + 1.939617493073456, + 2.2847584550036117, + 2.4747530421009287, + 2.663073147065006, + 2.8563417110126466, + 3.0499510790687054, + 3.2374943390022963, + 3.421671189018525, + 3.608449111110531, + 3.8041817831108347, + 3.99913028604351, + 4.196991304052062, + 4.397364735021256, + 4.599800737109035, + 4.8026302721118554, + 5.00663264200557, + 5.208265590015799, + 5.4092284220969304, + 5.610964315012097, + 5.812303364044055, + 6.0128302740631625, + 6.214007276110351, + 6.41597051511053, + 6.61707103101071, + 6.81562034401577, + 7.014860833063722, + 7.218779353075661, + 7.423574427026324, + 7.623718829010613, + 7.827528187073767, + 8.024635522044264, + 8.21955791010987, + 8.410489229019731, + 8.60269424400758, + 8.794049370102584, + 8.979546094080433, + 9.169746425002813, + 9.360184908029623, + 9.554141487111337, + 9.75599214702379, + 9.95242849108763, + 10.152197604067624, + 10.353366466006264, + 10.55552134802565, + 10.758134459028952, + 10.957095840014517, + 11.154467858024873, + 11.351243699085899, + 11.550694611039944, + 11.749328697100282, + 11.944514973089099, + 12.139425725094043, + 12.339608652051538, + 12.539282272104174, + 12.737815898028202, + 12.937201555003412, + 13.138210882083513, + 13.33584994811099, + 13.532329673063941, + 13.727659324998967, + 13.922281560022384, + 14.124466591048986, + 14.323115858016536, + 14.522959109046496, + 14.719196612015367, + 14.91311223700177, + 15.110407965024933, + 15.309410664020106, + 15.5121697060531, + 15.709940320113674, + 15.907310411101207, + 16.10474335111212, + 16.30510124505963, + 16.50370590609964, + 16.70147915009875, + 16.897927911020815, + 17.093179968069308, + 17.29073668899946, + 17.485157994087785, + 17.676920334110036, + 17.876935847103596, + 18.082019425113685, + 18.287671365076676, + 18.494283744017594, + 18.70053667901084, + 18.905453451094218, + 19.103767082095146, + 19.296307682059705, + 19.489273592014797, + 19.68194461101666, + 19.87670118000824, + 20.07017431501299, + 20.257901234086603, + 20.446706889080815, + 20.640091237029992, + 20.836330284015276, + 21.034531902056187, + 21.233265889110044, + 21.426891273004003, + 21.614944865112193, + 21.80465466203168, + 21.991231529042125, + 22.174364305101335, + 22.35145834099967, + 22.523045572103, + 22.680616973084398 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.77061406057328, + "p95": 262.9535050364211, + "max": 345.14096193015575 + } + }, + { + "start": 752849.826765424, + "first": 752849.989929299, + "last": 752872.350258247, + "end": 752872.350341602, + "elapsed_s": 22.523576178005897, + "chunk_offsets_s": [ + 0.16316387499682605, + 0.2248016189550981, + 0.46555706998333335, + 0.7284236659761518, + 1.003401005989872, + 1.2988924820674583, + 1.6114556060638279, + 1.9403047029627487, + 2.2839809759752825, + 2.4741333950078115, + 2.664726361981593, + 2.8567611509934068, + 3.0504217910347506, + 3.238581418991089, + 3.422833399963565, + 3.608581659034826, + 3.8052598839858547, + 3.9999887860612944, + 4.197718310984783, + 4.397945928038098, + 4.600550420000218, + 4.802929581957869, + 5.007547085988335, + 5.208852640003897, + 5.4093316060025245, + 5.610914387973025, + 5.813018865999766, + 6.013298745965585, + 6.214435965986922, + 6.416088534053415, + 6.6178152890643105, + 6.815972301992588, + 7.015301364008337, + 7.219221564009786, + 7.42401771096047, + 7.624062450020574, + 7.827870207023807, + 8.025025533977896, + 8.219325555954129, + 8.410832433961332, + 8.603039912995882, + 8.794421634986065, + 8.979723274009302, + 9.169838952017017, + 9.361089876969345, + 9.554671552032232, + 9.75617331196554, + 9.952361889998429, + 10.152726292028092, + 10.353465746040456, + 10.556161829037592, + 10.758764459053054, + 10.957427314016968, + 11.15483244205825, + 11.35184322996065, + 11.550797363976017, + 11.749657099018805, + 11.944989413022995, + 12.139867200050503, + 12.339960130047984, + 12.539455499034375, + 12.738109127036296, + 12.937796093057841, + 13.138564439956099, + 13.336191743961535, + 13.532867338974029, + 13.728284651064314, + 13.923078080057167, + 14.124632089049555, + 14.323541537974961, + 14.523465425008908, + 14.719721939065494, + 14.913708294043317, + 15.110897095059045, + 15.30992376501672, + 15.512594426982105, + 15.710097528994083, + 15.90796257299371, + 16.10516551102046, + 16.30545758700464, + 16.50404048501514, + 16.701962133985944, + 16.898323139059357, + 17.093921134015545, + 17.29077658499591, + 17.485590139054693, + 17.677386116003618, + 17.877075036987662, + 18.082498072995804, + 18.288229160010815, + 18.49477544904221, + 18.701018815045245, + 18.905747879995033, + 19.104210732039064, + 19.296878039021976, + 19.48973210901022, + 19.682382164988667, + 19.876713423989713, + 20.070694105001166, + 20.25830819096882, + 20.44714213802945, + 20.640377777046524, + 20.836659390013665, + 21.035092929028906, + 21.233566351002082, + 21.427353149978444, + 21.61509231198579, + 21.80512831604574, + 21.991761127952486, + 22.174790833028965, + 22.351615422056057, + 22.523492823005654 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.86504399962723, + "p95": 262.8665959928185, + "max": 343.6762730125338 + } + }, + { + "start": 752849.826632683, + "first": 752850.711084345, + "last": 752872.780425913, + "end": 752872.780497027, + "elapsed_s": 22.953864343930036, + "chunk_offsets_s": [ + 0.8844516619574279, + 1.003658405970782, + 1.2996340699028224, + 1.6120356479659677, + 1.9396097479620948, + 2.2839628289220855, + 2.474032227997668, + 2.663281307904981, + 2.856644045910798, + 3.0508305749390274, + 3.237945497967303, + 3.42188274289947, + 3.609152867924422, + 3.804664449999109, + 3.999303689925, + 4.197286341921426, + 4.39762330299709, + 4.599979292950593, + 4.802932414924726, + 5.007124622003175, + 5.2084910499397665, + 5.409743717988022, + 5.610846842988394, + 5.813552803942002, + 6.013502600952052, + 6.2150439789984375, + 6.4164981009671465, + 6.618429942987859, + 6.816656837938353, + 7.015898383921012, + 7.219552030903287, + 7.423741845996119, + 7.62423516495619, + 7.828142360900529, + 8.02519123989623, + 8.220135278999805, + 8.41077516193036, + 8.60324452095665, + 8.794592493912205, + 8.97936130000744, + 9.170331843895838, + 9.36073608091101, + 9.554154882906005, + 9.755923818913288, + 9.95238010189496, + 10.152459020959213, + 10.353675986989401, + 10.555824435898103, + 10.758450508932583, + 10.957728208974004, + 11.154721706989221, + 11.351545138983056, + 11.55170802900102, + 11.74931301700417, + 11.944512862944975, + 12.139656385988928, + 12.339803131995723, + 12.539057408925146, + 12.738015801995061, + 12.937505218898878, + 13.13824801100418, + 13.336122582899407, + 13.532640201970935, + 13.728904343908653, + 13.922988099977374, + 14.124339119996876, + 14.323308064951561, + 14.523213565000333, + 14.719474879908375, + 14.91352438100148, + 15.11082079692278, + 15.309652494965121, + 15.512477882904932, + 15.709866120945662, + 15.907992946915329, + 16.10505714500323, + 16.30537475994788, + 16.50428407394793, + 16.701886380906217, + 16.898248505895026, + 17.093624659930356, + 17.29099047393538, + 17.485431844950654, + 17.677348864963278, + 17.877354934928007, + 18.082326898002066, + 18.28885941300541, + 18.49456497200299, + 18.70093629299663, + 18.90570640389342, + 19.10394927195739, + 19.296943985973485, + 19.48972532490734, + 19.682183914002962, + 19.876741669955663, + 20.070432644919492, + 20.258300095913, + 20.447013273951598, + 20.64023807796184, + 20.836533869965933, + 21.034800600958988, + 21.233766253921203, + 21.42703272192739, + 21.615517894970253, + 21.804948948905803, + 21.991652713972144, + 22.17467344191391, + 22.351971538970247, + 22.52329948497936, + 22.682284015929326, + 22.82479714998044, + 22.953793230000883 + ], + "chunk_gaps_ms": { + "count": 111, + "p50": 197.38823804073036, + "p95": 206.37132099363953, + "max": 344.3530809599906 + } + } + ], + "equal_alone": [ + true, + true, + true, + true, + true, + true, + true, + true + ], + "comparisons": [ + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + }, + { + "concurrent": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "alone": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + }, + "serial": { + "tokens": 256, + "token_sha256": "e7870fdfe932aa09fa6a8ce2e44960ba54a3660fab4a8cea9473338d0b6c5807", + "output_sha256": "a85d7da2ec07eb0535b013466ad6b6698e3bd71a6949b7023192259b290d91cb", + "complete": true, + "error": null + } + } + ] + } + ] + } + ] +} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/hostwatch-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/hostwatch-rank0.log new file mode 100644 index 0000000000..883b1e9a70 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/hostwatch-rank0.log @@ -0,0 +1,663 @@ +10:23:13 114 +10:23:15 114 +10:23:17 111 +10:23:19 111 +10:23:21 110 +10:23:23 110 +10:23:25 110 +10:23:27 110 +10:23:29 108 +10:23:31 108 +10:23:33 107 +10:23:35 107 +10:23:37 106 +10:23:39 105 +10:23:42 105 +10:23:44 104 +10:23:46 104 +10:23:48 103 +10:23:50 102 +10:23:52 102 +10:23:54 101 +10:23:56 101 +10:23:58 100 +10:24:00 100 +10:24:02 100 +10:24:04 98 +10:24:06 98 +10:24:08 98 +10:24:10 96 +10:24:12 97 +10:24:14 95 +10:24:16 95 +10:24:19 95 +10:24:21 94 +10:24:23 94 +10:24:25 93 +10:24:27 92 +10:24:29 92 +10:24:31 91 +10:24:33 91 +10:24:35 90 +10:24:37 90 +10:24:39 90 +10:24:41 88 +10:24:43 88 +10:24:45 88 +10:24:47 86 +10:24:49 86 +10:24:51 86 +10:24:53 85 +10:24:55 84 +10:24:58 84 +10:25:00 84 +10:25:02 84 +10:25:04 82 +10:25:06 82 +10:25:08 81 +10:25:10 81 +10:25:12 80 +10:25:14 80 +10:25:16 80 +10:25:18 78 +10:25:20 78 +10:25:22 78 +10:25:24 77 +10:25:26 77 +10:25:28 76 +10:25:30 75 +10:25:32 75 +10:25:35 74 +10:25:37 74 +10:25:39 74 +10:25:41 73 +10:25:43 73 +10:25:45 71 +10:25:47 71 +10:25:49 70 +10:25:51 70 +10:25:53 70 +10:25:55 70 +10:25:57 68 +10:25:59 68 +10:26:01 67 +10:26:03 67 +10:26:05 67 +10:26:07 66 +10:26:09 66 +10:26:11 65 +10:26:14 64 +10:26:16 64 +10:26:18 63 +10:26:20 63 +10:26:22 63 +10:26:24 61 +10:26:26 61 +10:26:28 60 +10:26:30 60 +10:26:32 60 +10:26:34 59 +10:26:36 58 +10:26:38 58 +10:26:40 57 +10:26:42 57 +10:26:44 55 +10:26:46 55 +10:26:48 55 +10:26:51 54 +10:26:53 54 +10:26:55 53 +10:26:57 52 +10:26:59 50 +10:27:01 50 +10:27:03 50 +10:27:05 50 +10:27:07 50 +10:27:09 50 +10:27:11 11 +10:27:13 11 +10:27:15 11 +10:27:17 11 +10:27:19 11 +10:27:21 11 +10:27:23 11 +10:27:26 11 +10:27:28 11 +10:27:30 11 +10:27:32 11 +10:27:34 12 +10:27:36 12 +10:27:38 12 +10:27:40 12 +10:27:42 12 +10:27:44 12 +10:27:46 12 +10:27:48 12 +10:27:50 12 +10:27:52 12 +10:27:54 12 +10:27:57 12 +10:27:59 12 +10:28:01 12 +10:28:03 12 +10:28:05 12 +10:28:07 12 +10:28:09 12 +10:28:11 12 +10:28:13 12 +10:28:15 12 +10:28:17 12 +10:28:19 12 +10:28:21 12 +10:28:23 12 +10:28:25 12 +10:28:28 12 +10:28:30 12 +10:28:32 12 +10:28:34 12 +10:28:36 12 +10:28:38 12 +10:28:40 12 +10:28:42 12 +10:28:44 12 +10:28:46 12 +10:28:48 12 +10:28:50 12 +10:28:53 12 +10:28:55 12 +10:28:57 12 +10:28:59 12 +10:29:01 12 +10:29:03 12 +10:29:05 12 +10:29:07 12 +10:29:09 12 +10:29:11 12 +10:29:13 12 +10:29:15 12 +10:29:17 12 +10:29:19 12 +10:29:21 12 +10:29:24 12 +10:29:26 12 +10:29:28 12 +10:29:30 12 +10:29:32 12 +10:29:34 12 +10:29:36 12 +10:29:38 12 +10:29:40 12 +10:29:42 12 +10:29:44 12 +10:29:46 12 +10:29:48 12 +10:29:50 12 +10:29:52 12 +10:29:55 12 +10:29:57 12 +10:29:59 12 +10:30:01 12 +10:30:03 12 +10:30:05 12 +10:30:07 12 +10:30:09 12 +10:30:11 12 +10:30:13 12 +10:30:15 12 +10:30:17 12 +10:30:19 12 +10:30:21 12 +10:30:23 12 +10:30:26 12 +10:30:28 12 +10:30:30 12 +10:30:32 12 +10:30:34 12 +10:30:36 12 +10:30:38 12 +10:30:40 12 +10:30:42 12 +10:30:44 12 +10:30:46 12 +10:30:48 12 +10:30:50 12 +10:30:52 12 +10:30:55 12 +10:30:57 12 +10:30:59 12 +10:31:01 12 +10:31:03 12 +10:31:05 12 +10:31:07 12 +10:31:09 12 +10:31:11 12 +10:31:13 12 +10:31:15 12 +10:31:17 12 +10:31:19 12 +10:31:21 12 +10:31:24 12 +10:31:26 12 +10:31:28 12 +10:31:30 12 +10:31:32 12 +10:31:34 12 +10:31:36 12 +10:31:38 12 +10:31:40 12 +10:31:42 12 +10:31:44 12 +10:31:46 12 +10:31:48 12 +10:31:50 12 +10:31:52 12 +10:31:55 12 +10:31:57 12 +10:31:59 12 +10:32:01 12 +10:32:03 12 +10:32:05 12 +10:32:07 12 +10:32:09 12 +10:32:11 12 +10:32:13 12 +10:32:15 12 +10:32:17 12 +10:32:19 12 +10:32:21 12 +10:32:23 12 +10:32:26 12 +10:32:28 12 +10:32:30 12 +10:32:32 12 +10:32:34 12 +10:32:36 12 +10:32:38 12 +10:32:40 12 +10:32:42 12 +10:32:44 12 +10:32:46 12 +10:32:48 12 +10:32:50 12 +10:32:52 12 +10:32:54 12 +10:32:57 12 +10:32:59 12 +10:33:01 12 +10:33:03 12 +10:33:05 12 +10:33:07 12 +10:33:09 12 +10:33:11 12 +10:33:13 12 +10:33:15 12 +10:33:17 12 +10:33:19 12 +10:33:21 12 +10:33:23 12 +10:33:25 12 +10:33:28 12 +10:33:30 12 +10:33:32 12 +10:33:34 12 +10:33:36 12 +10:33:38 12 +10:33:40 12 +10:33:42 12 +10:33:44 12 +10:33:46 12 +10:33:48 12 +10:33:50 12 +10:33:52 12 +10:33:54 12 +10:33:56 12 +10:33:59 12 +10:34:01 12 +10:34:03 12 +10:34:05 12 +10:34:07 12 +10:34:09 12 +10:34:11 12 +10:34:13 12 +10:34:15 12 +10:34:17 12 +10:34:19 12 +10:34:21 12 +10:34:23 12 +10:34:25 12 +10:34:27 12 +10:34:30 12 +10:34:32 12 +10:34:34 12 +10:34:36 12 +10:34:38 12 +10:34:40 12 +10:34:42 12 +10:34:44 12 +10:34:46 12 +10:34:48 12 +10:34:50 12 +10:34:52 12 +10:34:54 12 +10:34:56 12 +10:34:58 12 +10:35:01 12 +10:35:03 12 +10:35:05 12 +10:35:07 12 +10:35:09 12 +10:35:11 12 +10:35:13 12 +10:35:15 12 +10:35:17 12 +10:35:19 12 +10:35:21 12 +10:35:23 12 +10:35:25 12 +10:35:27 12 +10:35:30 12 +10:35:32 12 +10:35:34 12 +10:35:36 12 +10:35:38 12 +10:35:40 12 +10:35:42 12 +10:35:44 12 +10:35:46 12 +10:35:48 12 +10:35:50 12 +10:35:52 12 +10:35:54 12 +10:35:56 12 +10:35:58 12 +10:36:01 12 +10:36:03 12 +10:36:05 12 +10:36:07 12 +10:36:09 12 +10:36:11 12 +10:36:13 12 +10:36:15 12 +10:36:17 12 +10:36:19 12 +10:36:21 12 +10:36:23 12 +10:36:25 12 +10:36:27 12 +10:36:29 12 +10:36:31 12 +10:36:34 12 +10:36:36 12 +10:36:38 12 +10:36:40 12 +10:36:42 12 +10:36:44 12 +10:36:46 12 +10:36:48 12 +10:36:50 12 +10:36:52 12 +10:36:54 12 +10:36:56 12 +10:36:58 12 +10:37:00 12 +10:37:02 12 +10:37:05 12 +10:37:07 12 +10:37:09 12 +10:37:11 12 +10:37:13 12 +10:37:15 12 +10:37:17 12 +10:37:19 12 +10:37:21 12 +10:37:23 12 +10:37:25 12 +10:37:27 12 +10:37:29 12 +10:37:31 12 +10:37:33 12 +10:37:36 12 +10:37:38 12 +10:37:40 12 +10:37:42 12 +10:37:44 12 +10:37:46 12 +10:37:48 12 +10:37:50 12 +10:37:52 12 +10:37:54 12 +10:37:56 12 +10:37:58 12 +10:38:00 12 +10:38:02 12 +10:38:04 12 +10:38:06 12 +10:38:09 12 +10:38:11 12 +10:38:13 12 +10:38:15 12 +10:38:17 12 +10:38:19 12 +10:38:21 12 +10:38:23 12 +10:38:25 12 +10:38:27 12 +10:38:29 12 +10:38:31 12 +10:38:33 12 +10:38:35 12 +10:38:37 12 +10:38:40 12 +10:38:42 12 +10:38:44 12 +10:38:46 12 +10:38:48 12 +10:38:50 12 +10:38:52 12 +10:38:54 12 +10:38:56 12 +10:38:58 12 +10:39:00 12 +10:39:02 12 +10:39:04 12 +10:39:06 12 +10:39:08 12 +10:39:11 12 +10:39:13 12 +10:39:15 12 +10:39:17 12 +10:39:19 12 +10:39:21 12 +10:39:23 12 +10:39:25 12 +10:39:27 12 +10:39:29 12 +10:39:31 12 +10:39:33 12 +10:39:35 12 +10:39:37 12 +10:39:39 12 +10:39:41 12 +10:39:43 12 +10:39:46 12 +10:39:48 12 +10:39:50 12 +10:39:52 12 +10:39:54 12 +10:39:56 12 +10:39:58 12 +10:40:00 12 +10:40:02 12 +10:40:04 12 +10:40:06 12 +10:40:08 12 +10:40:10 12 +10:40:12 12 +10:40:14 12 +10:40:17 12 +10:40:19 12 +10:40:21 12 +10:40:23 12 +10:40:25 12 +10:40:27 12 +10:40:29 12 +10:40:31 12 +10:40:33 12 +10:40:35 12 +10:40:37 12 +10:40:39 12 +10:40:41 12 +10:40:43 12 +10:40:45 12 +10:40:48 12 +10:40:50 12 +10:40:52 12 +10:40:54 12 +10:40:56 12 +10:40:58 12 +10:41:00 12 +10:41:02 12 +10:41:04 12 +10:41:06 12 +10:41:08 12 +10:41:10 12 +10:41:12 12 +10:41:14 12 +10:41:16 12 +10:41:18 12 +10:41:20 12 +10:41:23 12 +10:41:25 12 +10:41:27 12 +10:41:29 12 +10:41:31 12 +10:41:33 12 +10:41:35 12 +10:41:37 12 +10:41:39 12 +10:41:41 12 +10:41:43 12 +10:41:45 12 +10:41:47 12 +10:41:49 12 +10:41:51 12 +10:41:53 12 +10:41:55 12 +10:41:58 12 +10:42:00 12 +10:42:02 12 +10:42:04 12 +10:42:06 12 +10:42:08 12 +10:42:10 12 +10:42:12 12 +10:42:14 12 +10:42:16 12 +10:42:18 12 +10:42:20 12 +10:42:22 12 +10:42:24 12 +10:42:26 12 +10:42:28 12 +10:42:31 12 +10:42:33 12 +10:42:35 12 +10:42:37 12 +10:42:39 12 +10:42:41 12 +10:42:43 12 +10:42:45 12 +10:42:47 12 +10:42:49 12 +10:42:51 12 +10:42:53 12 +10:42:55 12 +10:42:57 12 +10:42:59 12 +10:43:01 12 +10:43:04 12 +10:43:06 12 +10:43:08 12 +10:43:10 12 +10:43:12 12 +10:43:14 12 +10:43:16 12 +10:43:18 12 +10:43:20 12 +10:43:22 12 +10:43:24 12 +10:43:26 12 +10:43:28 12 +10:43:30 12 +10:43:32 12 +10:43:34 12 +10:43:36 12 +10:43:39 12 +10:43:41 12 +10:43:43 12 +10:43:45 12 +10:43:47 12 +10:43:49 12 +10:43:51 12 +10:43:53 12 +10:43:55 12 +10:43:57 12 +10:43:59 12 +10:44:01 12 +10:44:03 12 +10:44:05 12 +10:44:07 12 +10:44:09 12 +10:44:11 12 +10:44:14 12 +10:44:16 12 +10:44:18 12 +10:44:20 12 +10:44:22 12 +10:44:24 12 +10:44:26 12 +10:44:28 12 +10:44:30 12 +10:44:32 12 +10:44:34 12 +10:44:36 12 +10:44:38 12 +10:44:40 12 +10:44:42 12 +10:44:44 12 +10:44:46 12 +10:44:49 12 +10:44:51 12 +10:44:53 12 +10:44:55 12 +10:44:57 12 +10:44:59 12 +10:45:01 12 +10:45:03 12 +10:45:05 12 +10:45:07 12 +10:45:09 12 +10:45:11 12 +10:45:13 12 +10:45:15 12 +10:45:17 12 +10:45:20 12 +10:45:22 12 +10:45:24 12 +10:45:26 12 +10:45:28 12 +10:45:30 12 +10:45:32 12 +10:45:34 12 +10:45:36 12 +10:45:38 12 +10:45:40 12 +10:45:42 12 +10:45:44 12 +10:45:46 12 +10:45:48 12 +10:45:51 12 +10:45:53 12 +10:45:55 12 +10:45:57 58 +10:45:59 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/hostwatch-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/hostwatch-rank1.log new file mode 100644 index 0000000000..dccaedfdd1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/hostwatch-rank1.log @@ -0,0 +1,662 @@ +10:23:13 114 +10:23:15 114 +10:23:17 112 +10:23:19 112 +10:23:22 111 +10:23:24 110 +10:23:26 110 +10:23:28 108 +10:23:30 108 +10:23:32 108 +10:23:34 107 +10:23:36 107 +10:23:38 107 +10:23:40 106 +10:23:42 106 +10:23:44 104 +10:23:46 104 +10:23:48 102 +10:23:50 102 +10:23:52 101 +10:23:54 101 +10:23:56 101 +10:23:59 101 +10:24:01 100 +10:24:03 99 +10:24:05 98 +10:24:07 98 +10:24:09 97 +10:24:11 97 +10:24:13 97 +10:24:15 95 +10:24:17 95 +10:24:19 95 +10:24:21 94 +10:24:23 94 +10:24:25 92 +10:24:27 92 +10:24:29 92 +10:24:31 91 +10:24:33 91 +10:24:36 90 +10:24:38 90 +10:24:40 89 +10:24:42 88 +10:24:44 88 +10:24:46 87 +10:24:48 87 +10:24:50 86 +10:24:52 85 +10:24:54 86 +10:24:56 84 +10:24:58 84 +10:25:00 84 +10:25:02 83 +10:25:04 83 +10:25:06 81 +10:25:08 81 +10:25:10 80 +10:25:12 80 +10:25:15 80 +10:25:17 79 +10:25:19 79 +10:25:21 77 +10:25:23 77 +10:25:25 77 +10:25:27 75 +10:25:29 76 +10:25:31 75 +10:25:33 73 +10:25:35 73 +10:25:37 72 +10:25:39 72 +10:25:41 72 +10:25:43 71 +10:25:45 71 +10:25:47 70 +10:25:49 70 +10:25:52 70 +10:25:54 69 +10:25:56 68 +10:25:58 68 +10:26:00 67 +10:26:02 67 +10:26:04 66 +10:26:06 65 +10:26:08 65 +10:26:10 64 +10:26:12 63 +10:26:14 63 +10:26:16 62 +10:26:18 62 +10:26:20 61 +10:26:22 61 +10:26:24 61 +10:26:26 59 +10:26:29 59 +10:26:31 59 +10:26:33 58 +10:26:35 58 +10:26:37 56 +10:26:39 56 +10:26:41 56 +10:26:43 55 +10:26:45 55 +10:26:47 55 +10:26:49 54 +10:26:51 54 +10:26:53 54 +10:26:55 50 +10:26:57 50 +10:26:59 50 +10:27:01 50 +10:27:03 50 +10:27:06 49 +10:27:08 49 +10:27:10 49 +10:27:12 11 +10:27:14 11 +10:27:16 11 +10:27:18 11 +10:27:20 11 +10:27:22 11 +10:27:24 11 +10:27:26 12 +10:27:28 12 +10:27:30 12 +10:27:32 12 +10:27:34 12 +10:27:36 12 +10:27:39 12 +10:27:41 12 +10:27:43 12 +10:27:45 12 +10:27:47 12 +10:27:49 12 +10:27:51 12 +10:27:53 12 +10:27:55 12 +10:27:57 12 +10:27:59 12 +10:28:01 12 +10:28:04 12 +10:28:06 12 +10:28:08 12 +10:28:10 12 +10:28:12 12 +10:28:14 12 +10:28:16 12 +10:28:18 12 +10:28:20 12 +10:28:22 12 +10:28:24 12 +10:28:27 12 +10:28:29 12 +10:28:31 12 +10:28:33 12 +10:28:35 12 +10:28:37 12 +10:28:39 12 +10:28:41 12 +10:28:43 12 +10:28:45 12 +10:28:47 12 +10:28:49 12 +10:28:52 12 +10:28:54 12 +10:28:56 12 +10:28:58 12 +10:29:00 12 +10:29:02 12 +10:29:04 12 +10:29:06 12 +10:29:08 12 +10:29:10 12 +10:29:12 12 +10:29:14 12 +10:29:16 12 +10:29:18 12 +10:29:20 12 +10:29:23 12 +10:29:25 12 +10:29:27 12 +10:29:29 12 +10:29:31 12 +10:29:33 12 +10:29:35 12 +10:29:37 12 +10:29:39 12 +10:29:41 12 +10:29:43 12 +10:29:45 12 +10:29:47 12 +10:29:49 12 +10:29:51 12 +10:29:54 12 +10:29:56 12 +10:29:58 12 +10:30:00 12 +10:30:02 12 +10:30:04 12 +10:30:06 12 +10:30:08 12 +10:30:10 12 +10:30:12 12 +10:30:14 12 +10:30:16 12 +10:30:18 12 +10:30:20 12 +10:30:22 12 +10:30:25 12 +10:30:27 12 +10:30:29 12 +10:30:31 12 +10:30:33 12 +10:30:35 12 +10:30:37 12 +10:30:39 12 +10:30:41 12 +10:30:43 12 +10:30:45 12 +10:30:47 12 +10:30:49 12 +10:30:51 12 +10:30:53 12 +10:30:56 12 +10:30:58 12 +10:31:00 12 +10:31:02 12 +10:31:04 12 +10:31:06 12 +10:31:08 12 +10:31:10 12 +10:31:12 12 +10:31:14 12 +10:31:16 12 +10:31:18 12 +10:31:20 12 +10:31:22 12 +10:31:24 12 +10:31:27 12 +10:31:29 12 +10:31:31 12 +10:31:33 12 +10:31:35 12 +10:31:37 12 +10:31:39 12 +10:31:41 12 +10:31:43 12 +10:31:45 12 +10:31:47 12 +10:31:49 12 +10:31:51 12 +10:31:53 12 +10:31:55 12 +10:31:58 12 +10:32:00 12 +10:32:02 12 +10:32:04 12 +10:32:06 12 +10:32:08 12 +10:32:10 12 +10:32:12 12 +10:32:14 12 +10:32:16 12 +10:32:18 12 +10:32:20 12 +10:32:22 12 +10:32:24 12 +10:32:26 12 +10:32:29 12 +10:32:31 12 +10:32:33 12 +10:32:35 12 +10:32:37 12 +10:32:39 12 +10:32:41 12 +10:32:43 12 +10:32:45 12 +10:32:47 12 +10:32:49 12 +10:32:51 12 +10:32:53 12 +10:32:55 12 +10:32:57 12 +10:33:00 12 +10:33:02 12 +10:33:04 12 +10:33:06 12 +10:33:08 12 +10:33:10 12 +10:33:12 12 +10:33:14 12 +10:33:16 12 +10:33:18 12 +10:33:20 12 +10:33:22 12 +10:33:24 12 +10:33:26 12 +10:33:28 12 +10:33:31 12 +10:33:33 12 +10:33:35 12 +10:33:37 12 +10:33:39 12 +10:33:41 12 +10:33:43 12 +10:33:45 12 +10:33:47 12 +10:33:49 12 +10:33:51 12 +10:33:53 12 +10:33:55 12 +10:33:57 12 +10:33:59 12 +10:34:02 12 +10:34:04 12 +10:34:06 12 +10:34:08 12 +10:34:10 12 +10:34:12 12 +10:34:14 12 +10:34:16 12 +10:34:18 12 +10:34:20 12 +10:34:22 12 +10:34:24 12 +10:34:26 12 +10:34:28 12 +10:34:30 12 +10:34:33 12 +10:34:35 12 +10:34:37 12 +10:34:39 12 +10:34:41 12 +10:34:43 12 +10:34:45 12 +10:34:47 12 +10:34:49 12 +10:34:51 12 +10:34:53 12 +10:34:55 12 +10:34:57 12 +10:34:59 12 +10:35:01 12 +10:35:04 12 +10:35:06 12 +10:35:08 12 +10:35:10 12 +10:35:12 12 +10:35:14 12 +10:35:16 12 +10:35:18 12 +10:35:20 12 +10:35:22 12 +10:35:24 12 +10:35:26 12 +10:35:28 12 +10:35:30 12 +10:35:33 12 +10:35:35 12 +10:35:37 12 +10:35:39 12 +10:35:41 12 +10:35:43 12 +10:35:45 12 +10:35:47 12 +10:35:49 12 +10:35:51 12 +10:35:53 12 +10:35:55 12 +10:35:57 12 +10:35:59 12 +10:36:01 12 +10:36:04 12 +10:36:06 12 +10:36:08 12 +10:36:10 12 +10:36:12 12 +10:36:14 12 +10:36:16 12 +10:36:18 12 +10:36:20 12 +10:36:22 12 +10:36:24 12 +10:36:26 12 +10:36:28 12 +10:36:30 12 +10:36:32 12 +10:36:34 12 +10:36:37 12 +10:36:39 12 +10:36:41 12 +10:36:43 12 +10:36:45 12 +10:36:47 12 +10:36:49 12 +10:36:51 12 +10:36:53 12 +10:36:55 12 +10:36:57 12 +10:36:59 12 +10:37:01 12 +10:37:03 12 +10:37:05 12 +10:37:08 12 +10:37:10 12 +10:37:12 12 +10:37:14 12 +10:37:16 12 +10:37:18 12 +10:37:20 12 +10:37:22 12 +10:37:24 12 +10:37:26 12 +10:37:28 12 +10:37:30 12 +10:37:32 12 +10:37:34 12 +10:37:36 12 +10:37:39 12 +10:37:41 12 +10:37:43 12 +10:37:45 12 +10:37:47 12 +10:37:49 12 +10:37:51 12 +10:37:53 12 +10:37:55 12 +10:37:57 12 +10:37:59 12 +10:38:01 12 +10:38:03 12 +10:38:05 12 +10:38:07 12 +10:38:09 12 +10:38:12 12 +10:38:14 12 +10:38:16 12 +10:38:18 12 +10:38:20 12 +10:38:22 12 +10:38:24 12 +10:38:26 12 +10:38:28 12 +10:38:30 12 +10:38:32 12 +10:38:34 12 +10:38:36 12 +10:38:38 12 +10:38:40 12 +10:38:43 12 +10:38:45 12 +10:38:47 12 +10:38:49 12 +10:38:51 12 +10:38:53 12 +10:38:55 12 +10:38:57 12 +10:38:59 12 +10:39:01 12 +10:39:03 12 +10:39:05 12 +10:39:07 12 +10:39:09 12 +10:39:11 12 +10:39:14 12 +10:39:16 12 +10:39:18 12 +10:39:20 12 +10:39:22 12 +10:39:24 12 +10:39:26 12 +10:39:28 12 +10:39:30 12 +10:39:32 12 +10:39:34 12 +10:39:36 12 +10:39:38 12 +10:39:40 12 +10:39:42 12 +10:39:44 12 +10:39:46 12 +10:39:49 12 +10:39:51 12 +10:39:53 12 +10:39:55 12 +10:39:57 12 +10:39:59 12 +10:40:01 12 +10:40:03 12 +10:40:05 12 +10:40:07 12 +10:40:09 12 +10:40:11 12 +10:40:13 12 +10:40:15 12 +10:40:17 12 +10:40:19 12 +10:40:22 12 +10:40:24 12 +10:40:26 12 +10:40:28 12 +10:40:30 12 +10:40:32 12 +10:40:34 12 +10:40:36 12 +10:40:38 12 +10:40:40 12 +10:40:42 12 +10:40:44 12 +10:40:46 12 +10:40:48 12 +10:40:50 12 +10:40:53 12 +10:40:55 12 +10:40:57 12 +10:40:59 12 +10:41:01 12 +10:41:03 12 +10:41:05 12 +10:41:07 12 +10:41:09 12 +10:41:11 12 +10:41:13 12 +10:41:15 12 +10:41:17 12 +10:41:19 12 +10:41:21 12 +10:41:23 12 +10:41:26 12 +10:41:28 12 +10:41:30 12 +10:41:32 12 +10:41:34 12 +10:41:36 12 +10:41:38 12 +10:41:40 12 +10:41:42 12 +10:41:44 12 +10:41:46 12 +10:41:48 12 +10:41:50 12 +10:41:52 12 +10:41:54 12 +10:41:56 12 +10:41:58 12 +10:42:01 12 +10:42:03 12 +10:42:05 12 +10:42:07 12 +10:42:09 12 +10:42:11 12 +10:42:13 12 +10:42:15 12 +10:42:17 12 +10:42:19 12 +10:42:21 12 +10:42:23 12 +10:42:25 12 +10:42:27 12 +10:42:29 12 +10:42:31 12 +10:42:34 12 +10:42:36 12 +10:42:38 12 +10:42:40 12 +10:42:42 12 +10:42:44 12 +10:42:46 12 +10:42:48 12 +10:42:50 12 +10:42:52 12 +10:42:54 12 +10:42:56 12 +10:42:58 12 +10:43:00 12 +10:43:02 12 +10:43:04 12 +10:43:07 12 +10:43:09 12 +10:43:11 12 +10:43:13 12 +10:43:15 12 +10:43:17 12 +10:43:19 12 +10:43:21 12 +10:43:23 12 +10:43:25 12 +10:43:27 12 +10:43:29 12 +10:43:31 12 +10:43:33 12 +10:43:35 12 +10:43:37 12 +10:43:39 12 +10:43:42 12 +10:43:44 12 +10:43:46 12 +10:43:48 12 +10:43:50 12 +10:43:52 12 +10:43:54 12 +10:43:56 12 +10:43:58 12 +10:44:00 12 +10:44:02 12 +10:44:04 12 +10:44:06 12 +10:44:08 12 +10:44:10 12 +10:44:12 12 +10:44:14 12 +10:44:17 12 +10:44:19 12 +10:44:21 12 +10:44:23 12 +10:44:25 12 +10:44:27 12 +10:44:29 12 +10:44:31 12 +10:44:33 12 +10:44:35 12 +10:44:37 12 +10:44:39 12 +10:44:41 12 +10:44:43 12 +10:44:45 12 +10:44:47 12 +10:44:50 12 +10:44:52 12 +10:44:54 12 +10:44:56 12 +10:44:58 12 +10:45:00 12 +10:45:02 12 +10:45:04 12 +10:45:06 12 +10:45:08 12 +10:45:10 12 +10:45:12 12 +10:45:14 12 +10:45:16 12 +10:45:18 12 +10:45:21 12 +10:45:23 12 +10:45:25 12 +10:45:27 12 +10:45:29 12 +10:45:31 12 +10:45:33 12 +10:45:35 12 +10:45:37 12 +10:45:39 12 +10:45:41 12 +10:45:43 12 +10:45:45 12 +10:45:47 12 +10:45:49 12 +10:45:51 12 +10:45:54 12 +10:45:56 12 +10:45:58 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/hostwatch-rank2.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/hostwatch-rank2.log new file mode 100644 index 0000000000..4f1a1764d2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/hostwatch-rank2.log @@ -0,0 +1,662 @@ +10:23:14 114 +10:23:16 112 +10:23:18 112 +10:23:20 112 +10:23:22 110 +10:23:24 110 +10:23:26 109 +10:23:28 109 +10:23:30 108 +10:23:32 108 +10:23:34 108 +10:23:37 106 +10:23:39 106 +10:23:41 106 +10:23:43 105 +10:23:45 105 +10:23:47 103 +10:23:49 103 +10:23:51 103 +10:23:53 102 +10:23:55 101 +10:23:57 101 +10:23:59 101 +10:24:01 100 +10:24:03 100 +10:24:05 99 +10:24:07 98 +10:24:09 98 +10:24:11 96 +10:24:14 97 +10:24:16 97 +10:24:18 96 +10:24:20 95 +10:24:22 95 +10:24:24 95 +10:24:26 93 +10:24:28 93 +10:24:30 93 +10:24:32 92 +10:24:34 92 +10:24:36 91 +10:24:38 91 +10:24:40 89 +10:24:42 89 +10:24:44 88 +10:24:46 88 +10:24:48 87 +10:24:51 87 +10:24:53 87 +10:24:55 86 +10:24:57 86 +10:24:59 84 +10:25:01 84 +10:25:03 83 +10:25:05 83 +10:25:07 83 +10:25:09 81 +10:25:11 82 +10:25:13 80 +10:25:15 80 +10:25:17 80 +10:25:19 79 +10:25:21 79 +10:25:23 77 +10:25:26 77 +10:25:28 77 +10:25:30 76 +10:25:32 76 +10:25:34 75 +10:25:36 75 +10:25:38 75 +10:25:40 73 +10:25:42 73 +10:25:44 72 +10:25:46 72 +10:25:48 72 +10:25:50 71 +10:25:52 71 +10:25:54 70 +10:25:56 70 +10:25:58 68 +10:26:00 68 +10:26:02 68 +10:26:05 67 +10:26:07 67 +10:26:09 67 +10:26:11 66 +10:26:13 66 +10:26:15 64 +10:26:17 65 +10:26:19 63 +10:26:21 63 +10:26:23 63 +10:26:25 62 +10:26:27 62 +10:26:29 60 +10:26:31 60 +10:26:33 60 +10:26:35 59 +10:26:37 58 +10:26:39 58 +10:26:41 56 +10:26:44 56 +10:26:46 56 +10:26:48 56 +10:26:50 56 +10:26:52 55 +10:26:54 55 +10:26:56 55 +10:26:58 55 +10:27:00 55 +10:27:02 55 +10:27:04 55 +10:27:06 55 +10:27:08 55 +10:27:10 34 +10:27:12 17 +10:27:14 17 +10:27:16 17 +10:27:19 17 +10:27:21 17 +10:27:23 17 +10:27:25 17 +10:27:27 17 +10:27:29 17 +10:27:31 17 +10:27:33 17 +10:27:35 17 +10:27:37 17 +10:27:39 17 +10:27:41 17 +10:27:43 17 +10:27:45 17 +10:27:47 17 +10:27:50 17 +10:27:52 17 +10:27:54 17 +10:27:56 17 +10:27:58 17 +10:28:00 17 +10:28:02 17 +10:28:04 17 +10:28:06 17 +10:28:08 17 +10:28:10 17 +10:28:12 17 +10:28:14 17 +10:28:16 17 +10:28:18 17 +10:28:21 17 +10:28:23 17 +10:28:25 17 +10:28:27 17 +10:28:29 17 +10:28:31 17 +10:28:33 17 +10:28:35 17 +10:28:37 17 +10:28:39 17 +10:28:41 17 +10:28:43 17 +10:28:45 17 +10:28:47 17 +10:28:50 17 +10:28:52 17 +10:28:54 17 +10:28:56 17 +10:28:58 17 +10:29:00 17 +10:29:02 17 +10:29:04 17 +10:29:06 17 +10:29:08 17 +10:29:10 17 +10:29:12 17 +10:29:14 17 +10:29:16 17 +10:29:18 17 +10:29:21 17 +10:29:23 17 +10:29:25 17 +10:29:27 17 +10:29:29 17 +10:29:31 17 +10:29:33 17 +10:29:35 17 +10:29:37 17 +10:29:39 17 +10:29:41 17 +10:29:43 17 +10:29:45 17 +10:29:47 17 +10:29:50 17 +10:29:52 17 +10:29:54 17 +10:29:56 17 +10:29:58 17 +10:30:00 17 +10:30:02 17 +10:30:04 17 +10:30:06 17 +10:30:08 17 +10:30:10 17 +10:30:12 17 +10:30:14 17 +10:30:16 17 +10:30:18 17 +10:30:21 17 +10:30:23 17 +10:30:25 17 +10:30:27 17 +10:30:29 17 +10:30:31 17 +10:30:33 17 +10:30:35 17 +10:30:37 17 +10:30:39 17 +10:30:41 17 +10:30:43 17 +10:30:45 17 +10:30:47 17 +10:30:50 17 +10:30:52 17 +10:30:54 17 +10:30:56 17 +10:30:58 17 +10:31:00 17 +10:31:02 17 +10:31:04 17 +10:31:06 17 +10:31:08 17 +10:31:10 17 +10:31:12 17 +10:31:14 17 +10:31:16 17 +10:31:18 17 +10:31:21 17 +10:31:23 17 +10:31:25 17 +10:31:27 17 +10:31:29 17 +10:31:31 17 +10:31:33 17 +10:31:35 17 +10:31:37 17 +10:31:39 17 +10:31:41 17 +10:31:43 17 +10:31:45 17 +10:31:47 17 +10:31:49 17 +10:31:52 17 +10:31:54 17 +10:31:56 17 +10:31:58 17 +10:32:00 17 +10:32:02 17 +10:32:04 17 +10:32:06 17 +10:32:08 17 +10:32:10 17 +10:32:12 17 +10:32:14 17 +10:32:16 17 +10:32:18 17 +10:32:20 17 +10:32:23 17 +10:32:25 17 +10:32:27 17 +10:32:29 17 +10:32:31 17 +10:32:33 17 +10:32:35 17 +10:32:37 17 +10:32:39 17 +10:32:41 17 +10:32:43 17 +10:32:45 17 +10:32:47 17 +10:32:49 17 +10:32:51 17 +10:32:54 17 +10:32:56 17 +10:32:58 17 +10:33:00 17 +10:33:02 17 +10:33:04 17 +10:33:06 17 +10:33:08 17 +10:33:10 17 +10:33:12 17 +10:33:14 17 +10:33:16 17 +10:33:18 17 +10:33:20 17 +10:33:23 17 +10:33:25 17 +10:33:27 17 +10:33:29 17 +10:33:31 17 +10:33:33 17 +10:33:35 17 +10:33:37 17 +10:33:39 17 +10:33:41 17 +10:33:43 17 +10:33:45 17 +10:33:47 17 +10:33:49 17 +10:33:52 17 +10:33:54 17 +10:33:56 17 +10:33:58 17 +10:34:00 17 +10:34:02 17 +10:34:04 17 +10:34:06 17 +10:34:08 17 +10:34:10 17 +10:34:12 17 +10:34:14 17 +10:34:16 17 +10:34:18 17 +10:34:20 17 +10:34:23 17 +10:34:25 17 +10:34:27 17 +10:34:29 17 +10:34:31 17 +10:34:33 17 +10:34:35 17 +10:34:37 17 +10:34:39 17 +10:34:41 17 +10:34:43 17 +10:34:45 17 +10:34:47 17 +10:34:49 17 +10:34:52 17 +10:34:54 17 +10:34:56 17 +10:34:58 17 +10:35:00 17 +10:35:02 17 +10:35:04 17 +10:35:06 17 +10:35:08 17 +10:35:10 17 +10:35:12 17 +10:35:14 17 +10:35:16 17 +10:35:18 17 +10:35:20 17 +10:35:23 17 +10:35:25 17 +10:35:27 17 +10:35:29 17 +10:35:31 17 +10:35:33 17 +10:35:35 17 +10:35:37 17 +10:35:39 17 +10:35:41 17 +10:35:43 17 +10:35:45 17 +10:35:47 17 +10:35:49 17 +10:35:51 17 +10:35:54 17 +10:35:56 17 +10:35:58 17 +10:36:00 17 +10:36:02 17 +10:36:04 17 +10:36:06 17 +10:36:08 17 +10:36:10 17 +10:36:12 17 +10:36:14 17 +10:36:16 17 +10:36:18 17 +10:36:20 17 +10:36:22 17 +10:36:24 17 +10:36:27 17 +10:36:29 17 +10:36:31 17 +10:36:33 17 +10:36:35 17 +10:36:37 17 +10:36:39 17 +10:36:41 17 +10:36:43 17 +10:36:45 17 +10:36:47 17 +10:36:49 17 +10:36:51 17 +10:36:53 17 +10:36:55 17 +10:36:58 17 +10:37:00 17 +10:37:02 17 +10:37:04 17 +10:37:06 17 +10:37:08 17 +10:37:10 17 +10:37:12 17 +10:37:14 17 +10:37:16 17 +10:37:18 17 +10:37:20 17 +10:37:22 17 +10:37:24 17 +10:37:26 17 +10:37:28 17 +10:37:31 17 +10:37:33 17 +10:37:35 17 +10:37:37 17 +10:37:39 17 +10:37:41 17 +10:37:43 17 +10:37:45 17 +10:37:47 17 +10:37:49 17 +10:37:51 17 +10:37:53 17 +10:37:55 17 +10:37:57 17 +10:37:59 17 +10:38:02 17 +10:38:04 17 +10:38:06 17 +10:38:08 17 +10:38:10 17 +10:38:12 17 +10:38:14 17 +10:38:16 17 +10:38:18 17 +10:38:20 17 +10:38:22 17 +10:38:24 17 +10:38:26 17 +10:38:28 17 +10:38:30 17 +10:38:33 17 +10:38:35 17 +10:38:37 17 +10:38:39 17 +10:38:41 17 +10:38:43 17 +10:38:45 17 +10:38:47 17 +10:38:49 17 +10:38:51 17 +10:38:53 17 +10:38:55 17 +10:38:57 17 +10:38:59 17 +10:39:01 17 +10:39:04 17 +10:39:06 17 +10:39:08 17 +10:39:10 17 +10:39:12 17 +10:39:14 17 +10:39:16 17 +10:39:18 17 +10:39:20 17 +10:39:22 17 +10:39:24 17 +10:39:26 17 +10:39:28 17 +10:39:30 17 +10:39:32 17 +10:39:34 17 +10:39:37 17 +10:39:39 17 +10:39:41 17 +10:39:43 17 +10:39:45 17 +10:39:47 17 +10:39:49 17 +10:39:51 17 +10:39:53 17 +10:39:55 17 +10:39:57 17 +10:39:59 17 +10:40:01 17 +10:40:03 17 +10:40:05 17 +10:40:08 17 +10:40:10 17 +10:40:12 17 +10:40:14 17 +10:40:16 17 +10:40:18 17 +10:40:20 17 +10:40:22 17 +10:40:24 17 +10:40:26 17 +10:40:28 17 +10:40:30 17 +10:40:32 17 +10:40:34 17 +10:40:36 17 +10:40:39 17 +10:40:41 17 +10:40:43 17 +10:40:45 17 +10:40:47 17 +10:40:49 17 +10:40:51 17 +10:40:53 17 +10:40:55 17 +10:40:57 17 +10:40:59 17 +10:41:01 17 +10:41:03 17 +10:41:05 17 +10:41:07 17 +10:41:09 17 +10:41:11 17 +10:41:14 17 +10:41:16 17 +10:41:18 17 +10:41:20 17 +10:41:22 17 +10:41:24 17 +10:41:26 17 +10:41:28 17 +10:41:30 17 +10:41:32 17 +10:41:34 17 +10:41:36 17 +10:41:38 17 +10:41:40 17 +10:41:42 17 +10:41:45 17 +10:41:47 17 +10:41:49 17 +10:41:51 17 +10:41:53 17 +10:41:55 17 +10:41:57 17 +10:41:59 17 +10:42:01 17 +10:42:03 17 +10:42:05 17 +10:42:07 17 +10:42:09 17 +10:42:11 17 +10:42:13 17 +10:42:15 17 +10:42:17 17 +10:42:20 17 +10:42:22 17 +10:42:24 17 +10:42:26 17 +10:42:28 17 +10:42:30 17 +10:42:32 17 +10:42:34 17 +10:42:36 17 +10:42:38 17 +10:42:40 17 +10:42:42 17 +10:42:44 17 +10:42:46 17 +10:42:48 17 +10:42:50 17 +10:42:52 17 +10:42:55 17 +10:42:57 17 +10:42:59 17 +10:43:01 17 +10:43:03 17 +10:43:05 17 +10:43:07 17 +10:43:09 17 +10:43:11 17 +10:43:13 17 +10:43:15 17 +10:43:17 17 +10:43:19 17 +10:43:21 17 +10:43:23 17 +10:43:25 17 +10:43:28 17 +10:43:30 17 +10:43:32 17 +10:43:34 17 +10:43:36 17 +10:43:38 17 +10:43:40 17 +10:43:42 17 +10:43:44 17 +10:43:46 17 +10:43:48 17 +10:43:50 17 +10:43:52 17 +10:43:54 17 +10:43:56 17 +10:43:58 17 +10:44:01 17 +10:44:03 17 +10:44:05 17 +10:44:07 17 +10:44:09 17 +10:44:11 17 +10:44:13 17 +10:44:15 17 +10:44:17 17 +10:44:19 17 +10:44:21 17 +10:44:23 17 +10:44:25 17 +10:44:27 17 +10:44:29 17 +10:44:31 17 +10:44:34 17 +10:44:36 17 +10:44:38 17 +10:44:40 17 +10:44:42 17 +10:44:44 17 +10:44:46 17 +10:44:48 17 +10:44:50 17 +10:44:52 17 +10:44:54 17 +10:44:56 17 +10:44:58 17 +10:45:00 17 +10:45:02 17 +10:45:05 17 +10:45:07 17 +10:45:09 17 +10:45:11 17 +10:45:13 17 +10:45:15 17 +10:45:17 17 +10:45:19 17 +10:45:21 17 +10:45:23 17 +10:45:25 17 +10:45:27 17 +10:45:29 17 +10:45:31 17 +10:45:33 17 +10:45:35 17 +10:45:38 17 +10:45:40 17 +10:45:42 17 +10:45:44 17 +10:45:46 17 +10:45:48 17 +10:45:50 17 +10:45:52 17 +10:45:54 17 +10:45:56 17 +10:45:58 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/long-beside-short.json b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/long-beside-short.json new file mode 100644 index 0000000000..886a44dce5 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/long-beside-short.json @@ -0,0 +1 @@ +{"alone": [{"prompt": "p2048-0-0", "seed": 1000, "sent": 753407.872357117, "first": 753409.133267049, "last": 753412.678982041, "tokens": 128, "pieces": [[753409.133267049, 4], [753409.194596665, 8], [753409.25676059, 13], [753409.316286592, 7], [753409.368522012, 11], [753409.422053209, 8], [753409.476707281, 14], [753409.531012177, 4], [753409.58694665, 4], [753409.646412843, 19], [753409.706469947, 22], [753409.766638077, 6], [753409.828829699, 5], [753409.887834136, 12], [753409.94069781, 12], [753409.994694199, 10], [753410.048451238, 7], [753410.102567565, 9], [753410.15704667, 4], [753410.211168166, 13], [753410.263771047, 4], [753410.320442687, 10], [753410.382396186, 12], [753410.446120546, 13], [753410.508140351, 7], [753410.570681017, 13], [753410.635760978, 14], [753410.705019044, 4], [753410.77420786, 14], [753410.839212988, 12], [753410.893836304, 10], [753410.953902634, 9], [753411.015679581, 15], [753411.072274207, 3], [753411.124874205, 11], [753411.177611183, 2], [753411.230147164, 6], [753411.283551054, 15], [753411.336248063, 3], [753411.389420796, 17], [753411.447983871, 7], [753411.507082735, 18], [753411.568159952, 6], [753411.628821144, 10], [753411.696063987, 21], [753411.76250753, 6], [753411.831709222, 9], [753411.897513645, 4], [753411.958330246, 11], [753412.017019576, 10], [753412.078674036, 10], [753412.137241732, 13], [753412.197084467, 7], [753412.249998741, 8], [753412.30532762, 9], [753412.365231061, 8], [753412.417967747, 1], [753412.470242486, 13], [753412.523050854, 19], [753412.575882727, 9], [753412.6282383, 13], [753412.678982041, 8]], "completed": 753412.679089676, "ttft_s": 1.260909931967035, "decode_tps": 35.81788166482201, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "7d62171119f8", "token_sha256": "7d62171119f86883ec9ea4e3542846c00509f1ad0baba0a0676ccd3b8e14cd56", "output_sha256": "ac406cb14fdb49d5cf4a0fcac1a4d3e1ea135c258abd0ad94ead39fc0308e2c7", "elapsed_s": 4.806732558994554, "chunk_offsets_s": [1.260909931967035, 1.3222395479679108, 1.3844034729991108, 1.4439294750336558, 1.4961648950120434, 1.5496960920281708, 1.6043501639505848, 1.6586550599895418, 1.714589532930404, 1.7740557260112837, 1.8341128299944103, 1.8942809599684551, 1.9564725819509476, 2.0154770190129057, 2.068340693018399, 2.1223370819352567, 2.176094120950438, 2.2302104480331764, 2.284689552965574, 2.3388110490050167, 2.39141392998863, 2.4480855700094253, 2.5100390689913183, 2.5737634289544076, 2.635783234029077, 2.698323899996467, 2.7634038609685376, 2.832661926979199, 2.9018507429864258, 2.966855871025473, 3.021479186951183, 3.0815455169649795, 3.1433224639622495, 3.1999170900089666, 3.252517087967135, 3.305254066013731, 3.357790046953596, 3.411193936946802, 3.4638909460045397, 3.517063678940758, 3.575626753969118, 3.6347256179433316, 3.6958028349326923, 3.7564640269847587, 3.823706869967282, 3.8901504130335525, 3.9593521050410345, 4.025156527990475, 4.0859731290256605, 4.144662459031679, 4.206316918949597, 4.264884614967741, 4.324727350031026, 4.377641623956151, 4.432970502995886, 4.49287394399289, 4.545610629953444, 4.597885368973948, 4.650693736970425, 4.703525610035285, 4.7558811829658225, 4.80662492394913], "chunk_gaps_ms": {"count": 61, "p50": 58.56769601814449, "p95": 67.24284298252314, "max": 69.25806601066142}}, {"prompt": "p2048-1-0", "seed": 1001, "sent": 753412.679489158, "first": 753413.930053271, "last": 753417.785493736, "tokens": 128, "pieces": [[753413.930053271, 3], [753413.989534422, 17], [753414.051775976, 6], [753414.109492011, 3], [753414.163637958, 10], [753414.218831178, 10], [753414.278822277, 19], [753414.34003051, 6], [753414.399310152, 9], [753414.462304653, 22], [753414.530230365, 18], [753414.596610374, 1], [753414.650105776, 4], [753414.701749485, 10], [753414.753723346, 7], [753414.807098265, 16], [753414.863186852, 5], [753414.923062668, 10], [753414.977255124, 11], [753415.031198326, 3], [753415.085126872, 5], [753415.138636271, 9], [753415.191997875, 6], [753415.24468711, 7], [753415.298310625, 3], [753415.352359974, 12], [753415.406016545, 7], [753415.45921456, 7], [753415.512421521, 8], [753415.565791076, 6], [753415.618637532, 7], [753415.67341621, 9], [753415.735737443, 13], [753415.798299498, 8], [753415.860062461, 13], [753415.912124561, 4], [753415.964008925, 5], [753416.017202058, 6], [753416.070407302, 8], [753416.123250138, 4], [753416.175981995, 8], [753416.228669131, 3], [753416.282654251, 10], [753416.336211328, 5], [753416.38905722, 14], [753416.442215871, 10], [753416.494261071, 8], [753416.547388682, 6], [753416.600783099, 5], [753416.654531445, 6], [753416.708508437, 7], [753416.761532381, 8], [753416.815197301, 6], [753416.870914655, 5], [753416.931318828, 11], [753416.988085277, 4], [753417.039391256, 7], [753417.09288156, 6], [753417.147302991, 5], [753417.200454327, 4], [753417.252776315, 16], [753417.30702611, 5], [753417.35921267, 9], [753417.411177961, 7], [753417.465378602, 10], [753417.519033567, 6], [753417.57339466, 13], [753417.626494475, 11], [753417.680199841, 7], [753417.733770675, 8], [753417.785493736, 6]], "completed": 753417.785572666, "ttft_s": 1.2505641130264848, "decode_tps": 32.94046455956617, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "c948b4bcbd58", "token_sha256": "c948b4bcbd5833ccd9c3bbd0f64e960f0ed36def137ba411ed3ade9437416854", "output_sha256": "78dbe5350838766da1d257beb3649f97e5ed2aab47e7249e7dcefae81dba7cef", "elapsed_s": 5.1060835079988465, "chunk_offsets_s": [1.2505641130264848, 1.3100452640792355, 1.3722868179902434, 1.4300028530415148, 1.4841488000238314, 1.5393420200562105, 1.599333118996583, 1.6605413520010188, 1.7198209940688685, 1.7828154950402677, 1.8507412070175633, 1.917121216072701, 1.970616618054919, 2.022260327008553, 2.074234188068658, 2.1276091070612893, 2.1836976939812303, 2.243573509971611, 2.297765966039151, 2.351709168055095, 2.4056377140805125, 2.4591471130261198, 2.512508717016317, 2.5651979519752786, 2.6188214670401067, 2.6728708159644157, 2.726527387043461, 2.779725402011536, 2.832932363031432, 2.8863019180716947, 2.9391483740182593, 2.9939270520117134, 3.056248284992762, 3.1188103400636464, 3.180573303019628, 3.2326354030519724, 3.2845197670394555, 3.337712900014594, 3.3909181440249085, 3.443760980037041, 3.49649283697363, 3.549179972964339, 3.603165092994459, 3.656722170067951, 3.7095680619822815, 3.7627267129719257, 3.8147719129920006, 3.8678995240479708, 3.9212939409771934, 3.9750422870274633, 4.029019279056229, 4.082043222966604, 4.135708143003285, 4.191425497061573, 4.251829670043662, 4.308596119051799, 4.359902098076418, 4.413392401998863, 4.467813833034597, 4.520965169067495, 4.573287156992592, 4.627536952029914, 4.679723512032069, 4.731688803061843, 4.78588944405783, 4.839544409071095, 4.893905502045527, 4.947005316964351, 5.000710683059879, 5.054281517048366, 5.106004578061402], "chunk_gaps_ms": {"count": 70, "p50": 53.639240039046854, "p95": 62.562055070884526, "max": 67.92571197729558}}, {"prompt": "p2048-2-0", "seed": 1002, "sent": 753417.785879921, "first": 753419.032314168, "last": 753422.916131895, "tokens": 128, "pieces": [[753419.032314168, 4], [753419.093442172, 5], [753419.146867316, 5], [753419.202058199, 11], [753419.255617907, 7], [753419.309352962, 4], [753419.362707353, 8], [753419.417513459, 12], [753419.475782193, 11], [753419.528703437, 2], [753419.582400987, 10], [753419.635033872, 12], [753419.688033166, 14], [753419.742322459, 8], [753419.796319873, 6], [753419.854775524, 6], [753419.907468506, 11], [753419.961619921, 9], [753420.016192306, 7], [753420.077025467, 17], [753420.136799626, 1], [753420.190730875, 5], [753420.244664812, 15], [753420.300339752, 3], [753420.360045382, 13], [753420.420520678, 11], [753420.480581436, 16], [753420.542162664, 7], [753420.61362658, 13], [753420.680557585, 18], [753420.745105925, 11], [753420.799329485, 6], [753420.852423946, 11], [753420.90745167, 9], [753420.958085828, 7], [753421.011072171, 4], [753421.064775491, 2], [753421.117095689, 2], [753421.170094065, 4], [753421.224387831, 2], [753421.277262555, 4], [753421.330425718, 3], [753421.383450286, 2], [753421.435790469, 9], [753421.487747107, 6], [753421.540567334, 7], [753421.593126674, 11], [753421.645629581, 6], [753421.698021382, 5], [753421.750429422, 5], [753421.803749949, 14], [753421.856320106, 9], [753421.908593728, 8], [753421.962336022, 6], [753422.015716291, 5], [753422.068556915, 8], [753422.120930824, 10], [753422.172273988, 14], [753422.226103452, 3], [753422.277414519, 18], [753422.331177949, 5], [753422.383395982, 18], [753422.435560414, 11], [753422.487339476, 9], [753422.540310263, 7], [753422.592930194, 10], [753422.647990952, 9], [753422.706814185, 6], [753422.758959992, 9], [753422.812055342, 14], [753422.864540534, 5], [753422.916131895, 4]], "completed": 753422.916234666, "ttft_s": 1.2464342470047995, "decode_tps": 32.69978380236319, "prompt_tokens": 2047, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "6a2fb209181c", "token_sha256": "6a2fb209181c72c924d2f9c33c76a176bd29daed59cf2e524f9797dc123b2b19", "output_sha256": "2c3b9b193b579f84e18d91f168239df0eea3505e429b38f4090ef34ba0e6f2f0", "elapsed_s": 5.130354744964279, "chunk_offsets_s": [1.2464342470047995, 1.3075622509932145, 1.3609873949317262, 1.4161782779265195, 1.469737985986285, 1.5234730408992618, 1.576827431912534, 1.631633537937887, 1.6899022719589993, 1.7428235159022734, 1.7965210659895092, 1.8491539509268478, 1.902153244940564, 1.9564425379503518, 2.010439951904118, 2.0688956029480323, 2.1215885849669576, 2.1757399999769405, 2.230312384897843, 2.2911455458961427, 2.350919705000706, 2.404850953957066, 2.458784890943207, 2.5144598309416324, 2.574165461002849, 2.634640756994486, 2.6947015149053186, 2.756282742950134, 2.8277466589352116, 2.8946776639204472, 2.9592260039644316, 3.0134495639940724, 3.066544024972245, 3.121571748983115, 3.172205906943418, 3.2251922499854118, 3.2788955699652433, 3.3312157680047676, 3.384214143967256, 3.4385079100029543, 3.491382633917965, 3.5445457969326526, 3.597570364945568, 3.6499105480033904, 3.701867185998708, 3.7546874129911885, 3.807246752898209, 3.859749659895897, 3.9121414609253407, 3.9645495009608567, 4.0178700279211625, 4.070440184907056, 4.1227138069225475, 4.176456101005897, 4.229836369981058, 4.28267699398566, 4.335050902911462, 4.386394066968933, 4.440223530982621, 4.49153459793888, 4.545298027922399, 4.597516060923226, 4.64968049293384, 4.701459554955363, 4.754430341999978, 4.807050272938795, 4.862111030961387, 4.920934263966046, 4.973080070922151, 5.026175420964137, 5.078660612925887, 5.130251973983832], "chunk_gaps_ms": {"count": 71, "p50": 53.35439101327211, "p95": 61.58122804481536, "max": 71.46391598507762}}, {"prompt": "p2048-0-1", "seed": 1003, "sent": 753422.916401326, "first": 753424.16304121, "last": 753427.583051099, "tokens": 128, "pieces": [[753424.16304121, 4], [753424.224498947, 5], [753424.278099567, 5], [753424.332063252, 11], [753424.386990784, 8], [753424.446683567, 15], [753424.508326252, 5], [753424.568447381, 17], [753424.626515757, 9], [753424.677977029, 13], [753424.73269422, 2], [753424.786376938, 4], [753424.840356319, 5], [753424.892938291, 5], [753424.945153579, 10], [753424.997342034, 1], [753425.04958201, 10], [753425.103465849, 11], [753425.155425531, 17], [753425.208715772, 13], [753425.261459616, 11], [753425.316851476, 4], [753425.377928368, 15], [753425.439978148, 12], [753425.502314463, 11], [753425.56625776, 4], [753425.625482288, 1], [753425.677660278, 6], [753425.733904366, 11], [753425.796589217, 14], [753425.856973677, 4], [753425.919515069, 12], [753425.983500667, 18], [753426.050582692, 7], [753426.112343069, 12], [753426.179123375, 12], [753426.243468966, 2], [753426.305525302, 11], [753426.366374665, 3], [753426.425335903, 4], [753426.477804969, 8], [753426.532225699, 13], [753426.592560858, 14], [753426.654050764, 5], [753426.716932512, 4], [753426.774880254, 6], [753426.82792239, 11], [753426.881910453, 10], [753426.941786639, 20], [753427.001924222, 10], [753427.06197529, 16], [753427.123296325, 6], [753427.19190283, 18], [753427.260626554, 11], [753427.327485706, 10], [753427.397337777, 10], [753427.466407221, 11], [753427.531128625, 19], [753427.583051099, 7]], "completed": 753427.583183646, "ttft_s": 1.2466398840770125, "decode_tps": 37.13439554974786, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "f8aaeecc59d0", "token_sha256": "f8aaeecc59d0c280cda17bf00ec2c94ab68bf77c8f76edf8f6198403d239a62b", "output_sha256": "2b70e346e03ff0a8636f7d9647c7f430909b6a7e725cf82d4cd2a4f3fab84952", "elapsed_s": 4.666782320011407, "chunk_offsets_s": [1.2466398840770125, 1.3080976210767403, 1.3616982410894707, 1.415661926032044, 1.4705894580110908, 1.5302822410594672, 1.5919249260332435, 1.6520460550673306, 1.7101144309854135, 1.7615757030434906, 1.8162928940728307, 1.8699756120331585, 1.9239549930207431, 1.9765369649976492, 2.0287522530416027, 2.080940708052367, 2.1331806840607896, 2.187064522993751, 2.2390242050169036, 2.292314446065575, 2.3450582900550216, 2.4004501500166953, 2.461527042090893, 2.523576822015457, 2.5859131370671093, 2.649856434087269, 2.709080961998552, 2.7612589520867914, 2.817503040074371, 2.8801878910744563, 2.9405723509844393, 3.0031137430341914, 3.067099341074936, 3.1341813660692424, 3.19594174309168, 3.2627220490248874, 3.3270676400279626, 3.3891239759977907, 3.4499733390985057, 3.508934576995671, 3.5614036429906264, 3.6158243730897084, 3.6761595320422202, 3.7376494380878285, 3.80053118604701, 3.8584789279848337, 3.9115210640011355, 3.9655091270105913, 4.02538531308528, 4.085522896028124, 4.145573963993229, 4.206894999020733, 4.275501504074782, 4.344225228065625, 4.41108438000083, 4.4809364510001615, 4.550005895085633, 4.614727299078368, 4.666649773018435], "chunk_gaps_ms": {"count": 58, "p50": 59.9636270198971, "p95": 68.7237239908427, "max": 69.85207099933177}}, {"prompt": "p2048-1-1", "seed": 1004, "sent": 753427.583473269, "first": 753428.833530606, "last": 753432.99395568, "tokens": 128, "pieces": [[753428.833530606, 4], [753428.892977848, 5], [753428.946606358, 5], [753429.000026974, 9], [753429.055926016, 16], [753429.116372079, 10], [753429.177297257, 5], [753429.237707063, 5], [753429.299838334, 8], [753429.368515024, 25], [753429.43580813, 14], [753429.502124429, 11], [753429.56899194, 15], [753429.638157522, 19], [753429.706154533, 8], [753429.763780657, 4], [753429.816556265, 10], [753429.868490301, 5], [753429.920709303, 12], [753429.972904447, 9], [753430.026101502, 16], [753430.080723999, 5], [753430.134389305, 10], [753430.189137421, 11], [753430.242579826, 4], [753430.296774744, 14], [753430.351718257, 9], [753430.40204694, 6], [753430.453792104, 4], [753430.505873757, 9], [753430.558802854, 8], [753430.611373894, 11], [753430.663731873, 14], [753430.71649239, 3], [753430.768471592, 6], [753430.821885164, 4], [753430.873877438, 1], [753430.927521095, 7], [753430.980639713, 6], [753431.033100028, 10], [753431.086140404, 21], [753431.138911974, 15], [753431.192475498, 5], [753431.24556666, 2], [753431.299417391, 5], [753431.352710717, 10], [753431.405319931, 4], [753431.458920113, 7], [753431.512636217, 1], [753431.566205038, 8], [753431.618591302, 15], [753431.671208756, 1], [753431.72581725, 4], [753431.778516562, 8], [753431.831378694, 5], [753431.883747918, 6], [753431.936463133, 5], [753431.987431713, 5], [753432.039908457, 10], [753432.093738113, 4], [753432.146791286, 5], [753432.200084001, 9], [753432.251970667, 7], [753432.306399041, 5], [753432.360642418, 5], [753432.412424474, 2], [753432.464545913, 8], [753432.516995856, 10], [753432.570976667, 5], [753432.623813324, 8], [753432.67682568, 4], [753432.729468348, 7], [753432.78330546, 5], [753432.835049834, 7], [753432.889342109, 9], [753432.943467642, 8], [753432.99395568, 6]], "completed": 753432.994044082, "ttft_s": 1.2500573369907215, "decode_tps": 30.525726996542936, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "854ffaa5d759", "token_sha256": "854ffaa5d75926074fa12f5f072ace31fab67f02428f5cc5cde0f8536b956273", "output_sha256": "5f25c0765ad2cacf455149436ab15653f18fa842e275ff063e318e99d60b7e45", "elapsed_s": 5.410570812993683, "chunk_offsets_s": [1.2500573369907215, 1.3095045789377764, 1.3631330890348181, 1.4165537049993873, 1.4724527469370514, 1.5328988099936396, 1.5938239879906178, 1.6542337939608842, 1.716365064959973, 1.7850417549489066, 1.8523348609451205, 1.9186511599691585, 1.9855186709901318, 2.0546842529438436, 2.1226812639506534, 2.180307387956418, 2.2330829959828407, 2.2850170319434255, 2.3372360339853913, 2.389431177987717, 2.442628232995048, 2.497250729938969, 2.550916036008857, 2.6056641519535333, 2.659106557024643, 2.7133014750434086, 2.768244987935759, 2.8185736710438505, 2.870318835019134, 2.922400487936102, 2.9753295850241557, 3.0279006250202656, 3.0802586040226743, 3.133019121014513, 3.184998322976753, 3.2384118949994445, 3.290404169005342, 3.344047826016322, 3.397166443988681, 3.4496267590438947, 3.502667134976946, 3.5554387050215155, 3.6090022289426997, 3.6620933909434825, 3.7159441219409928, 3.7692374479956925, 3.8218466619728133, 3.875446844031103, 3.9291629480430856, 3.9827317689778283, 4.03511803294532, 4.08773548703175, 4.142343980958685, 4.1950432929443195, 4.2479054250288755, 4.300274648936465, 4.352989863953553, 4.403958444017917, 4.456435187952593, 4.510264843935147, 4.563318016938865, 4.616610731929541, 4.668497398030013, 4.722925771959126, 4.777169149019755, 4.828951205010526, 4.881072644027881, 4.933522586943582, 4.98750339797698, 5.0403400550130755, 5.093352410942316, 5.14599507895764, 5.199832190992311, 5.25157656497322, 5.305868839961477, 5.359994372935034, 5.410482411039993], "chunk_gaps_ms": {"count": 76, "p50": 53.24488499900326, "p95": 67.29310599621385, "max": 69.16558195371181}}, {"prompt": "p2048-2-1", "seed": 1005, "sent": 753432.994220342, "first": 753434.241904254, "last": 753437.407757742, "tokens": 128, "pieces": [[753434.241904254, 3], [753434.307278382, 13], [753434.370177013, 13], [753434.431984898, 16], [753434.500196885, 11], [753434.568430745, 10], [753434.631791243, 2], [753434.686657988, 9], [753434.745071458, 1], [753434.798620108, 12], [753434.852236681, 10], [753434.906440323, 8], [753434.965905974, 2], [753435.017648131, 8], [753435.071391311, 10], [753435.127401921, 16], [753435.188727471, 6], [753435.248643628, 23], [753435.309686339, 18], [753435.367238473, 5], [753435.419027943, 17], [753435.473264943, 8], [753435.533517812, 17], [753435.593512115, 10], [753435.653621764, 17], [753435.716831102, 10], [753435.781989686, 7], [753435.837898693, 5], [753435.898939837, 15], [753435.959711163, 13], [753436.020321669, 15], [753436.079526222, 12], [753436.141471783, 9], [753436.209905256, 24], [753436.277150477, 18], [753436.348644949, 8], [753436.417160328, 10], [753436.484594625, 12], [753436.553690097, 26], [753436.622303558, 27], [753436.689956581, 20], [753436.754902404, 5], [753436.815543294, 15], [753436.874726375, 6], [753436.935899804, 13], [753436.997794417, 19], [753437.06109567, 3], [753437.116391808, 12], [753437.176888003, 16], [753437.238029238, 11], [753437.298931379, 7], [753437.356824314, 16], [753437.407757742, 3]], "completed": 753437.407833984, "ttft_s": 1.2476839119335636, "decode_tps": 40.115564564290885, "prompt_tokens": 2047, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "0effe3e2e261", "token_sha256": "0effe3e2e261cfefd427f381703c20e16f63a81a352520ec51fc2d517053699c", "output_sha256": "f287d0e9df37feb7e700f1bba37f292275b8c2f5ada5cc3161dbcba17024f69b", "elapsed_s": 4.413613641983829, "chunk_offsets_s": [1.2476839119335636, 1.313058039988391, 1.3759566709632054, 1.4377645560307428, 1.5059765429468825, 1.574210403021425, 1.6375709009589627, 1.6924376459792256, 1.7508511160267517, 1.8043997660279274, 1.8580163390142843, 1.9122199809644371, 1.9716856320155784, 2.023427788983099, 2.0771709689870477, 2.1331815789453685, 2.1945071290247142, 2.2544232859509066, 2.3154659969732165, 2.373018131009303, 2.4248076010262594, 2.4790446009719744, 2.539297470008023, 2.599291772930883, 2.659401422017254, 2.722610760014504, 2.787769343936816, 2.8436783510260284, 2.9047194949816912, 2.9654908209340647, 3.026101327035576, 3.085305879940279, 3.147251441027038, 3.2156849140301347, 3.282930135028437, 3.3544246070086956, 3.4229399859905243, 3.4903742829337716, 3.5594697549240664, 3.6280832160264254, 3.6957362389657646, 3.7606820620130748, 3.8213229519315064, 3.8805060329614207, 3.941679461975582, 4.003574074944481, 4.066875327960588, 4.122171466005966, 4.182667660992593, 4.243808895931579, 4.304711036966182, 4.362603971967474, 4.4135373999597505], "chunk_gaps_ms": {"count": 52, "p50": 60.83673349348828, "p95": 68.613461102359, "max": 71.49447198025882}}, {"prompt": "p2048-0-2", "seed": 1006, "sent": 753437.408017828, "first": 753438.660170394, "last": 753441.652425055, "tokens": 128, "pieces": [[753438.660170394, 3], [753438.724337504, 12], [753438.787384763, 18], [753438.84942992, 13], [753438.916110672, 6], [753438.985068321, 10], [753439.053781772, 16], [753439.121554018, 14], [753439.189689472, 26], [753439.256167591, 10], [753439.325801688, 9], [753439.391072515, 4], [753439.449877289, 14], [753439.509101784, 17], [753439.568843219, 16], [753439.62902084, 13], [753439.695403934, 24], [753439.760245551, 9], [753439.813456163, 8], [753439.8674278, 5], [753439.919860809, 8], [753439.97267008, 5], [753440.026044468, 5], [753440.079319334, 17], [753440.131426621, 13], [753440.185120009, 11], [753440.23846766, 7], [753440.291197906, 7], [753440.344865437, 11], [753440.398777821, 12], [753440.451564996, 4], [753440.507119467, 2], [753440.559077342, 21], [753440.613310438, 5], [753440.6671181, 11], [753440.722630842, 7], [753440.783176372, 12], [753440.84548684, 15], [753440.908474922, 16], [753440.972044168, 6], [753441.04146774, 25], [753441.113570813, 14], [753441.184465299, 14], [753441.254593512, 21], [753441.324570936, 5], [753441.394492424, 17], [753441.463268125, 16], [753441.531900319, 5], [753441.597324662, 6], [753441.652425055, 7]], "completed": 753441.652548994, "ttft_s": 1.252152565983124, "decode_tps": 42.44291157877803, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "2d30fcc66f99", "token_sha256": "2d30fcc66f9925bb2dbab2bfa2c3b819cfbe1e820d089d4774f26089df2b46b3", "output_sha256": "d34bdddcdc734baf9133c26fab126a3ff57ccc943549ce75ea13ddb4d7bfe2ef", "elapsed_s": 4.24453116604127, "chunk_offsets_s": [1.252152565983124, 1.3163196759996936, 1.379366934997961, 1.44141209195368, 1.5080928440438583, 1.57705049298238, 1.6457639440195635, 1.7135361899854615, 1.781671643955633, 1.848149762954563, 1.9177838600007817, 1.9830546870362014, 2.0418594609946012, 2.1010839559603482, 2.1608253909507766, 2.2210030120331794, 2.28738610597793, 2.3522277229931206, 2.4054383350303397, 2.459409971954301, 2.511842980980873, 2.564652252011001, 2.6180266400333494, 2.6713015059940517, 2.7234087929828092, 2.777102180989459, 2.8304498320212588, 2.883180078002624, 2.936847609002143, 2.9907599929720163, 3.0435471680248156, 3.099101639003493, 3.15105951402802, 3.205292609985918, 3.259100271970965, 3.3146130139939487, 3.3751585439313203, 3.4374690119875595, 3.5004570939345285, 3.5640263400273398, 3.6334499119548127, 3.705552984960377, 3.7764474709983915, 3.8465756840305403, 3.916553108021617, 3.986474595963955, 4.055250296951272, 4.123882491025142, 4.189306834014133, 4.244407227030024], "chunk_gaps_ms": {"count": 49, "p50": 62.045156955718994, "p95": 70.12821303214878, "max": 72.10307300556451}}], "together": [{"prompt": "p2048-0-0", "seed": 1000, "sent": 753443.655045728, "first": 753447.630586122, "last": 753462.948441311, "tokens": 128, "pieces": [[753447.630586122, 4], [753447.770119667, 8], [753449.176833691, 20], [753450.601750273, 22], [753452.03937769, 12], [753453.497813071, 14], [753453.707791063, 12], [753453.918255177, 24], [753454.125756606, 9], [753454.336121523, 12], [753454.55322067, 12], [753454.768038359, 10], [753454.988424648, 18], [753455.20602546, 17], [753455.426347707, 14], [753455.648906373, 21], [753455.864757081, 5], [753456.082820854, 17], [753456.305539287, 14], [753456.528500318, 4], [753456.754495433, 14], [753456.975042535, 12], [753457.203151524, 19], [753457.436224848, 15], [753457.664249291, 3], [753457.898181866, 13], [753458.131193641, 6], [753458.364553247, 16], [753458.596871597, 10], [753458.827477158, 13], [753459.059063708, 21], [753459.291498289, 6], [753459.521878057, 10], [753459.759550161, 21], [753459.993140364, 6], [753460.228822508, 9], [753460.464666127, 4], [753460.705201483, 11], [753460.937206553, 10], [753461.16803741, 13], [753461.387434319, 10], [753461.606473844, 7], [753461.82352691, 8], [753462.031926019, 17], [753462.228727365, 1], [753462.423837697, 32], [753462.615223516, 9], [753462.79438945, 17], [753462.948441311, 4]], "completed": 753462.948578434, "ttft_s": 3.9755403940798715, "decode_tps": 8.29097797526055, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "7d62171119f8", "token_sha256": "7d62171119f86883ec9ea4e3542846c00509f1ad0baba0a0676ccd3b8e14cd56", "output_sha256": "ac406cb14fdb49d5cf4a0fcac1a4d3e1ea135c258abd0ad94ead39fc0308e2c7", "elapsed_s": 19.293532705982216, "chunk_offsets_s": [3.9755403940798715, 4.115073939086869, 5.521787962992676, 6.94670454505831, 8.384331961977296, 9.842767343041487, 10.052745335036889, 10.263209449010901, 10.470710878027603, 10.681075795087963, 10.898174942005426, 11.11299263103865, 11.333378919982351, 11.550979732070118, 11.771301979082637, 11.993860644986853, 12.20971135306172, 12.42777512606699, 12.650493559078313, 12.873454589978792, 13.099449704983272, 13.319996807025746, 13.548105795984156, 13.781179120065644, 14.009203562978655, 14.243136138073169, 14.47614791302476, 14.709507519030012, 14.941825869027525, 15.172431430080906, 15.404017979977652, 15.636452561011538, 15.866832329076715, 16.104504432994872, 16.338094635982998, 16.57377678004559, 16.809620399028063, 17.05015575501602, 17.282160824979655, 17.51299168204423, 17.732388591044582, 17.95142811606638, 18.168481182074174, 18.376880291034468, 18.57368163706269, 18.768791969050653, 18.960177788045257, 19.139343722024933, 19.293395583052188], "chunk_gaps_ms": {"count": 48, "p50": 222.8397319559008, "p95": 1424.9165820656344, "max": 1458.4353810641915}}, {"prompt": "p2048-1-0", "seed": 1001, "sent": 753443.655857219, "first": 753449.015055929, "last": 753463.892914962, "tokens": 128, "pieces": [[753449.015055929, 3], [753449.176725176, 17], [753450.601846051, 6], [753452.039122325, 3], [753453.498058229, 10], [753453.707460047, 10], [753453.917247491, 19], [753454.125248882, 7], [753454.335055643, 13], [753454.552567776, 17], [753454.767760465, 18], [753454.98776409, 1], [753455.206121014, 4], [753455.425596651, 17], [753455.648571246, 21], [753455.864611766, 10], [753456.083025386, 11], [753456.305105118, 3], [753456.527829167, 5], [753456.753989389, 15], [753456.974707135, 7], [753457.203541277, 3], [753457.436295697, 19], [753457.663852354, 11], [753457.898068104, 10], [753458.130914019, 7], [753458.36423876, 9], [753458.596462228, 16], [753458.827113198, 10], [753459.058884504, 12], [753459.291191962, 5], [753459.521555937, 6], [753459.7593279, 8], [753459.992956647, 4], [753460.228459092, 11], [753460.464378025, 15], [753460.704935109, 24], [753460.937048181, 8], [753461.16758372, 6], [753461.387357933, 11], [753461.606332753, 7], [753461.823297465, 8], [753462.031430824, 11], [753462.228666899, 11], [753462.423896947, 4], [753462.614950758, 11], [753462.794122964, 7], [753462.948806103, 4], [753463.086113975, 16], [753463.211689382, 5], [753463.312343747, 16], [753463.408537151, 10], [753463.500964008, 6], [753463.599996689, 13], [753463.699440804, 11], [753463.798803668, 7], [753463.892914962, 14]], "completed": 753463.893026436, "ttft_s": 5.359198709949851, "decode_tps": 8.536174440003494, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "c948b4bcbd58", "token_sha256": "c948b4bcbd5833ccd9c3bbd0f64e960f0ed36def137ba411ed3ade9437416854", "output_sha256": "78dbe5350838766da1d257beb3649f97e5ed2aab47e7249e7dcefae81dba7cef", "elapsed_s": 20.23716921696905, "chunk_offsets_s": [5.359198709949851, 5.52086795703508, 6.945988832041621, 8.383265106007457, 9.842201009974815, 10.051602827967145, 10.261390271945857, 10.469391662976705, 10.679198423982598, 10.896710556931794, 11.111903246026486, 11.331906870938838, 11.550263794953935, 11.769739432027563, 11.99271402694285, 12.208754546940327, 12.427168166963384, 12.649247898953035, 12.871971947955899, 13.098132170038298, 13.318849915987812, 13.547684057964943, 13.780438477988355, 14.007995135034434, 14.242210885044187, 14.475056799943559, 14.708381540956907, 14.940605008974671, 15.171255978988484, 15.403027285006829, 15.635334742954, 15.865698718000203, 16.103470681002364, 16.337099427939393, 16.572601872961968, 16.80852080602199, 17.049077889998443, 17.281190961948596, 17.511726500932127, 17.73150071396958, 17.950475533958524, 18.16744024597574, 18.37557360494975, 18.57280968001578, 18.768039727932774, 18.95909353904426, 19.138265745015815, 19.29294888395816, 19.430256756022573, 19.55583216296509, 19.656486527994275, 19.752679931931198, 19.84510678902734, 19.944139469997026, 20.043583584949374, 20.142946448992006, 20.237057743011974], "chunk_gaps_ms": {"count": 56, "p50": 219.6249250555411, "p95": 1425.1208750065416, "max": 1458.9359039673582}}, {"prompt": "p2048-2-0", "seed": 1002, "sent": 753443.65652725, "first": 753444.928882222, "last": 753463.212012813, "tokens": 128, "pieces": [[753444.928882222, 4], [753445.014375074, 5], [753446.376040585, 13], [753447.769683417, 10], [753449.176460562, 12], [753450.601547405, 25], [753452.039039651, 18], [753453.497903473, 4], [753453.707199737, 14], [753453.917450647, 8], [753454.123678143, 12], [753454.334724468, 17], [753454.5524011, 10], [753454.767535404, 17], [753454.987557189, 1], [753455.206678466, 13], [753455.425538697, 7], [753455.648242982, 11], [753455.864254542, 5], [753456.082917784, 11], [753456.30495385, 18], [753456.527777694, 9], [753456.753936524, 23], [753456.974610797, 15], [753457.203361801, 6], [753457.436672426, 11], [753457.664050807, 16], [753457.898029287, 5], [753458.130983892, 7], [753458.364286457, 2], [753458.596635864, 4], [753458.827164591, 3], [753459.058925129, 2], [753459.29125286, 9], [753459.52174847, 6], [753459.759588914, 18], [753459.993178236, 6], [753460.228679849, 7], [753460.464703152, 3], [753460.705100184, 14], [753460.937131063, 17], [753461.167841117, 6], [753461.387480672, 5], [753461.606394914, 18], [753461.823571023, 17], [753462.032227482, 23], [753462.228468975, 29], [753462.423799872, 16], [753462.615172923, 10], [753462.794257255, 15], [753462.949104926, 23], [753463.086556993, 5], [753463.212012813, 4]], "completed": 753463.212157009, "ttft_s": 1.2723549720831215, "decode_tps": 6.946293982228077, "prompt_tokens": 2047, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "6a2fb209181c", "token_sha256": "6a2fb209181c72c924d2f9c33c76a176bd29daed59cf2e524f9797dc123b2b19", "output_sha256": "2c3b9b193b579f84e18d91f168239df0eea3505e429b38f4090ef34ba0e6f2f0", "elapsed_s": 19.55562975909561, "chunk_offsets_s": [1.2723549720831215, 1.3578478239942342, 2.7195133350323886, 4.113156167091802, 5.5199333120835945, 6.9450201550498605, 8.382512401090935, 9.841376223019324, 10.050672487006523, 10.260923397028819, 10.467150893062353, 10.678197217988782, 10.895873850095086, 11.111008153995499, 11.331029938999563, 11.550151216099039, 11.769011447089724, 11.99171573203057, 12.207727292086929, 12.426390534033999, 12.648426600033417, 12.871250444091856, 13.097409274079837, 13.318083547055721, 13.54683455103077, 13.780145175987855, 14.00752355705481, 14.241502037039027, 14.474456642055884, 14.707759206998162, 14.940108614042401, 15.170637341099791, 15.402397879050113, 15.63472561002709, 15.865221220068634, 16.103061664034612, 16.336650985991582, 16.572152598993853, 16.808175901998766, 17.04857293399982, 17.280603813007474, 17.511313867056742, 17.730953422025777, 17.94986766402144, 18.167043772991747, 18.375700231990777, 18.571941725094803, 18.767272622091696, 18.958645673003048, 19.13773000508081, 19.292577676009387, 19.430029743001796, 19.555485563003458], "chunk_gaps_ms": {"count": 52, "p50": 222.7640644996427, "p95": 1425.086842966266, "max": 1458.8638219283894}}, {"prompt": "p2048-0-1", "seed": 1003, "sent": 753443.657396902, "first": 753453.284260423, "last": 753463.212425302, "tokens": 128, "pieces": [[753453.284260423, 4], [753453.498095477, 5], [753453.708753436, 5], [753453.917761342, 19], [753454.125650443, 16], [753454.33580846, 12], [753454.552881991, 23], [753454.768082856, 10], [753454.98821514, 9], [753455.205820703, 5], [753455.426064549, 10], [753455.648649472, 1], [753455.86548921, 20], [753456.083090092, 18], [753456.305963089, 13], [753456.528592704, 15], [753456.755108006, 17], [753456.975227035, 21], [753457.203427114, 4], [753457.436849182, 1], [753457.663785073, 8], [753457.897927524, 13], [753458.130756271, 14], [753458.364085444, 15], [753458.596418867, 22], [753458.826972858, 12], [753459.058740901, 12], [753459.291097912, 2], [753459.521272747, 12], [753459.759190825, 6], [753459.992761779, 8], [753460.228289568, 13], [753460.464295047, 15], [753460.704524876, 6], [753460.93674027, 8], [753461.167498966, 21], [753461.387235706, 20], [753461.60628696, 10], [753461.82290704, 16], [753462.031742543, 6], [753462.228542785, 18], [753462.423624493, 11], [753462.615114185, 10], [753462.794084868, 10], [753462.948908522, 11], [753463.086057302, 19], [753463.212425302, 7]], "completed": 753463.212533449, "ttft_s": 9.626863521058112, "decode_tps": 12.791890701667326, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "f8aaeecc59d0", "token_sha256": "f8aaeecc59d0c280cda17bf00ec2c94ab68bf77c8f76edf8f6198403d239a62b", "output_sha256": "2b70e346e03ff0a8636f7d9647c7f430909b6a7e725cf82d4cd2a4f3fab84952", "elapsed_s": 19.555136547074653, "chunk_offsets_s": [9.626863521058112, 9.840698575018905, 10.051356534007937, 10.260364440036938, 10.468253541039303, 10.678411558037624, 10.895485089044087, 11.110685953986831, 11.330818237969652, 11.548423800966702, 11.768667647032999, 11.991252570063807, 12.20809230802115, 12.425693190074526, 12.648566187010147, 12.87119580199942, 13.097711103968322, 13.317830133019015, 13.546030211960897, 13.779452280024998, 14.006388170993887, 14.240530622075312, 14.473359368974343, 14.706688542035408, 14.93902196502313, 15.169575956068002, 15.401343998964876, 15.633701010025106, 15.863875844981521, 16.101793923065998, 16.33536487701349, 16.570892666000873, 16.806898145005107, 17.04712797398679, 17.27934336801991, 17.51010206399951, 17.729838803992607, 17.948890057974495, 18.165510138031095, 18.374345641001128, 18.571145883062854, 18.7662275909679, 18.957717283046804, 19.13668796606362, 19.29151162004564, 19.42866039997898, 19.555028400034644], "chunk_gaps_ms": {"count": 46, "p50": 220.18806502455845, "p95": 236.00547900423408, "max": 240.22982898168266}}, {"prompt": "p2048-1-1", "seed": 1004, "sent": 753443.657990004, "first": 753450.422419372, "last": 753464.4392566, "tokens": 128, "pieces": [[753450.422419372, 4], [753450.601699728, 5], [753452.039496557, 5], [753453.498117814, 9], [753453.70839938, 16], [753453.918078789, 10], [753454.124857994, 8], [753454.335142461, 10], [753454.553028714, 25], [753454.768290381, 14], [753454.988162131, 11], [753455.206057108, 15], [753455.426312267, 19], [753455.648966439, 8], [753455.865290613, 4], [753456.083311745, 10], [753456.30564465, 5], [753456.528542543, 12], [753456.75481848, 15], [753456.975163433, 10], [753457.203288631, 15], [753457.435982666, 11], [753457.664387134, 4], [753457.898883642, 14], [753458.131239482, 15], [753458.364682577, 4], [753458.597187172, 9], [753458.827679018, 8], [753459.059384596, 11], [753459.291850441, 14], [753459.52222832, 3], [753459.759690068, 6], [753459.993227933, 5], [753460.22874649, 7], [753460.464740897, 6], [753460.705127097, 10], [753460.93716916, 21], [753461.167998353, 20], [753461.387682468, 2], [753461.606525813, 5], [753461.8236068, 10], [753462.032271979, 11], [753462.228608034, 1], [753462.423957124, 8], [753462.615258973, 15], [753462.794208438, 1], [753462.949137071, 4], [753463.086610946, 13], [753463.212367925, 11], [753463.312823438, 15], [753463.409085067, 4], [753463.501442354, 5], [753463.600487756, 9], [753463.699798187, 7], [753463.798884662, 5], [753463.893587953, 7], [753463.957351315, 18], [753464.010891254, 5], [753464.063990046, 8], [753464.118684025, 4], [753464.171565789, 7], [753464.226027747, 5], [753464.278746052, 7], [753464.334063405, 9], [753464.38848893, 8], [753464.4392566, 6]], "completed": 753464.439332169, "ttft_s": 6.764429367962293, "decode_tps": 9.060531839989434, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "854ffaa5d759", "token_sha256": "854ffaa5d75926074fa12f5f072ace31fab67f02428f5cc5cde0f8536b956273", "output_sha256": "5f25c0765ad2cacf455149436ab15653f18fa842e275ff063e318e99d60b7e45", "elapsed_s": 20.781342164962552, "chunk_offsets_s": [6.764429367962293, 6.943709724000655, 8.381506552919745, 9.840127809904516, 10.050409375922754, 10.260088784969412, 10.46686798997689, 10.677152456948534, 10.895038709975779, 11.110300376894884, 11.330172126996331, 11.548067103954963, 11.768322262912989, 11.990976434899494, 12.207300608977675, 12.425321740913205, 12.647654645959847, 12.870552538894117, 13.096828475943767, 13.3171734289499, 13.545298626995645, 13.777992661925964, 14.006397129967809, 14.240893637994304, 14.473249477916397, 14.706692572915927, 14.939197167987004, 15.169689013971947, 15.401394591899589, 15.633860436966643, 15.864238315960392, 16.101700063911267, 16.335237928899005, 16.570756485918537, 16.806750892894343, 17.047137092915364, 17.279179155942984, 17.51000834896695, 17.72969246399589, 17.948535808944143, 18.165616796002723, 18.37428197497502, 18.57061802991666, 18.7659671199508, 18.957268968923017, 19.13621843396686, 19.291147066978738, 19.42862094193697, 19.554377920925617, 19.654833433916792, 19.751095062936656, 19.843452350003645, 19.942497751908377, 20.04180818295572, 20.140894657932222, 20.235597948892973, 20.299361310899258, 20.352901249891147, 20.406000041984953, 20.460694020963274, 20.513575784978457, 20.568037742981687, 20.620756047894247, 20.676073401002213, 20.730498925899155, 20.78126659593545], "chunk_gaps_ms": {"count": 65, "p50": 217.08098705857992, "p95": 237.46174795087427, "max": 1458.6212569847703}}, {"prompt": "p2048-2-1", "seed": 1005, "sent": 753443.659625994, "first": 753446.254648311, "last": 753462.031820433, "tokens": 128, "pieces": [[753446.254648311, 3], [753446.376161468, 13], [753447.77022503, 21], [753449.176539076, 14], [753450.601933813, 8], [753452.039606655, 9], [753453.497692716, 10], [753453.708460774, 12], [753453.91812607, 18], [753454.124468385, 2], [753454.335540582, 8], [753454.552838262, 10], [753454.768253276, 17], [753454.98838756, 26], [753455.20566742, 2], [753455.425966659, 23], [753455.649192988, 23], [753455.865227796, 19], [753456.082575664, 10], [753456.305711627, 18], [753456.528310714, 9], [753456.754778047, 7], [753456.975116072, 18], [753457.203476987, 15], [753457.437016705, 18], [753457.664679221, 18], [753457.898618004, 24], [753458.131300427, 18], [753458.364751091, 8], [753458.596973887, 10], [753458.827709819, 12], [753459.05914411, 26], [753459.291748167, 27], [753459.52212443, 20], [753459.759652387, 5], [753459.993263214, 15], [753460.228784507, 6], [753460.464780466, 13], [753460.70506788, 19], [753460.937255914, 3], [753461.1679584, 12], [753461.387728021, 22], [753461.606578038, 5], [753461.823457116, 11], [753462.031820433, 15]], "completed": 753462.031958772, "ttft_s": 2.59502231690567, "decode_tps": 8.049604771861052, "prompt_tokens": 2047, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "0effe3e2e261", "token_sha256": "0effe3e2e261cfefd427f381703c20e16f63a81a352520ec51fc2d517053699c", "output_sha256": "f287d0e9df37feb7e700f1bba37f292275b8c2f5ada5cc3161dbcba17024f69b", "elapsed_s": 18.372332777944393, "chunk_offsets_s": [2.59502231690567, 2.716535473940894, 4.110599035979249, 5.516913081984967, 6.942307818913832, 8.37998066097498, 9.83806672191713, 10.048834779998288, 10.25850007601548, 10.464842390967533, 10.675914587918669, 10.893212268012576, 11.10862728196662, 11.328761565964669, 11.546041426016018, 11.766340664937161, 11.989566994016059, 12.205601801979356, 12.42294966999907, 12.646085632964969, 12.868684720015153, 13.095152052934282, 13.315490077948198, 13.543850992922671, 13.777390711009502, 14.005053226952441, 14.238992010010406, 14.471674432978034, 14.705125096952543, 14.937347892904654, 15.168083824915811, 15.399518115911633, 15.63212217297405, 15.862498436006717, 16.10002639295999, 16.333637219970115, 16.56915851298254, 16.805154472007416, 17.045441885944456, 17.277629919932224, 17.508332405937836, 17.72810202697292, 17.9469520439161, 18.163831121986732, 18.372194438939914], "chunk_gaps_ms": {"count": 44, "p50": 228.01171545870602, "p95": 1425.3947369288653, "max": 1458.0860609421507}}, {"prompt": "p2048-0-2", "seed": 1006, "sent": 753443.659940881, "first": 753451.842582604, "last": 753462.794048675, "tokens": 128, "pieces": [[753451.842582604, 3], [753452.03979266, 12], [753453.497762782, 21], [753453.707622035, 10], [753453.917588954, 6], [753454.125447063, 10], [753454.33592299, 16], [753454.552783525, 14], [753454.767910164, 26], [753454.988032176, 10], [753455.206529599, 9], [753455.426413533, 4], [753455.649470178, 14], [753455.865454857, 22], [753456.083266976, 23], [753456.305419749, 12], [753456.52824212, 5], [753456.75438119, 8], [753456.975378318, 9], [753457.203226758, 8], [753457.436473957, 5], [753457.664606531, 8], [753457.898417471, 18], [753458.131272507, 9], [753458.364719346, 24], [753458.597030529, 14], [753458.827528199, 11], [753459.059213152, 12], [753459.291568195, 6], [753459.521948586, 21], [753459.759274139, 14], [753459.993041497, 17], [753460.228408515, 17], [753460.46424607, 18], [753460.704873619, 7], [753460.936892722, 26], [753461.167544039, 20], [753461.387300876, 14], [753461.606179566, 15], [753461.823081492, 9], [753462.031588956, 15], [753462.228344988, 16], [753462.423683102, 1], [753462.615073369, 6], [753462.794048675, 7]], "completed": 753462.794412379, "ttft_s": 8.182641723076813, "decode_tps": 11.596620870409147, "prompt_tokens": 2048, "cached_tokens": 0, "cache_state": "cold", "complete": true, "token_sha": "2d30fcc66f99", "token_sha256": "2d30fcc66f9925bb2dbab2bfa2c3b819cfbe1e820d089d4774f26089df2b46b3", "output_sha256": "d34bdddcdc734baf9133c26fab126a3ff57ccc943549ce75ea13ddb4d7bfe2ef", "elapsed_s": 19.13447149808053, "chunk_offsets_s": [8.182641723076813, 8.379851779085584, 9.837821901077405, 10.047681154101156, 10.257648073020391, 10.465506182052195, 10.67598210903816, 10.89284264401067, 11.10796928300988, 11.328091295086779, 11.546588718076237, 11.76647265208885, 11.989529296988621, 12.205513976048678, 12.423326095100492, 12.645478868042119, 12.868301239097491, 13.09444030909799, 13.315437437035143, 13.543285877094604, 13.776533076073974, 14.004665650078095, 14.238476590020582, 14.471331626060419, 14.704778465093113, 14.937089648097754, 15.167587317992002, 15.399272271082737, 15.631627314025536, 15.8620077050291, 16.099333257996477, 16.333100616000593, 16.56846763403155, 16.804305189056322, 17.044932738062926, 17.2769518410787, 17.50760315801017, 17.727359995013103, 17.946238685050048, 18.163140611024573, 18.371648075059056, 18.568404107005335, 18.76374222105369, 18.95513248804491, 19.13410779403057], "chunk_gaps_ms": {"count": 44, "p50": 222.48757199849933, "p95": 237.325552967377, "max": 1457.9701219918206}}, {"prompt": "long65536", "seed": 999, "sent": 753441.654142263, "first": 753443.103762382, "last": 753461.161889818, "tokens": 128, "pieces": [[753443.103762382, 7], [753443.187957773, 8], [753443.249701737, 8], [753443.312219318, 17], [753443.38129673, 8], [753443.449017503, 7], [753443.511846116, 10], [753443.566923958, 8], [753443.627691467, 8], [753443.689935426, 8], [753445.014171789, 16], [753446.375825941, 19], [753447.769498485, 29], [753449.176254157, 6], [753450.601336072, 5], [753452.039188326, 13], [753453.498164551, 15], [753453.707988059, 2], [753453.917279059, 5], [753454.123866628, 15], [753454.334942648, 17], [753454.5529438, 14], [753454.767589501, 2], [753454.987607318, 3], [753455.20551012, 9], [753455.425644796, 7], [753455.648307448, 13], [753455.86435136, 17], [753456.082478462, 13], [753456.305154111, 5], [753456.527882768, 14], [753456.754073439, 5], [753456.974658958, 15], [753457.203721201, 11], [753457.435876264, 8], [753457.664750662, 10], [753457.89872271, 11], [753458.131084566, 11], [753458.364130005, 12], [753458.596583927, 5], [753458.827053052, 16], [753459.058814519, 16], [753459.291145337, 19], [753459.521616307, 17], [753459.759247835, 20], [753459.992909782, 17], [753460.228609191, 2], [753460.464571789, 4], [753460.704617838, 10], [753460.936792719, 14], [753461.161889818, 7]], "completed": 753461.16198846, "ttft_s": 1.4496201189467683, "decode_tps": 7.032844377108333, "prompt_tokens": 65536, "cached_tokens": 63488, "cache_state": "reused", "complete": true, "token_sha": "37aa1a600539", "token_sha256": "37aa1a600539ec62fc9123edbb8a23c5014fb4d7b318fa839fd6fe79d626df8c", "output_sha256": "cfbc23fbdd79a4a87c680fe7c5f58640080f30ba1ac2d03226876b86f6123c86", "elapsed_s": 19.50784619699698, "chunk_offsets_s": [1.4496201189467683, 1.5338155099889264, 1.5955594739643857, 1.658077055006288, 1.7271544670220464, 1.7948752399533987, 1.8577038530493155, 1.912781695020385, 1.9735492039471865, 2.0357931629987434, 3.360029526054859, 4.721683677984402, 6.115356222027913, 7.522111893980764, 8.947193808970042, 10.385046062991023, 11.844022287987173, 12.053845796035603, 12.263136795954779, 12.469724365044385, 12.680800384958275, 12.89880153699778, 13.11344723799266, 13.33346505498048, 13.551367857027799, 13.771502532996237, 13.99416518502403, 14.21020909701474, 14.42833619902376, 14.651011848007329, 14.87374050496146, 15.09993117605336, 15.320516695035622, 15.549578938051127, 15.781734001007862, 16.010608399054036, 16.244580446975306, 16.4769423030084, 16.709987742011435, 16.94244166405406, 17.17291078902781, 17.4046722559724, 17.63700307405088, 17.867474043974653, 18.10510557203088, 18.33876751898788, 18.574466928024776, 18.810429526027292, 19.05047557502985, 19.282650455948897, 19.507747555035166], "chunk_gaps_ms": {"count": 50, "p50": 225.6438850890845, "p95": 1425.0819149892777, "max": 1458.9762249961495}}], "rows": [{"name": "p2048-0-0", "prompt": 2048, "sent": 2.002239335910417, "first": 5.977779729990289, "last": 21.295634918962605, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-1-0", "prompt": 2048, "sent": 2.0030508269555867, "first": 7.362249536905438, "last": 22.24010856996756, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-2-0", "prompt": 2047, "sent": 2.0037208579014987, "first": 3.27607582998462, "last": 21.559206420904957, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-0-1", "prompt": 2048, "sent": 2.0045905099250376, "first": 11.63145403098315, "last": 21.55961890995968, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-1-1", "prompt": 2048, "sent": 2.005183611996472, "first": 8.769612979958765, "last": 22.78645020793192, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-2-1", "prompt": 2047, "sent": 2.006819601985626, "first": 4.601841918891296, "last": 20.37901404092554, "error": null, "equal_alone": true, "started_after": []}, {"name": "p2048-0-2", "prompt": 2048, "sent": 2.007134488900192, "first": 10.189776211977005, "last": 21.14124228293076, "error": null, "equal_alone": true, "started_after": []}, {"name": "long65536", "prompt": 65536, "sent": 0.0013358709402382374, "first": 1.4509559898870066, "last": 19.509083425975405, "error": null, "equal_alone": null, "started_after": []}]} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/rows-without-bodies.jsonl b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/rows-without-bodies.jsonl new file mode 100644 index 0000000000..e3f2d348e5 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/rows-without-bodies.jsonl @@ -0,0 +1,44 @@ +{"i": 0, "conv": "S1", "turn": 1, "arm": "drafted", "group": "", "history": 7757, "allowed": [0], "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.134685469, "prefill": 5.132856122, "wall": 6.7744025870924816, "finish": "tool_calls"}, "ok": true, "body_sha256": "c89863023849e1c523580f91d6e3e557e187ee1345ec141afe8843f12f6a48bb", "body_bytes": 29036} +{"i": 1, "conv": "S1", "turn": 1, "arm": "plain", "group": "", "history": 7757, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 6144, "prompt": 7759, "tokens": 61, "ttft": 1.023564152, "prefill": 1.021553665, "wall": 3.7860231139929965, "finish": "tool_calls"}, "ok": true, "body_sha256": "990b29e03db5234f33dad6e1fa5929871234fa91c7f57ae49f27a1167d79423a", "body_bytes": 29050} +{"i": 2, "conv": "S1", "turn": 2, "arm": "drafted", "group": "", "history": 15548, "allowed": {"min": 1, "max": 7757}, "got": {"sha": "539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b", "cached": 6144, "prompt": 15550, "tokens": 46, "ttft": 6.145163485, "prefill": 6.142624225, "wall": 6.998688450083137, "finish": "tool_calls"}, "ok": true, "body_sha256": "1a19b65fdd199f0da26353d94699c49aa43f369adfc8c1dec9be7356d0fa0c33", "body_bytes": 52453} +{"i": 3, "conv": "S1", "turn": 2, "arm": "plain", "group": "", "history": 15548, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b", "cached": 14336, "prompt": 15550, "tokens": 46, "ttft": 0.828594711, "prefill": 0.825407421, "wall": 2.912110407021828, "finish": "tool_calls"}, "ok": true, "body_sha256": "4ce8f1ba60095661493f67fc4f85b7194f34821555018896027c3af3648c2bad", "body_bytes": 52467} +{"i": 4, "conv": "S1", "turn": 3, "arm": "drafted", "group": "", "history": 19460, "allowed": {"min": 7356, "max": 15548}, "got": {"sha": "8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa", "cached": 14336, "prompt": 19462, "tokens": 96, "ttft": 3.442631446, "prefill": 3.439261016, "wall": 5.7573707500705495, "finish": "tool_calls"}, "ok": true, "body_sha256": "156b7392acfc521c9c42aafe6fcc1bbd98b376513853a422a48f8c7fa38d2f68", "body_bytes": 64237} +{"i": 5, "conv": "S1", "turn": 3, "arm": "plain", "group": "", "history": 19460, "allowed": {"min": 11268, "max": 19460}, "got": {"sha": "8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa", "cached": 18432, "prompt": 19462, "tokens": 96, "ttft": 0.746726321, "prefill": 0.742931317, "wall": 5.091789445024915, "finish": "tool_calls"}, "ok": true, "body_sha256": "caf5f4d2359c1ccb74649b1f9eae57c63b4ff649985304ed1c758ed3be3fa584", "body_bytes": 64251} +{"i": 6, "conv": "S1", "turn": 4, "arm": "drafted", "group": "", "history": 21890, "allowed": {"min": 11268, "max": 19460}, "got": {"sha": "dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af", "cached": 18432, "prompt": 21892, "tokens": 113, "ttft": 2.321770406, "prefill": 2.317984213, "wall": 5.688801237032749, "finish": "tool_calls"}, "ok": true, "body_sha256": "373805b4976170ad95fcc1598631c81c362d74955106210979a78a1310fb2e4f", "body_bytes": 76205} +{"i": 7, "conv": "S1", "turn": 4, "arm": "plain", "group": "", "history": 21890, "allowed": {"min": 13698, "max": 21890}, "got": {"sha": "dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af", "cached": 20480, "prompt": 21892, "tokens": 113, "ttft": 0.931154921, "prefill": 0.926436525, "wall": 6.265577740967274, "finish": "tool_calls"}, "ok": true, "body_sha256": "aa8fd9486be28f95f77039bf7db9aaf10e1990d10f83594f3a915db60f290c99", "body_bytes": 76219} +{"i": 8, "conv": "S1", "turn": 5, "arm": "drafted", "group": "", "history": 24386, "allowed": {"min": 13698, "max": 21890}, "got": {"sha": "8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa", "cached": 20480, "prompt": 24388, "tokens": 160, "ttft": 2.548026564, "prefill": 2.543166181, "wall": 7.1600711540086195, "finish": "length"}, "ok": true, "body_sha256": "92f41b4871f17cfb0bd0d5b9fae7933c5f1d933e1f974a1fceead22cb788fbdb", "body_bytes": 88280} +{"i": 9, "conv": "S1", "turn": 5, "arm": "plain", "group": "", "history": 24386, "allowed": {"min": 16194, "max": 24386}, "got": {"sha": "8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa", "cached": 22528, "prompt": 24388, "tokens": 160, "ttft": 1.151008764, "prefill": 1.145715848, "wall": 8.428101971046999, "finish": "length"}, "ok": true, "body_sha256": "afa13346abd95c81758e980d9fc4c2cec22150b349720926a934e8b8e3b85c41", "body_bytes": 88294} +{"i": 10, "conv": "S1", "turn": 6, "arm": "drafted", "group": "", "history": 26893, "allowed": {"min": 16194, "max": 24386}, "got": {"sha": "c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1", "cached": 22528, "prompt": 26895, "tokens": 154, "ttft": 3.115561958, "prefill": 3.110212336, "wall": 7.700930665014312, "finish": "tool_calls"}, "ok": true, "body_sha256": "084dec4cb07280ee4f652f86dca6ad78f1b2e0da48343aed4b7523e178939aaa", "body_bytes": 100825} +{"i": 11, "conv": "S1", "turn": 6, "arm": "plain", "group": "", "history": 26893, "allowed": {"min": 18701, "max": 26893}, "got": {"sha": "c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1", "cached": 26624, "prompt": 26895, "tokens": 154, "ttft": 0.402602014, "prefill": 0.396357858, "wall": 7.369505221955478, "finish": "tool_calls"}, "ok": true, "body_sha256": "a28f28d0155fd90a6167a92006aae8b232092e290143ca6d7f7f32c904494ca8", "body_bytes": 100839} +{"i": 12, "conv": "S1", "turn": 7, "arm": "drafted", "group": "", "history": 29796, "allowed": {"min": 18701, "max": 26893}, "got": {"sha": "7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb", "cached": 26624, "prompt": 29798, "tokens": 136, "ttft": 2.218251966, "prefill": 2.212090879, "wall": 5.958830542978831, "finish": "tool_calls"}, "ok": true, "body_sha256": "d5501649c15be62caf0d40a12656bec16a6ac3e9b55aa5624aab250ed8b80156", "body_bytes": 113157} +{"i": 13, "conv": "S1", "turn": 7, "arm": "plain", "group": "", "history": 29796, "allowed": {"min": 21604, "max": 29796}, "got": {"sha": "7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb", "cached": 28672, "prompt": 29798, "tokens": 136, "ttft": 0.823062285, "prefill": 0.816189121, "wall": 7.212595800985582, "finish": "tool_calls"}, "ok": true, "body_sha256": "2a0f93e3b14e6adecb05f147f8e98a6e26ca482c3a7ff5ecb0f4a731a4d7268a", "body_bytes": 113171} +{"i": 14, "conv": "S1", "turn": 8, "arm": "drafted", "group": "", "history": 35432, "allowed": {"min": 21604, "max": 29796}, "got": {"sha": "1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e", "cached": 28672, "prompt": 35434, "tokens": 160, "ttft": 4.655256096, "prefill": 4.6476374, "wall": 8.485998181044124, "finish": "length"}, "ok": true, "body_sha256": "578d4b2d102a620558f79bf6cdc60de28eddbafa00ea3cdf1142d46a992c2149", "body_bytes": 136885} +{"i": 15, "conv": "S1", "turn": 8, "arm": "plain", "group": "", "history": 35432, "allowed": {"min": 27240, "max": 35432}, "got": {"sha": "1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e", "cached": 34816, "prompt": 35434, "tokens": 160, "ttft": 0.58093044, "prefill": 0.572253852, "wall": 7.8449738279450685, "finish": "length"}, "ok": true, "body_sha256": "a538a0863e0ecefde36f5aa380951fe8de08dd502459cf8370e4d2b1b1381afd", "body_bytes": 136899} +{"i": 16, "conv": "S1", "turn": 9, "arm": "drafted", "group": "", "history": 38150, "allowed": {"min": 27240, "max": 35432}, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 34816, "prompt": 38152, "tokens": 160, "ttft": 2.310885687, "prefill": 2.302586873, "wall": 4.69343441107776, "finish": "length"}, "ok": true, "body_sha256": "02b05f809ca914befcf64b1122f339c6059cde9d012dfd7ca4ecfc7a0742207a", "body_bytes": 149338} +{"i": 17, "conv": "S1", "turn": 9, "arm": "plain", "group": "", "history": 38150, "allowed": {"min": 29958, "max": 38150}, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 36864, "prompt": 38152, "tokens": 160, "ttft": 0.905400886, "prefill": 0.895398626, "wall": 8.410262683988549, "finish": "length"}, "ok": true, "body_sha256": "d77e94b64d11d1ae00f502915693a61415185c9c21c83781536f733fbdcff245", "body_bytes": 149352} +{"i": 18, "conv": "X", "turn": 1, "arm": "drafted", "group": "", "history": 2113, "allowed": [0], "got": {"sha": "7b1461bf71da95a2b74977aec25ecf7215b9f0fa044c82d2e1f4905ed66b798a", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.485235326, "prefill": 1.484491993, "wall": 5.790843086084351, "finish": "length"}, "ok": true, "body_sha256": "ea7f1fb5d610d52f549d66c8d5bac3876b0c32a1f369f7230266491a9a674737", "body_bytes": 9524} +{"i": 19, "conv": "A", "turn": 1, "arm": "drafted", "group": "", "history": 7410, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666", "cached": 6144, "prompt": 7412, "tokens": 90, "ttft": 0.899575238, "prefill": 0.897666407, "wall": 3.0834817009745166, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a153933bfbacdc64dd54d832df32d21139497927a2527fd60217851614cf603", "body_bytes": 28006} +{"i": 20, "conv": "A", "turn": 1, "arm": "plain", "group": "", "history": 7410, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666", "cached": 6144, "prompt": 7412, "tokens": 90, "ttft": 0.861819775, "prefill": 0.85984854, "wall": 4.919933769037016, "finish": "tool_calls"}, "ok": true, "body_sha256": "3a6c1cda2af13b7752b30514bbb91dc76381167e03b207540e31a0f810a462b3", "body_bytes": 28020} +{"i": 21, "conv": "B", "turn": 1, "arm": "drafted", "group": "", "history": 7472, "allowed": {"min": 1, "max": 6611}, "got": {"sha": "9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb", "cached": 6144, "prompt": 7474, "tokens": 73, "ttft": 0.927166125, "prefill": 0.925217632, "wall": 2.5044140219688416, "finish": "tool_calls"}, "ok": true, "body_sha256": "6685678d8326dff3c264f306810b45e76303da9dbc2ff27f9b4eec69f8a6384c", "body_bytes": 27976} +{"i": 22, "conv": "B", "turn": 1, "arm": "plain", "group": "", "history": 7472, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb", "cached": 6144, "prompt": 7474, "tokens": 73, "ttft": 0.880260822, "prefill": 0.87823293, "wall": 4.180341063067317, "finish": "tool_calls"}, "ok": true, "body_sha256": "ac1e48a68c55011216bc0008268ca26a6a788c311372fed6bc6dc9ed62769bf4", "body_bytes": 27990} +{"i": 23, "conv": "A", "turn": 2, "arm": "drafted", "group": "", "history": 12630, "allowed": {"min": 1, "max": 7410}, "got": {"sha": "e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe", "cached": 6144, "prompt": 12632, "tokens": 160, "ttft": 4.416058671, "prefill": 4.413014742, "wall": 8.708445438998751, "finish": "length"}, "ok": true, "body_sha256": "fb28d9b4a36a463bf91787edf362051f435742c60a3ea29de592c3ebdc11361b", "body_bytes": 51833} +{"i": 24, "conv": "A", "turn": 2, "arm": "plain", "group": "", "history": 12630, "allowed": {"min": 4438, "max": 12630}, "got": {"sha": "e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe", "cached": 12288, "prompt": 12632, "tokens": 160, "ttft": 0.433633409, "prefill": 0.430455845, "wall": 7.643164610024542, "finish": "length"}, "ok": true, "body_sha256": "2a6d20aaeedeccf2ebac186af1b76fd38a0fb806f645184f6630e367987d358d", "body_bytes": 51847} +{"i": 25, "conv": "B", "turn": 2, "arm": "drafted", "group": "", "history": 12719, "allowed": {"min": 1, "max": 7472}, "got": {"sha": "f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2", "cached": 6144, "prompt": 12721, "tokens": 117, "ttft": 4.444553598, "prefill": 4.441558424, "wall": 7.18896645703353, "finish": "tool_calls"}, "ok": true, "body_sha256": "fda373c63b23d357d58d8a9ecb9f16cb96fc10d6e5e9d2ea3cafb903bcf8608b", "body_bytes": 51401} +{"i": 26, "conv": "B", "turn": 2, "arm": "plain", "group": "", "history": 12719, "allowed": {"min": 4527, "max": 12719}, "got": {"sha": "f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2", "cached": 12288, "prompt": 12721, "tokens": 117, "ttft": 0.466420948, "prefill": 0.463393776, "wall": 5.749252839945257, "finish": "tool_calls"}, "ok": true, "body_sha256": "93626d5e61a91b5921a88051b0630ff2859498c2a8517353934d6171544701fc", "body_bytes": 51415} +{"i": 27, "conv": "A", "turn": 3, "arm": "drafted", "group": "", "history": 15381, "allowed": {"min": 4438, "max": 12630}, "got": {"sha": "7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8", "cached": 12288, "prompt": 15383, "tokens": 137, "ttft": 2.126357153, "prefill": 2.123435355, "wall": 6.00847516907379, "finish": "tool_calls"}, "ok": true, "body_sha256": "a38c39111b73b667c3d75f1060a245e7d3b0cb36c35f7eeeee68bb91be6c17d1", "body_bytes": 63752} +{"i": 28, "conv": "A", "turn": 3, "arm": "plain", "group": "", "history": 15381, "allowed": {"min": 7189, "max": 15381}, "got": {"sha": "7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8", "cached": 14336, "prompt": 15383, "tokens": 137, "ttft": 0.759124328, "prefill": 0.755323764, "wall": 6.960504766087979, "finish": "tool_calls"}, "ok": true, "body_sha256": "3db5a4468923842c916edfd757ee277fed929a6ec5105aa8467fe3036728719d", "body_bytes": 63766} +{"i": 29, "conv": "B", "turn": 3, "arm": "drafted", "group": "", "history": 17328, "allowed": {"min": 4527, "max": 12719}, "got": {"sha": "8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49", "cached": 12288, "prompt": 17330, "tokens": 160, "ttft": 3.416033683, "prefill": 3.412444198, "wall": 7.472774176974781, "finish": "length"}, "ok": true, "body_sha256": "267fa933c7f606d6d6314fc029a6cb97bce0bc7066b344a7b18fa635f7adc215", "body_bytes": 75090} +{"i": 30, "conv": "B", "turn": 3, "arm": "plain", "group": "", "history": 17328, "allowed": {"min": 9136, "max": 17328}, "got": {"sha": "8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49", "cached": 16384, "prompt": 17330, "tokens": 160, "ttft": 0.71745539, "prefill": 0.713449994, "wall": 7.959949289914221, "finish": "length"}, "ok": true, "body_sha256": "c3473c967c2cc3573877babbc60d1f29eabb0cdd204d42e6e7347afbed112064", "body_bytes": 75104} +{"i": 31, "conv": "E", "turn": 1, "arm": "drafted", "group": "", "history": 40899, "allowed": {"min": 0, "max": 6611}, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 6144, "prompt": 40901, "tokens": 160, "ttft": 22.645388799, "prefill": 22.636170994, "wall": 24.483482643961906, "finish": "length"}, "ok": true, "body_sha256": "34c6d718b76ef5b5545a86356a3aec31ffe590751ef69e5f4e45a391479a3ed8", "body_bytes": 161836} +{"i": 32, "conv": "E", "turn": 1, "arm": "plain", "group": "", "history": 40899, "allowed": {"min": 32707, "max": 40899}, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 38912, "prompt": 40901, "tokens": 160, "ttft": 1.258470202, "prefill": 1.248741997, "wall": 8.581305033061653, "finish": "length"}, "ok": true, "body_sha256": "134a574f4d8861c34ebbc98989a39fe4278d1680c3c0e8134d20df70c2e137dc", "body_bytes": 161850} +{"i": 33, "conv": "T", "turn": 1, "arm": "drafted", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89", "cached": 0, "prompt": 1520, "tokens": 150, "ttft": 0.919404922, "prefill": 0.91851861, "wall": 4.701138237956911, "finish": "stop"}, "ok": true, "body_sha256": "45cd3d7f3f990fbf5badf7e4179006aece0dac6ff89e43c1a049c9c47e65c0af", "body_bytes": 6672} +{"i": 34, "conv": "T", "turn": 1, "arm": "plain", "group": "", "history": 1517, "allowed": [0], "got": {"sha": "033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89", "cached": 0, "prompt": 1520, "tokens": 150, "ttft": 0.894626444, "prefill": 0.893927024, "wall": 7.524360149051063, "finish": "stop"}, "ok": true, "body_sha256": "17133648c09b8f37c9e3f3f57cf38d2952a766d4633876fb61d87b50e01f6e13", "body_bytes": 6686} +{"i": 35, "conv": "T", "turn": 2, "arm": "drafted", "group": "", "history": 4199, "allowed": [0], "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4202, "tokens": 12, "ttft": 3.051102866, "prefill": 3.049469086, "wall": 3.300818693009205, "finish": "stop"}, "ok": true, "body_sha256": "febc9e78b42428ba1d15c676438f3972b3e8b00c9da6d814d866087291e9b894", "body_bytes": 19045} +{"i": 36, "conv": "T", "turn": 2, "arm": "plain", "group": "", "history": 4199, "allowed": {"min": 1, "max": 4199}, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 4096, "prompt": 4202, "tokens": 12, "ttft": 0.272944211, "prefill": 0.271091171, "wall": 0.78609260590747, "finish": "stop"}, "ok": true, "body_sha256": "b0d4fa26b0d4b22fbe1d08df95e42b66857dd19fd41ad69195d183c829e4b327", "body_bytes": 19059} +{"i": 37, "conv": "C", "turn": 1, "arm": "cancel", "group": "", "history": 0, "allowed": null, "got": {"cancelled_after": 4.0}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 38, "conv": "C", "turn": 1, "arm": "drafted", "group": "", "history": 9700, "allowed": {"min": 0, "max": 9688}, "got": {"sha": "88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c", "cached": 6144, "prompt": 9702, "tokens": 110, "ttft": 2.394422732, "prefill": 2.325312022, "wall": 5.154998168931343, "finish": "tool_calls"}, "ok": true, "body_sha256": "19859217d49ada9aff105ddb4c415000d127fdf996ab3c7bc03d8291acece152", "body_bytes": 43537} +{"i": 39, "conv": "C", "turn": 1, "arm": "plain", "group": "", "history": 9700, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c", "cached": 8192, "prompt": 9702, "tokens": 110, "ttft": 0.972062838, "prefill": 0.969729102, "wall": 5.945752001018263, "finish": "tool_calls"}, "ok": true, "body_sha256": "a3d8ec39bf405406cdbde28193f74e4e2ac61b2857b32a308870da9fadd5869f", "body_bytes": 43551} +{"i": 40, "conv": "C", "turn": 2, "arm": "drafted", "group": "", "history": 15178, "allowed": {"min": 1508, "max": 9700}, "got": {"sha": "5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8", "cached": 8192, "prompt": 15180, "tokens": 160, "ttft": 4.68475377, "prefill": 4.681670057, "wall": 8.985663464991376, "finish": "length"}, "ok": true, "body_sha256": "efba18951a0026a5a71e35669811a161b4fa8942e1157f882467153ad3f58a50", "body_bytes": 67043} +{"i": 41, "conv": "C", "turn": 2, "arm": "plain", "group": "", "history": 15178, "allowed": {"min": 6986, "max": 15178}, "got": {"sha": "5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8", "cached": 14336, "prompt": 15180, "tokens": 160, "ttft": 0.67525181, "prefill": 0.672183032, "wall": 7.90281171107199, "finish": "length"}, "ok": true, "body_sha256": "9c483daa3c3535b10871eee66995ca6a2ce6e961c7f9abce173647ee97c5baa6", "body_bytes": 67057} +{"i": 42, "conv": "A", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 18244, "allowed": {"min": 7189, "max": 15381}, "got": {"sha": "9599ff0fa9b19bbb5cb5218a5b6f538de1146ce12dad36c729686f12f666389e", "cached": 14336, "prompt": 18246, "tokens": 160, "ttft": 2.542231339, "prefill": 2.538231555, "wall": 10.371066455030814, "finish": "length"}, "ok": true, "body_sha256": "402e0659a4a6d2fb96c8c8242a52736ec580ff23baa883d07793215bcb208aa4", "body_bytes": 75737} +{"i": 43, "conv": "B", "turn": 4, "arm": "drafted", "group": "concurrent", "history": 19806, "allowed": {"min": 9136, "max": 17328}, "got": {"sha": "025c4856c697674513547e9be3c33a84537e8a4fcb74e731ea09263af3e6d80d", "cached": 16384, "prompt": 19808, "tokens": 141, "ttft": 4.965768961, "prefill": 4.961347363, "wall": 9.614195440080948, "finish": "tool_calls"}, "ok": true, "body_sha256": "3135bceff4ad7c63fbaf488be28fd5b132b6bd297ad487045e89ae7cf3d0dbdf", "body_bytes": 86804} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/server-rank0.log new file mode 100644 index 0000000000..9293fa6521 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/server-rank0.log @@ -0,0 +1,12070 @@ +[kvp-serve] 2026-10-23T01:23:06+00:00 rank 0 of 3 context 1048576 parallel 8 binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=unset TF_GLM_MM=unset args +info: rank 0: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:112:112 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:112:112 [0] NCCL INFO cudaDriverVersion 13030 +host-a:112:112 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:112:112 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:112:112 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:112:112 [0] NCCL INFO Count not open libibverbs.so +host-a:112:112 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:112:112 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:112:112 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:112:112 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x77a35c0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:112:112 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x77e4cc0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:112:112 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x78263c0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:112:112 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x7867ac0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:112:112 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:112:112 [0] NCCL INFO Initialized NET plugin IB +host-a:112:112 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:112:112 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:112:112 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:112:112 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:112:112 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:112:112 [0] NCCL INFO Using network IB +host-a:112:112 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:112:112 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x4b6bf10 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9563dc80451a714b - Init START +host-a:112:112 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:112:112 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:112:112 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:112:112 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:112:112 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:112:112 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:112:112 [0] NCCL INFO Rank 0: 6 Net devices +host-a:112:112 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:112:112 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:112:112 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:112:112 [0] NCCL INFO comm 0x4b6bf10 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:112:112 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:112:112 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:112:112 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:112:112 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:112:112 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:112:112 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:112:112 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:112:112 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:112:130 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-a:112:129 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-a:112:129 [0] NCCL INFO proxy listening socket at <43703> +host-a:112:112 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:112:131 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 10 +host-a:112:112 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:112:112 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:112:112 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c0010000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c0054000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c009c000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6906 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6907 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6906 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6906 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6907 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b9797 ctsFifoLkey=0x1b9797 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6907 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c0145000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c018d000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c01d1000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c0219000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c025d000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1884 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1885 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1886 pkey=0 pd=0xe761c00f2d40 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6938 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6939 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6938 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6938 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6939 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6939 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2012 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2013 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2014 pkey=0 pd=0xe761c02f1b30 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1887 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1888 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1889 pkey=0 pd=0xe761c00f2d40 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6940 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6941 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6940 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ce7e3 ctsFifoLkey=0x1ce7e3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6940 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6941 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1ce7e3 ctsFifoLkey=0x1ce7e3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6941 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2015 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2016 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2017 pkey=0 pd=0xe761c02f1b30 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6906 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6907 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6938 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6939 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6908 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6909 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6908 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1afaf7 ctsFifoLkey=0x1afaf7 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6908 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6909 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1afaf7 ctsFifoLkey=0x1afaf7 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6909 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6940 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6941 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6908 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6909 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:112 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:112:112 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:112:112 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c05b5000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c05f6000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1890 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1891 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1890 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1890 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1891 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1891 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c068e000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c06d2000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1890 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1891 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2018 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2019 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2018 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b4ad ctsFifoLkey=0x19b4ad +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2018 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2019 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b4ad ctsFifoLkey=0x19b4ad +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2019 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2018 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2019 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1892 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1893 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1892 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17924b ctsFifoLkey=0x17924b +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1892 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1893 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17924b ctsFifoLkey=0x17924b +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1893 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2020 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2021 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2020 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b7b4 ctsFifoLkey=0x19b7b4 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2020 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2021 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b7b4 ctsFifoLkey=0x19b7b4 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2021 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1892 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1893 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2020 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2021 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:112 [0] NCCL INFO Connected all trees +host-a:112:112 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:112:112 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:112:112 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c0823000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c0868000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c08ad000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c08f5000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6910 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6911 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6912 pkey=0 pd=0xe761c0053590 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6942 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6943 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6944 pkey=0 pd=0xe761c02bfad0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6913 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6914 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6915 pkey=0 pd=0xe761c0053590 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6945 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6946 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6947 pkey=0 pd=0xe761c02bfad0 +host-a:112:112 [0] NCCL INFO Connected binomial trees +host-a:112:112 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:112:112 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:112:112 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:112:112 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:112:112 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-a:112:112 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:112:112 [0] NCCL INFO ncclCommInitRank comm 0x4b6bf10 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9563dc80451a714b - Init COMPLETE +host-a:112:112 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 2.54 (kernels 0.23, alloc 0.03, bootstrap 2.02, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.25, rest 0.00) +host-a:112:112 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:112:112 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:112:112 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:112:112 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:112:112 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:112:112 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:112:112 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:112:112 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c0b19000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c0b5a000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c0b9f000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6916 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6917 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6916 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6916 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6917 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1a3633 ctsFifoLkey=0x1a3633 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6917 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1894 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1895 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1896 pkey=0 pd=0xe761c00f2d40 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6948 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6949 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6948 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1cd2d6 ctsFifoLkey=0x1cd2d6 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6948 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6949 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1cd2d6 ctsFifoLkey=0x1cd2d6 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6949 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c0d13000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6916 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6917 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6948 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6949 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2022 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2023 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2024 pkey=0 pd=0xe761c02f1b30 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c0dcd000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c0e11000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xe761c0e59000) +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xe761c0e9d000) +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6918 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6919 pkey=0 pd=0xe761c0053590 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6920 pkey=0 pd=0xe761c0053590 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2025 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2026 pkey=0 pd=0xe761c02f1b30 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2025 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a6a8 ctsFifoLkey=0x19a6a8 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2025 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2026 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19a6a8 ctsFifoLkey=0x19a6a8 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2026 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1897 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1898 pkey=0 pd=0xe761c00f2d40 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1897 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1897 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1898 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1898 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6950 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6951 pkey=0 pd=0xe761c02bfad0 oooRq=0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6952 pkey=0 pd=0xe761c02bfad0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2025 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2026 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +einfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 65.28 GiB; 8 streams at once, the engine's own and 7 more taking 3355 pages of 512 rows (20.89 GiB) as they need them (enough for one at a 1048576-token window), 131 MiB fixed each, of 37.8 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, a 16.0 GiB prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 1048576; loaded in 245.1s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 1, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-b3523f068a41cf7ca3af957acba3ea90 prompt=14 cached=0 thinking=False effort=none tokens=16 sha=e195338a9a9d finish=length rounds=6 accepted=9/13 ttft=0.111 decode=0.367 elapsed=0.478 queued=0.000 tok/s=40.8 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 2, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-9d92ff54f6b8e63fbd13889ea6603db9 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.134 decode=5.416 elapsed=5.550 queued=0.000 tok/s=47.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 3, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-ba32f6d695b00eb413e26374f1339788 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=108 accepted=147/202 ttft=0.109 decode=6.171 elapsed=6.280 queued=0.000 tok/s=41.3 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 4, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-e45c5a911a01e7c21e26f72068eab918 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=106 accepted=150/212 ttft=0.109 decode=6.232 elapsed=6.342 queued=0.000 tok/s=40.9 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 5, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-04e83cc29688cc5f4bc0eed4c4cb4436 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=105 accepted=150/218 ttft=0.108 decode=6.204 elapsed=6.312 queued=0.000 tok/s=41.1 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 6, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-edbc89498517bd4f333c97c48cea915d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=77 accepted=179/227 ttft=0.111 decode=5.027 elapsed=5.138 queued=0.000 tok/s=50.7 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 7, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-ef0149a71a5686d9cc73fe98034e7397 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=104 accepted=151/218 ttft=0.110 decode=6.397 elapsed=6.507 queued=0.000 tok/s=39.9 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 8, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-4d3246cfceec72ff3ec307b5d14bc84f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=106 accepted=149/206 ttft=0.111 decode=6.263 elapsed=6.375 queued=0.000 tok/s=40.7 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 9, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-7780cf28cecb72f641a9ccc1534761a7 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=119 accepted=136/196 ttft=0.110 decode=6.822 elapsed=6.932 queued=0.000 tok/s=37.4 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 10, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-a38c2266e01d1756cf4d20b9693c0e68 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=255 accepted=0/0 ttft=0.109 decode=11.343 elapsed=11.452 queued=0.000 tok/s=22.5 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 11, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-4a0415c3156c2c48dfdf483f7cace231 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.573 elapsed=11.675 queued=0.000 tok/s=22.0 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 12, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-6f0d263459dda6230dc72bb43012bcca prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.399 elapsed=11.501 queued=0.000 tok/s=22.4 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 13, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-3ac9ea3e5ad5bf8dd1ff1fdb21acee96 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.632 elapsed=11.735 queued=0.000 tok/s=21.9 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 14, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-cfc3fd2d60b5264d2daf365e09583e3f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.646 elapsed=11.748 queued=0.000 tok/s=21.9 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 15, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-64b3894fa0be8e99a4d85ede56e959f8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.355 elapsed=11.458 queued=0.000 tok/s=22.5 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 16, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-8f41d10d9e0a4c3d64baf371feaa323a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=255 accepted=0/0 ttft=0.102 decode=11.403 elapsed=11.504 queued=0.000 tok/s=22.4 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 17, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-fc6b72288b784a25fa938c0078735a82 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.590 elapsed=11.693 queued=0.000 tok/s=22.0 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 18, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-15f58611b0d1544f7f69af5d5900c9ec prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=80 accepted=176/229 ttft=0.108 decode=5.308 elapsed=5.416 queued=0.000 tok/s=48.0 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 19, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 20, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-c57c15a72f3e15ab2f72f290eb45497b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.277 decode=7.493 elapsed=7.771 queued=0.169 tok/s=34.0 seed=1234 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-b717933e8b2e71783cce4856cbc14a4c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=96 accepted=159/273 ttft=0.110 decode=8.767 elapsed=8.878 queued=0.000 tok/s=29.1 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 23, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 24, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-0041e97b5420e9c20b9e09f25193b920 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.696 decode=11.161 elapsed=11.857 queued=0.166 tok/s=22.8 seed=1234 +[tensorfold] done cmpl-335a0e2868daa290c923695780a2932c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=89 accepted=166/264 ttft=0.290 decode=12.751 elapsed=13.040 queued=0.000 tok/s=20.0 seed=1236 +[tensorfold] done cmpl-2ec7e7f399dbe63bab81bf8748ca9fd0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=89 accepted=167/264 ttft=0.478 decode=12.640 elapsed=13.119 queued=0.000 tok/s=20.2 seed=1237 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-4a367fe3de416012b79591db7b6b0d76 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=95 accepted=161/283 ttft=0.108 decode=13.274 elapsed=13.382 queued=0.000 tok/s=19.2 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 25, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 32, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-385c28fc4d5ec27f17f3e8ecdcaf7b7e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=c547e852b429 finish=length rounds=77 accepted=179/228 ttft=0.689 decode=17.425 elapsed=18.113 queued=0.168 tok/s=14.6 seed=1234 +[tensorfold] done cmpl-fc8b5ce6a99af41cbb281279ec76d7de prompt=14 cached=0 thinking=False effort=none tokens=256 sha=3f4413b23169 finish=length rounds=79 accepted=176/236 ttft=0.928 decode=17.797 elapsed=18.725 queued=0.167 tok/s=14.3 seed=1238 +[tensorfold] done cmpl-e13c310205ec0b68b0359ecc6b21704a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=e9b7a497dd30 finish=length rounds=92 accepted=163/271 ttft=0.110 decode=20.269 elapsed=20.379 queued=0.000 tok/s=12.6 seed=1240 +[tensorfold] done cmpl-b7e1ad8a585da8869573287f353046c0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=51fe79c071ab finish=length rounds=89 accepted=166/264 ttft=1.189 decode=19.502 elapsed=20.691 queued=0.167 tok/s=13.1 seed=1239 +[tensorfold] done cmpl-e09c1d74da4b7f1a8af7b89a19ee177c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=0b4d014c8872 finish=length rounds=89 accepted=166/264 ttft=1.469 decode=19.350 elapsed=20.819 queued=0.167 tok/s=13.2 seed=1236 +[tensorfold] done cmpl-4c2481c9717c06737ae16b232c186003 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=b82191a9a6ec finish=length rounds=89 accepted=167/264 ttft=1.765 decode=19.163 elapsed=20.927 queued=0.167 tok/s=13.3 seed=1237 +[tensorfold] done cmpl-bbff5794e860843e65b4efe50a1866e2 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=eb0a1898a022 finish=length rounds=95 accepted=160/284 ttft=0.468 decode=20.555 elapsed=21.023 queued=0.169 tok/s=12.4 seed=1235 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-6bcda8af9957cf4d96a9ebff0d2b4fd5 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=097c9fe5896b finish=length rounds=102 accepted=153/301 ttft=0.278 decode=21.111 elapsed=21.389 queued=0.169 tok/s=12.1 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 33, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-e600e10278e9b25d6c9f494342825e66 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=3f46b826a241 finish=length rounds=10 accepted=5/13 ttft=0.160 decode=0.544 elapsed=0.704 queued=0.000 tok/s=27.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 34, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-14f09e497a1522e3171280208fc42c3d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=135 accepted=120/176 ttft=0.158 decode=7.289 elapsed=7.446 queued=0.000 tok/s=35.0 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 35, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-e6468610990e900e9ae7f86a38719be3 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=106 accepted=150/234 ttft=0.169 decode=6.612 elapsed=6.780 queued=0.000 tok/s=38.6 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 36, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-9e91c20617841fb81b2b7d8e8e52a54f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=133 accepted=123/191 ttft=0.160 decode=7.296 elapsed=7.456 queued=0.000 tok/s=35.0 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 37, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-f1bbc03e66aa5382f2fcb0cd1050d598 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=141 accepted=115/175 ttft=0.162 decode=7.564 elapsed=7.725 queued=0.000 tok/s=33.7 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 38, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-d5f65cc3228e5526a0d86d55f23b29be prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=135 accepted=120/197 ttft=0.161 decode=7.477 elapsed=7.638 queued=0.000 tok/s=34.1 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 39, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-9f3a1424a284114372a0be9146591a79 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=128 accepted=128/208 ttft=0.159 decode=7.402 elapsed=7.561 queued=0.000 tok/s=34.4 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 40, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-df6fd5d4bf0e5ea252bc09eb424ff79f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=124 accepted=131/205 ttft=0.159 decode=6.986 elapsed=7.145 queued=0.000 tok/s=36.5 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 41, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-2848c833576a6bb6495115e9567c5004 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=123 accepted=132/204 ttft=0.161 decode=6.953 elapsed=7.113 queued=0.000 tok/s=36.7 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 42, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-78e42ef77cffadb041a88e8cb714300f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.284 elapsed=11.437 queued=0.000 tok/s=22.6 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 43, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-ca95f1f14b866e083b73423d26839a8e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.469 elapsed=11.621 queued=0.000 tok/s=22.2 seed=1235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 44, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-cdbc9c166edfedc1d4ecb4f19bf72058 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=255 accepted=0/0 ttft=0.151 decode=11.258 elapsed=11.410 queued=0.000 tok/s=22.6 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 45, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-a5db765702eca014b7828213d31f2d46 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=255 accepted=0/0 ttft=0.153 decode=11.253 elapsed=11.406 queued=0.000 tok/s=22.7 seed=1237 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 46, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-c0f827642fde4b49da9a8e702b084f47 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=255 accepted=0/0 ttft=0.151 decode=11.477 elapsed=11.628 queued=0.000 tok/s=22.2 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 47, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-2b6293c86b2005392b06c065c7a38c93 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=255 accepted=0/0 ttft=0.151 decode=11.255 elapsed=11.406 queued=0.000 tok/s=22.7 seed=1239 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 48, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-14b461625e53368c53a483a5708dea73 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.266 elapsed=11.418 queued=0.000 tok/s=22.6 seed=1240 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 49, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-8c0ee8eed4eba3dcff66f9c42b9673e8 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=255 accepted=0/0 ttft=0.152 decode=11.474 elapsed=11.626 queued=0.000 tok/s=22.2 seed=1241 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 50, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-03629ddd27c3c9edf4ab4e00af855947 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=135 accepted=120/176 ttft=0.161 decode=7.278 elapsed=7.439 queued=0.000 tok/s=35.0 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 51, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 52, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-4f048aacc522ce6a138a2a91d178af34 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=94 accepted=161/280 ttft=0.160 decode=9.053 elapsed=9.214 queued=0.000 tok/s=28.2 seed=1235 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-f5f3e814d1280e7c513871ea990cd6df prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=104 accepted=151/291 ttft=0.378 decode=9.409 elapsed=9.788 queued=0.221 tok/s=27.1 seed=1234 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 53, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 56, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-e290b9228c5a359499c007473871e924 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=93 accepted=162/278 ttft=0.868 decode=13.213 elapsed=14.080 queued=0.217 tok/s=19.3 seed=1235 +[tensorfold] done chatcmpl-1ecb411201dd1fd644d8dbb422ae751f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=100 accepted=156/297 ttft=0.376 decode=14.301 elapsed=14.677 queued=0.217 tok/s=17.8 seed=1237 +[tensorfold] done chatcmpl-e579050c95bc0d0aa2b8bd6c8113e574 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=102 accepted=154/303 ttft=0.609 decode=14.322 elapsed=14.932 queued=0.217 tok/s=17.8 seed=1234 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-bb6e4df6577ff5f4138b51019e696f0f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=105 accepted=151/309 ttft=0.159 decode=14.824 elapsed=14.984 queued=0.000 tok/s=17.2 seed=1236 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 57, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 64, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-c8263fb19168f8c61677c5b3673279d0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=5743eea16ab2 finish=length rounds=93 accepted=162/278 ttft=1.463 decode=20.435 elapsed=21.899 queued=0.216 tok/s=12.5 seed=1235 +[tensorfold] done chatcmpl-8856582bfa4e32ea9f6bb0f3c4f6257e prompt=32 cached=0 thinking=False effort=none tokens=256 sha=864744d80fdf finish=length rounds=102 accepted=154/303 ttft=0.610 decode=22.478 elapsed=23.088 queued=0.217 tok/s=11.3 seed=1234 +[tensorfold] done chatcmpl-9dd5b4214cab621d1d880250680441a2 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c28174794bf0 finish=length rounds=103 accepted=153/304 ttft=0.375 decode=22.714 elapsed=23.089 queued=0.218 tok/s=11.2 seed=1239 +[tensorfold] done chatcmpl-db978a04584b8312079107f10a1a92ef prompt=32 cached=0 thinking=False effort=none tokens=256 sha=1a4e0d70467b finish=length rounds=101 accepted=154/298 ttft=1.163 decode=22.082 elapsed=23.245 queued=0.216 tok/s=11.5 seed=1241 +[tensorfold] done chatcmpl-e748187f74a39a63638df9195a72e80b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=c6d6ec2a35a8 finish=length rounds=106 accepted=149/314 ttft=0.160 decode=23.222 elapsed=23.382 queued=0.000 tok/s=11.0 seed=1240 +[tensorfold] done chatcmpl-9af021660146a6f73cbd07b64f67da27 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=28cedcb814fc finish=length rounds=100 accepted=156/297 ttft=2.128 decode=21.360 elapsed=23.488 queued=0.216 tok/s=11.9 seed=1237 +[tensorfold] done chatcmpl-dd2d6950f509bb1cee9ede493da25518 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=0a24c104609f finish=length rounds=105 accepted=151/311 ttft=1.784 decode=22.061 elapsed=23.845 queued=0.216 tok/s=11.6 seed=1236 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-1e719b65cddb82cd0883693e7312ea6d prompt=32 cached=0 thinking=False effort=none tokens=256 sha=22e8f3f09dfa finish=length rounds=112 accepted=143/329 ttft=0.881 decode=23.181 elapsed=24.062 queued=0.216 tok/s=11.0 seed=1238 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 65, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-ca97815416535f8fa39279d378112d77 prompt=14 cached=0 thinking=False effort=none tokens=16 sha=3000ccd48624 finish=length rounds=5 accepted=10/11 ttft=0.110 decode=0.377 elapsed=0.486 queued=0.000 tok/s=39.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 66, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-273830bd4db1707c93071d2406ce2fe8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=51 accepted=204/222 ttft=0.137 decode=3.708 elapsed=3.845 queued=0.000 tok/s=68.8 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 67, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-a9d58844b8c6b4597582966420608b61 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=255 accepted=0/0 ttft=0.103 decode=11.270 elapsed=11.372 queued=0.000 tok/s=22.6 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 68, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-13a92a308c980fec1a9416ca4652438f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=51 accepted=204/222 ttft=0.110 decode=3.613 elapsed=3.723 queued=0.000 tok/s=70.6 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 70, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 70, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-42ac21e5bc81ae38f288081aecc37343 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.109 decode=6.175 elapsed=6.284 queued=0.000 tok/s=41.3 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-06bdd1c36ba6f58daa1aa55add77ed9a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.276 decode=6.065 elapsed=6.341 queued=0.000 tok/s=42.0 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 71, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 74, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 74, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 74, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-293e4cf10e9059bc0e50c8f9c9672faa prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.112 decode=8.863 elapsed=8.976 queued=0.000 tok/s=28.8 +[tensorfold] done cmpl-835f844163e5066a67c56c01827dea9e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.279 decode=8.799 elapsed=9.078 queued=0.171 tok/s=29.0 +[tensorfold] done cmpl-8c1438905410d22d097e4c78cdd19231 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.467 decode=8.694 elapsed=9.161 queued=0.171 tok/s=29.3 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-ac42a193e2f5df48387be98d90a06ce8 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.677 decode=8.540 elapsed=9.217 queued=0.170 tok/s=29.9 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 75, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +info: prompt cache: 14 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 82, evicted 0, refused 0, failed 0) +[tensorfold] done cmpl-0c6b7f13a441393b5ff85fd561a043c1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/207 ttft=0.109 decode=12.649 elapsed=12.758 queued=0.000 tok/s=20.2 +[tensorfold] done cmpl-429a09c77603b9e94ecca35c71bff55e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.276 decode=12.641 elapsed=12.917 queued=0.168 tok/s=20.2 +[tensorfold] done cmpl-1cc335ee9223e6000eefa8e02703aa5c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.463 decode=12.598 elapsed=13.062 queued=0.167 tok/s=20.2 +[tensorfold] done cmpl-da545a3e3d7a23b40221893c3b1c661f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.675 decode=12.517 elapsed=13.192 queued=0.167 tok/s=20.4 +[tensorfold] done cmpl-968e6d9185e49e49b9bf9a1e1afbc90e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=0.905 decode=12.402 elapsed=13.306 queued=0.167 tok/s=20.6 +[tensorfold] done cmpl-1600c5b701cd1aa356ee145fd1b7605b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.153 decode=12.254 elapsed=13.407 queued=0.166 tok/s=20.8 +[tensorfold] done cmpl-2241ff80cbf6fd4b8b80503e77c1d325 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.417 decode=12.073 elapsed=13.490 queued=0.166 tok/s=21.1 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done cmpl-5a7abceaccc81682e9794a4d3662ff2a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=f1f736f680e3 finish=length rounds=70 accepted=185/208 ttft=1.694 decode=11.853 elapsed=13.547 queued=0.166 tok/s=21.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 83, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-e370fe423a20fd5ecff6bcdb7cfc7704 prompt=32 cached=0 thinking=False effort=none tokens=16 sha=e3dfbb415b68 finish=length rounds=5 accepted=10/12 ttft=0.159 decode=0.311 elapsed=0.470 queued=0.000 tok/s=48.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 84, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-01134681936b85c771dbcc7972c71ad8 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=136 accepted=119/185 ttft=0.159 decode=7.327 elapsed=7.487 queued=0.000 tok/s=34.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 85, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-5d6c55e03e753ee47b9c9df69cf6213f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=255 accepted=0/0 ttft=0.150 decode=11.470 elapsed=11.620 queued=0.000 tok/s=22.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 86, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-a19ce4fa3576e08f91c516eac517066b prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=136 accepted=119/185 ttft=0.162 decode=7.311 elapsed=7.474 queued=0.000 tok/s=34.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 87, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 88, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-b9bbdc5a92b84e603f7a211adb5eb8f0 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.160 decode=10.029 elapsed=10.188 queued=0.000 tok/s=25.4 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-e3b4f055b5c0e7f764897d4d2a42e238 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.379 decode=9.858 elapsed=10.237 queued=0.222 tok/s=25.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 89, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 92, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-7f696fbc771094464b054ae518d15063 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.159 decode=14.407 elapsed=14.566 queued=0.000 tok/s=17.7 +[tensorfold] done chatcmpl-fb9728923b3146080d0564762c8a31c6 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.378 decode=14.278 elapsed=14.656 queued=0.220 tok/s=17.9 +[tensorfold] done chatcmpl-4ffefbb4293548745369b14bd1b665ec prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.619 decode=14.103 elapsed=14.722 queued=0.220 tok/s=18.1 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-7b1f7143ab6eac1727a838f8e7fea806 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.883 decode=13.888 elapsed=14.771 queued=0.220 tok/s=18.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 93, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +info: prompt cache: 32 tokens, resumed at 0, kept 0; 0 states, 0 MiB and 16384 MiB spare of 16384 MiB (hits 0, misses 100, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-c029aa0d10319f4dc4906b8842e607e7 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/328 ttft=0.161 decode=22.360 elapsed=22.521 queued=0.000 tok/s=11.4 +[tensorfold] done chatcmpl-8a62bf44336dd03246bd2797c8420b09 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.379 decode=22.299 elapsed=22.678 queued=0.222 tok/s=11.4 +[tensorfold] done chatcmpl-c24efead832602007260014d8616e152 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.618 decode=22.204 elapsed=22.821 queued=0.221 tok/s=11.5 +[tensorfold] done chatcmpl-e748e81a3ecaa599639a5fcb97a3e369 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=0.880 decode=22.069 elapsed=22.950 queued=0.220 tok/s=11.6 +[tensorfold] done chatcmpl-7795c8a1699f51b2a8e52af6c4188e7f prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.156 decode=21.905 elapsed=23.061 queued=0.220 tok/s=11.6 +[tensorfold] done chatcmpl-93902322089f15b023f4f4f5e90dba0a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.451 decode=21.701 elapsed=23.152 queued=0.220 tok/s=11.8 +[tensorfold] done chatcmpl-b09c184919e87243307ac28c83699d7a prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=1.763 decode=21.455 elapsed=23.218 queued=0.220 tok/s=11.9 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-760068bb21f2111b08afb73774432c97 prompt=32 cached=0 thinking=False effort=none tokens=256 sha=e7870fdfe932 finish=length rounds=111 accepted=145/329 ttft=2.092 decode=21.174 elapsed=23.266 queued=0.220 tok/s=12.0 +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 0, kept 2; 2 states, 392 MiB and 15992 MiB spare of 16384 MiB (hits 0, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-31d2884d010872b0d89aa76eea4175a7 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=29 accepted=32/43 ttft=5.135 decode=1.638 elapsed=6.773 queued=0.000 tok/s=36.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7759 tokens, resumed at 6144, kept 0; 2 states, 392 MiB and 15992 MiB spare of 16384 MiB (hits 1, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-47c3527c19f3e1414a74af8f197f92f0 prompt=7759 cached=6144 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=1.024 decode=2.761 elapsed=3.784 queued=0.000 tok/s=21.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 6144, kept 1; 3 states, 702 MiB and 15682 MiB spare of 16384 MiB (hits 2, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-6f87e44b9a7c09dd47a9f74ef43cc223 prompt=15550 cached=6144 thinking=True effort=high tokens=46 sha=539973cb8b13 finish=stop rounds=13 accepted=33/36 ttft=6.145 decode=0.852 elapsed=6.997 queued=0.000 tok/s=52.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15550 tokens, resumed at 14336, kept 0; 3 states, 702 MiB and 15682 MiB spare of 16384 MiB (hits 3, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-5b4865efe813de3504002a8df64d580b prompt=15550 cached=14336 thinking=True effort=high tokens=46 sha=539973cb8b13 finish=stop rounds=45 accepted=0/0 ttft=0.829 decode=2.081 elapsed=2.910 queued=0.000 tok/s=21.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 19462 tokens, resumed at 14336, kept 1; 4 states, 1064 MiB and 15320 MiB spare of 16384 MiB (hits 4, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-077c50c38f70649065f65d54f44a5400 prompt=19462 cached=14336 thinking=True effort=high tokens=96 sha=8fc3c26162f1 finish=stop rounds=38 accepted=58/82 ttft=3.443 decode=2.313 elapsed=5.756 queued=0.000 tok/s=41.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 19462 tokens, resumed at 18432, kept 0; 4 states, 1064 MiB and 15320 MiB spare of 16384 MiB (hits 5, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-f59f74a77a624c70426426ce61206efb prompt=19462 cached=18432 thinking=True effort=high tokens=96 sha=8fc3c26162f1 finish=stop rounds=95 accepted=0/0 ttft=0.747 decode=4.343 elapsed=5.090 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 21892 tokens, resumed at 18432, kept 1; 5 states, 1452 MiB and 14932 MiB spare of 16384 MiB (hits 6, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-344874bf14060a60e7a8ed5e8674bcd6 prompt=21892 cached=18432 thinking=True effort=high tokens=113 sha=dedd451779f3 finish=stop rounds=61 accepted=52/80 ttft=2.322 decode=3.366 elapsed=5.687 queued=0.000 tok/s=33.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 21892 tokens, resumed at 20480, kept 0; 5 states, 1452 MiB and 14932 MiB spare of 16384 MiB (hits 7, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-d1d6f812f0eeb1b79034e47b2f32eb95 prompt=21892 cached=20480 thinking=True effort=high tokens=113 sha=dedd451779f3 finish=stop rounds=112 accepted=0/0 ttft=0.931 decode=5.332 elapsed=6.263 queued=0.000 tok/s=21.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 24388 tokens, resumed at 20480, kept 1; 6 states, 1864 MiB and 14520 MiB spare of 16384 MiB (hits 8, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bc4884837e5baa74797d4a59b68a7847 prompt=24388 cached=20480 thinking=True effort=high tokens=160 sha=8bcbd355dc04 finish=length rounds=81 accepted=79/121 ttft=2.548 decode=4.610 elapsed=7.158 queued=0.000 tok/s=34.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 24388 tokens, resumed at 22528, kept 0; 6 states, 1864 MiB and 14520 MiB spare of 16384 MiB (hits 9, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-3b58129de042e4a6020903f19cd9ecf9 prompt=24388 cached=22528 thinking=True effort=high tokens=160 sha=8bcbd355dc04 finish=length rounds=159 accepted=0/0 ttft=1.151 decode=7.276 elapsed=8.427 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 26895 tokens, resumed at 22528, kept 1; 7 states, 2328 MiB and 14056 MiB spare of 16384 MiB (hits 10, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-7dcf7542865c68b169f05be551b44794 prompt=26895 cached=22528 thinking=True effort=high tokens=154 sha=c8e79afc5a10 finish=stop rounds=81 accepted=72/122 ttft=3.116 decode=4.584 elapsed=7.699 queued=0.000 tok/s=33.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 26895 tokens, resumed at 26624, kept 0; 7 states, 2328 MiB and 14056 MiB spare of 16384 MiB (hits 11, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-7f608285dcf84555474826063e1201ae prompt=26895 cached=26624 thinking=True effort=high tokens=154 sha=c8e79afc5a10 finish=stop rounds=153 accepted=0/0 ttft=0.403 decode=6.965 elapsed=7.367 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 29798 tokens, resumed at 26624, kept 1; 8 states, 2818 MiB and 13566 MiB spare of 16384 MiB (hits 12, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-3ba7dd0d284c3dd678289459064eed50 prompt=29798 cached=26624 thinking=True effort=high tokens=136 sha=7629aa6bf5df finish=stop rounds=66 accepted=69/100 ttft=2.218 decode=3.739 elapsed=5.957 queued=0.000 tok/s=36.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 29798 tokens, resumed at 28672, kept 0; 8 states, 2818 MiB and 13566 MiB spare of 16384 MiB (hits 13, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-295759371400f7d8e176cbbb47dd9ce5 prompt=29798 cached=28672 thinking=True effort=high tokens=136 sha=7629aa6bf5df finish=stop rounds=135 accepted=0/0 ttft=0.823 decode=6.388 elapsed=7.211 queued=0.000 tok/s=21.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 35434 tokens, resumed at 28672, kept 1; 9 states, 3384 MiB and 13000 MiB spare of 16384 MiB (hits 14, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ca08bd0e9a1699c054fce11064c580b1 prompt=35434 cached=28672 thinking=True effort=high tokens=160 sha=1adac255ff32 finish=length rounds=60 accepted=99/148 ttft=4.655 decode=3.829 elapsed=8.484 queued=0.000 tok/s=41.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 35434 tokens, resumed at 34816, kept 0; 9 states, 3384 MiB and 13000 MiB spare of 16384 MiB (hits 15, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-cba4e063182e9f2aaefd30ccdfb6aab5 prompt=35434 cached=34816 thinking=True effort=high tokens=160 sha=1adac255ff32 finish=length rounds=159 accepted=0/0 ttft=0.581 decode=7.262 elapsed=7.843 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 38152 tokens, resumed at 34816, kept 1; 10 states, 3976 MiB and 12408 MiB spare of 16384 MiB (hits 16, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-3ae97935091a031692f8896f994f32cd prompt=38152 cached=34816 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=34 accepted=125/133 ttft=2.311 decode=2.380 elapsed=4.691 queued=0.000 tok/s=66.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 38152 tokens, resumed at 36864, kept 0; 10 states, 3976 MiB and 12408 MiB spare of 16384 MiB (hits 17, misses 101, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-46ec8db0544fc7120b66da456591ee65 prompt=38152 cached=36864 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=159 accepted=0/0 ttft=0.905 decode=7.502 elapsed=8.408 queued=0.000 tok/s=21.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 2115 tokens, resumed at 0, kept 1; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 17, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-b10b06e2174e9297f3f1ba3c63eeac0d prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=7b1461bf71da finish=length rounds=75 accepted=84/126 ttft=1.485 decode=4.305 elapsed=5.790 queued=0.000 tok/s=36.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 18, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-517e1a15653d5faf0c98e7d25aae4831 prompt=7412 cached=6144 thinking=True effort=high tokens=90 sha=d0573eef0609 finish=stop rounds=35 accepted=55/81 ttft=0.900 decode=2.183 elapsed=3.082 queued=0.000 tok/s=40.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7412 tokens, resumed at 6144, kept 0; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 19, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-d66ac1d13e3f9ad87b8d4c99d21132b1 prompt=7412 cached=6144 thinking=True effort=high tokens=90 sha=d0573eef0609 finish=stop rounds=89 accepted=0/0 ttft=0.862 decode=4.057 elapsed=4.918 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 20, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-d56547600c52cc9ce0ba7dd2092863bf prompt=7474 cached=6144 thinking=True effort=high tokens=73 sha=9a2685e9905f finish=stop rounds=24 accepted=49/65 ttft=0.927 decode=1.576 elapsed=2.503 queued=0.000 tok/s=45.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 7474 tokens, resumed at 6144, kept 0; 11 states, 4122 MiB and 12262 MiB spare of 16384 MiB (hits 21, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-2b7f79be71530777d33dc209653e5af2 prompt=7474 cached=6144 thinking=True effort=high tokens=73 sha=9a2685e9905f finish=stop rounds=72 accepted=0/0 ttft=0.880 decode=3.299 elapsed=4.179 queued=0.000 tok/s=21.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12632 tokens, resumed at 6144, kept 1; 12 states, 4408 MiB and 11976 MiB spare of 16384 MiB (hits 22, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-bf2624a4d8a3c5c3bcd2c4d62d1b7201 prompt=12632 cached=6144 thinking=True effort=high tokens=160 sha=e1f2511e60f3 finish=length rounds=75 accepted=84/122 ttft=4.416 decode=4.291 elapsed=8.707 queued=0.000 tok/s=37.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12632 tokens, resumed at 12288, kept 0; 12 states, 4408 MiB and 11976 MiB spare of 16384 MiB (hits 23, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-d5d28207bf19e54bcf207f41a49eba19 prompt=12632 cached=12288 thinking=True effort=high tokens=160 sha=e1f2511e60f3 finish=length rounds=159 accepted=0/0 ttft=0.434 decode=7.207 elapsed=7.641 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 12721 tokens, resumed at 6144, kept 1; 13 states, 4694 MiB and 11690 MiB spare of 16384 MiB (hits 24, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-4c80dbdcefaa12590565dc3f209ca88e prompt=12721 cached=6144 thinking=True effort=high tokens=117 sha=f202ab6075ae finish=stop rounds=44 accepted=72/103 ttft=4.445 decode=2.743 elapsed=7.188 queued=0.000 tok/s=42.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 12721 tokens, resumed at 12288, kept 0; 13 states, 4694 MiB and 11690 MiB spare of 16384 MiB (hits 25, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-56bf6f775b7a999b29fd90144ab8e4a6 prompt=12721 cached=12288 thinking=True effort=high tokens=117 sha=f202ab6075ae finish=stop rounds=116 accepted=0/0 ttft=0.466 decode=5.281 elapsed=5.748 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15383 tokens, resumed at 12288, kept 1; 14 states, 5004 MiB and 11380 MiB spare of 16384 MiB (hits 26, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-eea5da39af5b3f25c202db5c8e2daa31 prompt=15383 cached=12288 thinking=True effort=high tokens=137 sha=7695627844cd finish=stop rounds=70 accepted=67/97 ttft=2.126 decode=3.880 elapsed=6.007 queued=0.000 tok/s=35.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15383 tokens, resumed at 14336, kept 0; 14 states, 5004 MiB and 11380 MiB spare of 16384 MiB (hits 27, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-dcc19b6bb559e99ac146b353ef5925cb prompt=15383 cached=14336 thinking=True effort=high tokens=137 sha=7695627844cd finish=stop rounds=136 accepted=0/0 ttft=0.759 decode=6.199 elapsed=6.958 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 17330 tokens, resumed at 12288, kept 1; 15 states, 5340 MiB and 11044 MiB spare of 16384 MiB (hits 28, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-f5a83eadc03ffd5abfdb4fd160655f96 prompt=17330 cached=12288 thinking=True effort=high tokens=160 sha=8b15f58457e8 finish=length rounds=69 accepted=90/124 ttft=3.416 decode=4.055 elapsed=7.471 queued=0.000 tok/s=39.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 17330 tokens, resumed at 16384, kept 0; 15 states, 5340 MiB and 11044 MiB spare of 16384 MiB (hits 29, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-57ddac6795c151e7bb8c10e258a5e9f9 prompt=17330 cached=16384 thinking=True effort=high tokens=160 sha=8b15f58457e8 finish=length rounds=159 accepted=0/0 ttft=0.717 decode=7.241 elapsed=7.959 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 40901 tokens, resumed at 6144, kept 1; 16 states, 5956 MiB and 10428 MiB spare of 16384 MiB (hits 30, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-fdf89bede5fad3bae12086b6a21346c4 prompt=40901 cached=6144 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=22 accepted=137/137 ttft=22.645 decode=1.836 elapsed=24.481 queued=0.000 tok/s=86.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 40901 tokens, resumed at 38912, kept 0; 16 states, 5956 MiB and 10428 MiB spare of 16384 MiB (hits 31, misses 102, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-260d3758e98a0ab4183194b21acae767 prompt=40901 cached=38912 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=159 accepted=0/0 ttft=1.258 decode=7.321 elapsed=8.580 queued=0.000 tok/s=21.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 16 states, 5956 MiB and 10428 MiB spare of 16384 MiB (hits 31, misses 103, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-1ce1b12b14254eaba2f0a763aacb17ad prompt=1520 cached=0 thinking=False effort=none tokens=150 sha=033a197da586 finish=stop rounds=64 accepted=85/131 ttft=0.919 decode=3.781 elapsed=4.700 queued=0.000 tok/s=39.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 1520 tokens, resumed at 0, kept 0; 16 states, 5956 MiB and 10428 MiB spare of 16384 MiB (hits 31, misses 104, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-8b105eb4d6165f2d6500be670aca17c0 prompt=1520 cached=0 thinking=False effort=none tokens=150 sha=033a197da586 finish=stop rounds=149 accepted=0/0 ttft=0.895 decode=6.628 elapsed=7.523 queued=0.000 tok/s=22.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 4202 tokens, resumed at 0, kept 1; 17 states, 6140 MiB and 10244 MiB spare of 16384 MiB (hits 31, misses 105, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-f8ed7395caec6911aed0a942f9836130 prompt=4202 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.051 decode=0.248 elapsed=3.299 queued=0.000 tok/s=44.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 4202 tokens, resumed at 4096, kept 0; 17 states, 6140 MiB and 10244 MiB spare of 16384 MiB (hits 32, misses 105, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-97b22dd2486fe1e6666a2eccc94052c6 prompt=4202 cached=4096 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=0.273 decode=0.512 elapsed=0.785 queued=0.000 tok/s=21.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] ended chatcmpl-1f87bd6bc29ff3ba0cbc47191b07f5dc reason=client-left prompt=9702 tokens=0 after=4.08s +info: prompt cache: 9702 tokens, resumed at 6144, kept 1; 19 states, 6582 MiB and 9802 MiB spare of 16384 MiB (hits 33, misses 106, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-bc6936ea3dd86eda7b11203c682434ba prompt=9702 cached=6144 thinking=True effort=high tokens=110 sha=88f98460968a finish=stop rounds=47 accepted=63/86 ttft=2.394 decode=2.759 elapsed=5.154 queued=0.066 tok/s=39.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 9702 tokens, resumed at 8192, kept 0; 19 states, 6582 MiB and 9802 MiB spare of 16384 MiB (hits 34, misses 106, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-0a6d76c248a2fe131746478ed719257c prompt=9702 cached=8192 thinking=True effort=high tokens=110 sha=88f98460968a finish=stop rounds=109 accepted=0/0 ttft=0.972 decode=4.972 elapsed=5.944 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 15180 tokens, resumed at 8192, kept 1; 20 states, 6892 MiB and 9492 MiB spare of 16384 MiB (hits 35, misses 106, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-07f8d1d144f564c5863ddda37daafca6 prompt=15180 cached=8192 thinking=True effort=high tokens=160 sha=5694b66a8995 finish=length rounds=76 accepted=83/115 ttft=4.685 decode=4.299 elapsed=8.984 queued=0.000 tok/s=37.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 15180 tokens, resumed at 14336, kept 0; 20 states, 6892 MiB and 9492 MiB spare of 16384 MiB (hits 36, misses 106, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-ec6746224c432b7864dd1c724ec45e45 prompt=15180 cached=14336 thinking=True effort=high tokens=160 sha=5694b66a8995 finish=length rounds=159 accepted=0/0 ttft=0.675 decode=7.225 elapsed=7.901 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/tokenize HTTP/1.1" 200 - +info: prompt cache: 18246 tokens, resumed at 14336, kept 1; 21 states, 7228 MiB and 9156 MiB spare of 16384 MiB (hits 37, misses 106, evicted 0, refused 0, failed 0) +info: prompt cache: 19808 tokens, resumed at 16384, kept 2; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 106, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-1b86d340616b609293e967de8cc697d0 prompt=19808 cached=16384 thinking=True effort=high tokens=141 sha=025c4856c697 finish=stop rounds=48 accepted=93/143 ttft=4.966 decode=4.646 elapsed=9.612 queued=0.000 tok/s=30.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-34d3217e29eb57839cd214c3f7fbe08d prompt=18246 cached=14336 thinking=True effort=high tokens=160 sha=9599ff0fa9b1 finish=length rounds=62 accepted=97/175 ttft=2.542 decode=7.826 elapsed=10.369 queued=0.000 tok/s=20.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 1023 tokens, resumed at 0, kept 0; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 107, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-b49d151f176f3e5fe2f15a5349f5ef2d prompt=1023 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=0.713 decode=0.051 elapsed=0.763 queued=0.000 tok/s=19.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 108, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-3e32e99805e1afb41987c7909dd856c9 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=1/1 ttft=1.243 decode=0.053 elapsed=1.296 queued=0.000 tok/s=18.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 109, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-a7355958c3a7b1d5b51a11299ca741f4 prompt=2048 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.241 decode=0.052 elapsed=1.293 queued=0.000 tok/s=19.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 22 states, 7590 MiB and 8794 MiB spare of 16384 MiB (hits 38, misses 110, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-d0cea79534152cfcda76469359b4d674 prompt=2047 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=1.234 decode=0.053 elapsed=1.287 queued=0.000 tok/s=18.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 8191 tokens, resumed at 0, kept 1; 23 states, 7798 MiB and 8586 MiB spare of 16384 MiB (hits 38, misses 111, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-1b54906e878ed3ebda342a9492c58595 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=5.396 decode=0.055 elapsed=5.451 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 8192 tokens, resumed at 0, kept 1; 24 states, 8006 MiB and 8378 MiB spare of 16384 MiB (hits 38, misses 112, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-2461b654e947eb1b588958de932ebc6e prompt=8192 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=5.406 decode=0.055 elapsed=5.460 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 8191 tokens, resumed at 0, kept 1; 25 states, 8214 MiB and 8170 MiB spare of 16384 MiB (hits 38, misses 113, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-320f7a994d2c56c142cab1c1ec6e9770 prompt=8191 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=5.403 decode=0.054 elapsed=5.457 queued=0.000 tok/s=18.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 16384 tokens, resumed at 0, kept 1; 26 states, 8524 MiB and 7860 MiB spare of 16384 MiB (hits 38, misses 114, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-9e12af8424da908495c94617e1418695 prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=10.687 decode=0.054 elapsed=10.741 queued=0.000 tok/s=18.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 16384 tokens, resumed at 0, kept 1; 27 states, 8834 MiB and 7550 MiB spare of 16384 MiB (hits 38, misses 115, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-d9c5818d2fe275301feb747f0975a03f prompt=16384 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=10.693 decode=0.053 elapsed=10.746 queued=0.000 tok/s=18.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 16383 tokens, resumed at 0, kept 1; 28 states, 9144 MiB and 7240 MiB spare of 16384 MiB (hits 38, misses 116, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-548c155a08551b626f8b3a2433c3e0e1 prompt=16383 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=10.712 decode=0.055 elapsed=10.767 queued=0.000 tok/s=18.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32768 tokens, resumed at 0, kept 1; 29 states, 9658 MiB and 6726 MiB spare of 16384 MiB (hits 38, misses 117, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-0d417d37168c065b477e51478e5fce10 prompt=32768 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=21.403 decode=0.056 elapsed=21.459 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32766 tokens, resumed at 0, kept 1; 30 states, 10172 MiB and 6212 MiB spare of 16384 MiB (hits 38, misses 118, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-6afdc02d35cca45f0ae2ec1b903d986c prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=21.407 decode=0.054 elapsed=21.461 queued=0.000 tok/s=18.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 32766 tokens, resumed at 0, kept 1; 31 states, 10686 MiB and 5698 MiB spare of 16384 MiB (hits 38, misses 119, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-61df09e5368547e8eba30ef6ea1784dd prompt=32766 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=21.377 decode=0.054 elapsed=21.430 queued=0.000 tok/s=18.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 32 states, 11608 MiB and 4776 MiB spare of 16384 MiB (hits 38, misses 120, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-7e07eb3500540f35cce77d278a4cba6e prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=e52c3fd88437 finish=length rounds=1 accepted=0/1 ttft=43.388 decode=0.054 elapsed=43.443 queued=0.000 tok/s=18.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 33 states, 12530 MiB and 3854 MiB spare of 16384 MiB (hits 38, misses 121, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-1e14bc80e13a9fc7001492f5da3298a1 prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=8f5d1cf69c81 finish=length rounds=1 accepted=1/1 ttft=43.329 decode=0.055 elapsed=43.384 queued=0.000 tok/s=18.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 0, kept 1; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 122, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-29659a8f328bb45bb2664cdab39f38cd prompt=65536 cached=0 thinking=False effort=none tokens=2 sha=29c6afa4746e finish=length rounds=1 accepted=0/1 ttft=43.328 decode=0.055 elapsed=43.383 queued=0.000 tok/s=18.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 123, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-53383c86added440b888fbf139331ce7 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=7d62171119f8 finish=length rounds=61 accepted=66/101 ttft=1.243 decode=3.546 elapsed=4.789 queued=0.000 tok/s=35.8 seed=1000 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 124, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-afad6f064079343494d5a2d614658c24 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=c948b4bcbd58 finish=length rounds=70 accepted=57/88 ttft=1.249 decode=3.856 elapsed=5.105 queued=0.000 tok/s=32.9 seed=1001 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 125, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-7484ce59ca941cb2ec0617ba99a5adca prompt=2047 cached=0 thinking=False effort=none tokens=128 sha=6a2fb209181c finish=length rounds=71 accepted=56/87 ttft=1.245 decode=3.884 elapsed=5.129 queued=0.000 tok/s=32.7 seed=1002 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 126, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-ee27b4f3a7d44b6425a4589fe8bcf264 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=f8aaeecc59d0 finish=length rounds=58 accepted=69/103 ttft=1.245 decode=3.420 elapsed=4.665 queued=0.000 tok/s=37.1 seed=1003 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 127, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-b5ce3c1ad9f09448a88954aecb8e88eb prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=854ffaa5d759 finish=length rounds=76 accepted=51/94 ttft=1.249 decode=4.161 elapsed=5.410 queued=0.000 tok/s=30.5 seed=1004 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 128, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-dd7bdb59a416609111bc772e31bc48cb prompt=2047 cached=0 thinking=False effort=none tokens=128 sha=0effe3e2e261 finish=length rounds=52 accepted=75/106 ttft=1.247 decode=3.166 elapsed=4.413 queued=0.000 tok/s=40.1 seed=1005 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 38, misses 129, evicted 0, refused 0, failed 0) +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-692f006d96b5d330ad66796dfe7cdbaa prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=2d30fcc66f99 finish=length rounds=49 accepted=78/99 ttft=1.251 decode=2.993 elapsed=4.244 queued=0.000 tok/s=42.4 seed=1006 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 65536 tokens, resumed at 63488, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 129, evicted 0, refused 0, failed 0) +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2047 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +info: prompt cache: 2048 tokens, resumed at 0, kept 0; 34 states, 13452 MiB and 2932 MiB spare of 16384 MiB (hits 39, misses 136, evicted 0, refused 0, failed 0) +[tensorfold] done chatcmpl-9b4f049d2a40ee7047e87273961fce5f prompt=65536 cached=63488 thinking=False effort=none tokens=128 sha=37aa1a600539 finish=length rounds=50 accepted=77/140 ttft=1.448 decode=18.058 elapsed=19.506 queued=0.000 tok/s=7.0 seed=999 +[tensorfold] done chatcmpl-192ca4d20c34fcee5c93633da95a34d7 prompt=2047 cached=0 thinking=False effort=none tokens=128 sha=0effe3e2e261 finish=length rounds=44 accepted=83/131 ttft=2.594 decode=15.777 elapsed=18.371 queued=0.028 tok/s=8.0 seed=1005 +[tensorfold] done chatcmpl-f5aadc76ad6f9a5283117ddc509e8ca7 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=2d30fcc66f99 finish=length rounds=44 accepted=83/130 ttft=8.182 decode=10.952 elapsed=19.133 queued=0.028 tok/s=11.6 seed=1006 +[tensorfold] done chatcmpl-aeccb8de1d222352ce0220de31523ff3 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=7d62171119f8 finish=length rounds=48 accepted=80/141 ttft=3.974 decode=15.318 elapsed=19.292 queued=0.033 tok/s=8.3 seed=1000 +[tensorfold] done chatcmpl-1363c586a4d68c7a416c1e1dd96075cc prompt=2047 cached=0 thinking=False effort=none tokens=128 sha=6a2fb209181c finish=length rounds=52 accepted=75/152 ttft=1.270 decode=18.283 elapsed=19.553 queued=0.031 tok/s=6.9 seed=1002 +[tensorfold] done chatcmpl-03d5af26f2fc53c1bc46b321bb6f53e1 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=f8aaeecc59d0 finish=length rounds=46 accepted=81/135 ttft=9.623 decode=9.928 elapsed=19.551 queued=0.028 tok/s=12.8 seed=1003 +[tensorfold] done chatcmpl-490c595461f3d3590ebc493ebf4e82c0 prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=c948b4bcbd58 finish=length rounds=56 accepted=71/167 ttft=5.358 decode=14.878 elapsed=20.236 queued=0.032 tok/s=8.5 seed=1001 +info: kv pool idle: promised 0, free 3355 + store 0 == total 3355 +[tensorfold] done chatcmpl-d388c08fe0a7cf51fd91893f048744bf prompt=2048 cached=0 thinking=False effort=none tokens=128 sha=854ffaa5d759 finish=length rounds=65 accepted=62/176 ttft=6.763 decode=14.017 elapsed=20.780 queued=0.030 tok/s=9.1 seed=1004 +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +host-a:112:129 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1897 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:129 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1898 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:112:112 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/server-rank1.log new file mode 100644 index 0000000000..8bad6d7e5b --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/server-rank1.log @@ -0,0 +1,5575 @@ +[kvp-serve] 2026-10-23T01:23:05+00:00 rank 1 of 3 context 1048576 parallel 8 binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=unset TF_GLM_MM=unset args +info: rank 1: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-b:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:111:111 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:111:111 [0] NCCL INFO cudaDriverVersion 13030 +host-b:111:111 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:111:111 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:111:111 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:111:111 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:111:111 [0] NCCL INFO Count not open libibverbs.so +host-b:111:111 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:111:111 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:111:111 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:111:111 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x190e62a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:111:111 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x191279a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:111:111 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x191690a0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:111:111 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x191aa7a0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:111:111 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:111:111 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:111:111 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:111:111 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:111:111 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:111:111 [0] NCCL INFO Initialized NET plugin IB +host-b:111:111 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:111:111 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:111:111 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:111:111 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:111:111 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:111:111 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:111:111 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:111:111 [0] NCCL INFO Using network IB +host-b:111:111 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:111:111 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x164aebf0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9563dc80451a714b - Init START +host-b:111:111 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:111:111 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:111:111 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:111:111 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:111:111 [0] NCCL INFO Rank 1: 6 Net devices +host-b:111:111 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:111:111 [0] NCCL INFO comm 0x164aebf0 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:111:111 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:111:111 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:111:111 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:111:111 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:111:111 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:111:126 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-b:111:126 [0] NCCL INFO proxy listening socket at <37963> +host-b:111:127 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-b:111:128 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-b:111:111 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:111:111 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:111:111 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:111:111 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:111:111 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:111:111 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:111:111 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:111:111 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad54010000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad54054000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad5409c000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2140 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2141 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2140 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2140 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2141 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2141 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6684 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6685 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6686 pkey=0 pd=0xf4ad540f2420 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2268 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2269 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2268 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d958b ctsFifoLkey=0x1d958b +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2268 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2269 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d958b ctsFifoLkey=0x1d958b +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2269 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad5422e000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2140 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2141 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6716 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6717 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6718 pkey=0 pd=0xf4ad5400f380 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad542f8000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad5433c000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad54384000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad543c8000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6719 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6720 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6721 pkey=0 pd=0xf4ad5400f380 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2268 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2269 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2142 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2143 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2142 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b4f4c ctsFifoLkey=0x1b4f4c +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2142 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2143 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b4f4c ctsFifoLkey=0x1b4f4c +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2143 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2270 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2271 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2270 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1dafa9 ctsFifoLkey=0x1dafa9 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2270 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2271 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1dafa9 ctsFifoLkey=0x1dafa9 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2271 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6687 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6688 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6689 pkey=0 pd=0xf4ad540f2420 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2142 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2143 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2270 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2271 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:111 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:111:111 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:111:111 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:111:111 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:111:111 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad545b2000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad545f7000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad5463c000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad54681000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2144 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2145 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2146 pkey=0 pd=0xf4ad54053590 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2272 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2273 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2274 pkey=0 pd=0xf4ad540f2f90 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2147 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2148 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2149 pkey=0 pd=0xf4ad54053590 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2275 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2276 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2277 pkey=0 pd=0xf4ad540f2f90 +host-b:111:111 [0] NCCL INFO Connected all trees +host-b:111:111 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:111:111 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:111:111 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:111:111 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad548a4000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad548e5000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6690 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6691 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6690 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1726e5 ctsFifoLkey=0x1726e5 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6690 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6691 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1726e5 ctsFifoLkey=0x1726e5 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6691 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6722 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6723 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6722 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6722 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6723 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6723 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad549df000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad54a23000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6690 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6691 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6722 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6723 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6692 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6693 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6692 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6692 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6693 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17a15e ctsFifoLkey=0x17a15e +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6693 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6724 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6725 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6724 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19e6e5 ctsFifoLkey=0x19e6e5 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6724 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6725 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19e6e5 ctsFifoLkey=0x19e6e5 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6725 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6692 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6693 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6724 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6725 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:111 [0] NCCL INFO Connected binomial trees +host-b:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:111:111 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:111:111 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:111:111 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:111:111 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-b:111:111 [0] NCCL INFO ncclCommInitRank comm 0x164aebf0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9563dc80451a714b - Init COMPLETE +host-b:111:111 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 1.59 (kernels 0.25, alloc 0.03, bootstrap 1.05, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.25, rest 0.00) +host-b:111:111 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:111:111 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:111:111 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:111:111 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:111:111 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:111:111 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:111:111 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:111:111 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad54b19000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad54b5a000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad54b9f000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad54be3000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad54c2b000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad54c6f000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf4ad54cb7000) +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf4ad54cfb000) +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2150 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2151 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2150 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2150 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2151 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bc9c6 ctsFifoLkey=0x1bc9c6 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2151 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6694 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6695 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6696 pkey=0 pd=0xf4ad540f2420 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2278 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2279 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2278 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de9e6 ctsFifoLkey=0x1de9e6 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2278 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2279 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1de9e6 ctsFifoLkey=0x1de9e6 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2279 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6726 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6727 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6728 pkey=0 pd=0xf4ad5400f380 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2150 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2151 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2278 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2279 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2280 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2281 pkey=0 pd=0xf4ad540f2f90 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2282 pkey=0 pd=0xf4ad540f2f90 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6697 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6698 pkey=0 pd=0xf4ad540f2420 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6697 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x16bd77 ctsFifoLkey=0x16bd77 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6697 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6698 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x16bd77 ctsFifoLkey=0x16bd77 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6698 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2152 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2153 pkey=0 pd=0xf4ad54053590 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2154 pkey=0 pd=0xf4ad54053590 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6729 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6730 pkey=0 pd=0xf4ad5400f380 oooRq=0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6729 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x192f2a ctsFifoLkey=0x192f2a +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6729 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6730 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x192f2a ctsFifoLkey=0x192f2a +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6730 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6697 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6698 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 1: rank 0 ended; leaving +2:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6729 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6730 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:111:111 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/server-rank2.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/server-rank2.log new file mode 100644 index 0000000000..6a2d338bc1 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6auto/server-rank2.log @@ -0,0 +1,1656 @@ +[kvp-serve] 2026-10-23T01:23:05+00:00 rank 2 of 3 context 1048576 parallel 8 binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=unset TF_GLM_MM=unset args +info: rank 2: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:111:111 [0] NCCL INFO cudaDriverVersion 13030 +host-c:111:111 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:111:111 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:111:111 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:111:111 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:111:111 [0] NCCL INFO Count not open libibverbs.so +host-c:111:111 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:111:111 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:111:111 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:111:111 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x247ba2a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:111:111 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x247fb9a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:111:111 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x2483d0a0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:111:111 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x2487e7a0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:111:111 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:111:111 [0] NCCL INFO Initialized NET plugin IB +host-c:111:111 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:111:111 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:111:111 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:111:111 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:111:111 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:111:111 [0] NCCL INFO Using network IB +host-c:111:111 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:111:111 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x21b82bf0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9563dc80451a714b - Init START +host-c:111:111 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:111:111 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:111:111 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:111:111 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:111:111 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:111:111 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:111:111 [0] NCCL INFO Rank 2: 6 Net devices +host-c:111:111 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:111:111 [0] NCCL INFO comm 0x21b82bf0 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:111:111 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:111:111 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:111:111 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:111:111 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:111:126 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-c:111:126 [0] NCCL INFO proxy listening socket at <55889> +host-c:111:127 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-c:111:128 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-c:111:111 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456c010000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456c054000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456c09c000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456c0e0000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456c128000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456c16c000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456c1b4000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456c1f8000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2140 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2141 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2140 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2140 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2141 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2141 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1884 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1885 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1886 pkey=0 pd=0xf5456c09bf70 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2268 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2269 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2268 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2268 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2269 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2269 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2142 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2143 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2142 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bd4d1 ctsFifoLkey=0x1bd4d1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2142 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2143 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bd4d1 ctsFifoLkey=0x1bd4d1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2143 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2270 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2271 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2270 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3f41 ctsFifoLkey=0x1d3f41 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2270 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2271 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3f41 ctsFifoLkey=0x1d3f41 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2271 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2140 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2141 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2268 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2269 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2012 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2013 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2014 pkey=0 pd=0xf5456c3fadd0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2142 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2143 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2270 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2271 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1887 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1888 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1889 pkey=0 pd=0xf5456c09bf70 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2015 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2016 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2017 pkey=0 pd=0xf5456c3fadd0 +host-c:111:111 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:111:111 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:111:111 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:111:111 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456c5ac000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456c5ed000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456c632000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1890 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1891 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1890 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17d08d ctsFifoLkey=0x17d08d +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1890 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1891 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17d08d ctsFifoLkey=0x17d08d +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1891 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2144 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2145 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2146 pkey=0 pd=0xf5456c053eb0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2018 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2019 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2018 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2018 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2019 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197e7c ctsFifoLkey=0x197e7c +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2019 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456c7a4000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456c7ec000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1890 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1891 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2018 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2019 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2272 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2273 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2274 pkey=0 pd=0xf5456c283e70 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1892 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1893 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1892 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17c683 ctsFifoLkey=0x17c683 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1892 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1893 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17c683 ctsFifoLkey=0x17c683 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1893 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456c8fd000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456c945000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456c986000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1892 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1893 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2147 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2148 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2149 pkey=0 pd=0xf5456c053eb0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2020 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2021 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2020 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2020 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2021 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2021 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2275 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2276 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2277 pkey=0 pd=0xf5456c283e70 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2020 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2021 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:111 [0] NCCL INFO Connected all trees +host-c:111:111 [0] NCCL INFO Connected binomial trees +host-c:111:111 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:111:111 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:111:111 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:111:111 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:111:111 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-c:111:111 [0] NCCL INFO ncclCommInitRank comm 0x21b82bf0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9563dc80451a714b - Init COMPLETE +host-c:111:111 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.48 (kernels 0.24, alloc 0.04, bootstrap 0.00, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.19, rest 0.00) +host-c:111:111 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:111:111 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:111:111 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:111:111 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:111:111 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:111:111 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:111:111 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:111:111 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456cb19000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456cb5a000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456cb9f000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2150 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2151 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2150 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bd6d3 ctsFifoLkey=0x1bd6d3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2150 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2151 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bd6d3 ctsFifoLkey=0x1bd6d3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2151 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456cc3e000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456cc86000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456ccca000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf5456cd12000) +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf5456cd56000) +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1894 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1895 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1896 pkey=0 pd=0xf5456c09bf70 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2150 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2151 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2278 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2279 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2278 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5e5e ctsFifoLkey=0x1d5e5e +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2278 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2279 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d5e5e ctsFifoLkey=0x1d5e5e +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2279 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2022 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2023 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2024 pkey=0 pd=0xf5456c3fadd0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2025 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2026 pkey=0 pd=0xf5456c3fadd0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2025 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19736b ctsFifoLkey=0x19736b +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2025 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2026 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19736b ctsFifoLkey=0x19736b +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2026 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1897 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1898 pkey=0 pd=0xf5456c09bf70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1897 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1897 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1898 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17f2af ctsFifoLkey=0x17f2af +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1898 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2278 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2279 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2025 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2026 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1897 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1898 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2280 pkey=0 pd=0xf5456c283e70 oooRq=0 +info: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 2: rank 0 ended; leaving +3:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2281 pkey=0 pd=0xf5456c283e70 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2282 pkey=0 pd=0xf5456c283e70 +host-c:111:126 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2152 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2153 pkey=0 pd=0xf5456c053eb0 oooRq=0 +host-c:111:126 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2154 pkey=0 pd=0xf5456c053eb0 +host-c:111:111 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/client.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/client.log new file mode 100644 index 0000000000..9bdaf43fe2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/client.log @@ -0,0 +1,45 @@ +server up 2026-10-23 11:09:30 + 0 S1 t1 drafted prompt 7759 cached 0 ttft 5.11s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 1 S1 t1 plain prompt 7759 cached 0 ttft 5.06s sha f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595 + 2 S1 t2 drafted prompt 15550 cached 0 ttft 10.21s sha 539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b + 3 S1 t2 plain prompt 15550 cached 0 ttft 10.27s sha 539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b + 4 S1 t3 drafted prompt 19462 cached 0 ttft 12.89s sha 8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa + 5 S1 t3 plain prompt 19462 cached 0 ttft 12.76s sha 8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa + 6 S1 t4 drafted prompt 21892 cached 0 ttft 14.37s sha dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af + 7 S1 t4 plain prompt 21892 cached 0 ttft 14.33s sha dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af + 8 S1 t5 drafted prompt 24388 cached 0 ttft 15.83s sha 8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa + 9 S1 t5 plain prompt 24388 cached 0 ttft 15.78s sha 8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa + 10 S1 t6 drafted prompt 26895 cached 0 ttft 17.71s sha c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1 + 11 S1 t6 plain prompt 26895 cached 0 ttft 17.73s sha c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1 + 12 S1 t7 drafted prompt 29798 cached 0 ttft 19.49s sha 7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb + 13 S1 t7 plain prompt 29798 cached 0 ttft 19.45s sha 7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb + 14 S1 t8 drafted prompt 35434 cached 0 ttft 23.25s sha 1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e + 15 S1 t8 plain prompt 35434 cached 0 ttft 23.23s sha 1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e + 16 S1 t9 drafted prompt 38152 cached 0 ttft 24.94s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 + 17 S1 t9 plain prompt 38152 cached 0 ttft 24.90s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 + 18 X t1 drafted prompt 2115 cached 0 ttft 1.48s sha 7b1461bf71da95a2b74977aec25ecf7215b9f0fa044c82d2e1f4905ed66b798a + 19 A t1 drafted prompt 7412 cached 0 ttft 4.96s sha d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666 + 20 A t1 plain prompt 7412 cached 0 ttft 4.92s sha d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666 + 21 B t1 drafted prompt 7474 cached 0 ttft 4.98s sha 9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb + 22 B t1 plain prompt 7474 cached 0 ttft 4.96s sha 9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb + 23 A t2 drafted prompt 12632 cached 0 ttft 8.47s sha e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe + 24 A t2 plain prompt 12632 cached 0 ttft 8.45s sha e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe + 25 B t2 drafted prompt 12721 cached 0 ttft 8.48s sha f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2 + 26 B t2 plain prompt 12721 cached 0 ttft 8.48s sha f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2 + 27 A t3 drafted prompt 15383 cached 0 ttft 10.13s sha 7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8 + 28 A t3 plain prompt 15383 cached 0 ttft 10.08s sha 7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8 + 29 B t3 drafted prompt 17330 cached 0 ttft 11.41s sha 8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49 + 30 B t3 plain prompt 17330 cached 0 ttft 11.38s sha 8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49 + 31 E t1 drafted prompt 40901 cached 0 ttft 26.65s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 + 32 E t1 plain prompt 40901 cached 0 ttft 26.60s sha 6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3 + 33 T t1 drafted prompt 1520 cached 0 ttft 0.92s sha 033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89 + 34 T t1 plain prompt 1520 cached 0 ttft 0.89s sha 033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89 + 35 T t2 drafted prompt 4202 cached 0 ttft 3.04s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 36 T t2 plain prompt 4202 cached 0 ttft 3.03s sha a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa + 38 C t1 drafted prompt 9702 cached 0 ttft 6.40s sha 88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c + 39 C t1 plain prompt 9702 cached 0 ttft 6.34s sha 88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c + 40 C t2 drafted prompt 15180 cached 0 ttft 10.02s sha 5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8 + 41 C t2 plain prompt 15180 cached 0 ttft 10.00s sha 5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8 + 42 A t4 drafted prompt 18246 cached 0 ttft 11.86s sha 9599ff0fa9b19bbb5cb5218a5b6f538de1146ce12dad36c729686f12f666389e + 43 B t4 drafted prompt 19808 cached 0 ttft 26.47s sha 025c4856c697674513547e9be3c33a84537e8a4fcb74e731ea09263af3e6d80d +=== kvp-client r6off rc 0 diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/compare-auto.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/compare-auto.log new file mode 100644 index 0000000000..2cea5bd1b8 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/compare-auto.log @@ -0,0 +1,84 @@ + 0 S1 t1 drafted SAME + 1 S1 t1 plain SAME + 2 S1 t2 drafted SAME + 3 S1 t2 plain SAME + 4 S1 t3 drafted SAME + 5 S1 t3 plain SAME + 6 S1 t4 drafted SAME + 7 S1 t4 plain SAME + 8 S1 t5 drafted SAME + 9 S1 t5 plain SAME + 10 S1 t6 drafted SAME + 11 S1 t6 plain SAME + 12 S1 t7 drafted SAME + 13 S1 t7 plain SAME + 14 S1 t8 drafted SAME + 15 S1 t8 plain SAME + 16 S1 t9 drafted SAME + 17 S1 t9 plain SAME + 18 X t1 drafted SAME + 19 A t1 drafted SAME + 20 A t1 plain SAME + 21 B t1 drafted SAME + 22 B t1 plain SAME + 23 A t2 drafted SAME + 24 A t2 plain SAME + 25 B t2 drafted SAME + 26 B t2 plain SAME + 27 A t3 drafted SAME + 28 A t3 plain SAME + 29 B t3 drafted SAME + 30 B t3 plain SAME + 31 E t1 drafted SAME + 32 E t1 plain SAME + 33 T t1 drafted SAME + 34 T t1 plain SAME + 35 T t2 drafted SAME + 36 T t2 plain SAME + 38 C t1 drafted SAME + 39 C t1 plain SAME + 40 C t2 drafted SAME + 41 C t2 plain SAME + 42 A t4 drafted SAME + 43 B t4 drafted SAME +drafted == plain on the same messages: 23/23 + S1 t1 plain prompt 7759 cached 6144: ttft 1.02 s vs fresh 5.06 s ( 4.9x) + S1 t2 drafted prompt 15550 cached 6144: ttft 6.15 s vs fresh 10.21 s ( 1.7x) + S1 t2 plain prompt 15550 cached 14336: ttft 0.83 s vs fresh 10.27 s ( 12.4x) + S1 t3 drafted prompt 19462 cached 14336: ttft 3.44 s vs fresh 12.89 s ( 3.7x) + S1 t3 plain prompt 19462 cached 18432: ttft 0.75 s vs fresh 12.76 s ( 17.1x) + S1 t4 drafted prompt 21892 cached 18432: ttft 2.32 s vs fresh 14.37 s ( 6.2x) + S1 t4 plain prompt 21892 cached 20480: ttft 0.93 s vs fresh 14.33 s ( 15.4x) + S1 t5 drafted prompt 24388 cached 20480: ttft 2.55 s vs fresh 15.83 s ( 6.2x) + S1 t5 plain prompt 24388 cached 22528: ttft 1.15 s vs fresh 15.78 s ( 13.7x) + S1 t6 drafted prompt 26895 cached 22528: ttft 3.12 s vs fresh 17.71 s ( 5.7x) + S1 t6 plain prompt 26895 cached 26624: ttft 0.40 s vs fresh 17.73 s ( 44.0x) + S1 t7 drafted prompt 29798 cached 26624: ttft 2.22 s vs fresh 19.49 s ( 8.8x) + S1 t7 plain prompt 29798 cached 28672: ttft 0.82 s vs fresh 19.45 s ( 23.6x) + S1 t8 drafted prompt 35434 cached 28672: ttft 4.66 s vs fresh 23.25 s ( 5.0x) + S1 t8 plain prompt 35434 cached 34816: ttft 0.58 s vs fresh 23.23 s ( 40.0x) + S1 t9 drafted prompt 38152 cached 34816: ttft 2.31 s vs fresh 24.94 s ( 10.8x) + S1 t9 plain prompt 38152 cached 36864: ttft 0.91 s vs fresh 24.90 s ( 27.5x) + A t1 drafted prompt 7412 cached 6144: ttft 0.90 s vs fresh 4.96 s ( 5.5x) + A t1 plain prompt 7412 cached 6144: ttft 0.86 s vs fresh 4.92 s ( 5.7x) + B t1 drafted prompt 7474 cached 6144: ttft 0.93 s vs fresh 4.98 s ( 5.4x) + B t1 plain prompt 7474 cached 6144: ttft 0.88 s vs fresh 4.96 s ( 5.6x) + A t2 drafted prompt 12632 cached 6144: ttft 4.42 s vs fresh 8.47 s ( 1.9x) + A t2 plain prompt 12632 cached 12288: ttft 0.43 s vs fresh 8.45 s ( 19.5x) + B t2 drafted prompt 12721 cached 6144: ttft 4.44 s vs fresh 8.48 s ( 1.9x) + B t2 plain prompt 12721 cached 12288: ttft 0.47 s vs fresh 8.48 s ( 18.2x) + A t3 drafted prompt 15383 cached 12288: ttft 2.13 s vs fresh 10.13 s ( 4.8x) + A t3 plain prompt 15383 cached 14336: ttft 0.76 s vs fresh 10.08 s ( 13.3x) + B t3 drafted prompt 17330 cached 12288: ttft 3.42 s vs fresh 11.41 s ( 3.3x) + B t3 plain prompt 17330 cached 16384: ttft 0.72 s vs fresh 11.38 s ( 15.9x) + E t1 drafted prompt 40901 cached 6144: ttft 22.65 s vs fresh 26.65 s ( 1.2x) + E t1 plain prompt 40901 cached 38912: ttft 1.26 s vs fresh 26.60 s ( 21.1x) + T t2 plain prompt 4202 cached 4096: ttft 0.27 s vs fresh 3.03 s ( 11.1x) + C t1 drafted prompt 9702 cached 6144: ttft 2.39 s vs fresh 6.40 s ( 2.7x) + C t1 plain prompt 9702 cached 8192: ttft 0.97 s vs fresh 6.34 s ( 6.5x) + C t2 drafted prompt 15180 cached 8192: ttft 4.68 s vs fresh 10.02 s ( 2.1x) + C t2 plain prompt 15180 cached 14336: ttft 0.68 s vs fresh 10.00 s ( 14.8x) + A t4 drafted prompt 18246 cached 14336: ttft 2.54 s vs fresh 11.86 s ( 4.7x) + B t4 drafted prompt 19808 cached 16384: ttft 4.97 s vs fresh 26.47 s ( 5.3x) +PASS: 43/43 replies equal and as cached as planned +timing: 0 resumed replies no sooner than fresh diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/fresh.jsonl b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/fresh.jsonl new file mode 100644 index 0000000000..cd0acf9b67 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/fresh.jsonl @@ -0,0 +1,43 @@ +{"i": 0, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.106426489, "prefill": 5.101740784, "wall": 6.756380434031598, "finish": "tool_calls"}} +{"i": 1, "got": {"sha": "f0f55b35a3041a6716cb165a5123890e5529def20f710539eabeab0f645d5595", "cached": 0, "prompt": 7759, "tokens": 61, "ttft": 5.063107714, "prefill": 5.061697141, "wall": 7.804028348065913, "finish": "tool_calls"}} +{"i": 2, "got": {"sha": "539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b", "cached": 0, "prompt": 15550, "tokens": 46, "ttft": 10.209161529, "prefill": 10.205303687, "wall": 11.072627502959222, "finish": "tool_calls"}} +{"i": 3, "got": {"sha": "539973cb8b1338012688798a31f9be73a16145d5519666f26482b6ca9dba075b", "cached": 0, "prompt": 15550, "tokens": 46, "ttft": 10.274763748, "prefill": 10.272817635, "wall": 12.348692246945575, "finish": "tool_calls"}} +{"i": 4, "got": {"sha": "8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa", "cached": 0, "prompt": 19462, "tokens": 96, "ttft": 12.89218201, "prefill": 12.889346401, "wall": 15.22256526502315, "finish": "tool_calls"}} +{"i": 5, "got": {"sha": "8fc3c26162f14ba8483ee67abcd4150e6796fd196dc5725076a025830aff7eaa", "cached": 0, "prompt": 19462, "tokens": 96, "ttft": 12.757421913, "prefill": 12.754934372, "wall": 17.198896247078665, "finish": "tool_calls"}} +{"i": 6, "got": {"sha": "dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af", "cached": 0, "prompt": 21892, "tokens": 113, "ttft": 14.367776716, "prefill": 14.36324658, "wall": 17.785851689986885, "finish": "tool_calls"}} +{"i": 7, "got": {"sha": "dedd451779f3b6be763b095da54f5ca69962325e26c090a61a60fa296cdd03af", "cached": 0, "prompt": 21892, "tokens": 113, "ttft": 14.326833841, "prefill": 14.323725398, "wall": 19.444774080999196, "finish": "tool_calls"}} +{"i": 8, "got": {"sha": "8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa", "cached": 0, "prompt": 24388, "tokens": 160, "ttft": 15.830377489, "prefill": 15.825365381, "wall": 20.417026109062135, "finish": "length"}} +{"i": 9, "got": {"sha": "8bcbd355dc0446c05f07f16f95e98d734d0a227a0addadb1b85b2a904f8e77aa", "cached": 0, "prompt": 24388, "tokens": 160, "ttft": 15.784174286, "prefill": 15.780350156, "wall": 23.04665946692694, "finish": "length"}} +{"i": 10, "got": {"sha": "c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1", "cached": 0, "prompt": 26895, "tokens": 154, "ttft": 17.708142435, "prefill": 17.702577042, "wall": 22.28615393396467, "finish": "tool_calls"}} +{"i": 11, "got": {"sha": "c8e79afc5a1062b02819420d6496258462263a9f609485bdcc31da6855267af1", "cached": 0, "prompt": 26895, "tokens": 154, "ttft": 17.725022117, "prefill": 17.720505886, "wall": 24.684802685980685, "finish": "tool_calls"}} +{"i": 12, "got": {"sha": "7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb", "cached": 0, "prompt": 29798, "tokens": 136, "ttft": 19.490247247, "prefill": 19.483498723, "wall": 23.221443012007512, "finish": "tool_calls"}} +{"i": 13, "got": {"sha": "7629aa6bf5dfeea920770da75f63b6ab1fc010cd0b669ec22529cee049e38efb", "cached": 0, "prompt": 29798, "tokens": 136, "ttft": 19.446495225, "prefill": 19.441928304, "wall": 25.611674760002643, "finish": "tool_calls"}} +{"i": 14, "got": {"sha": "1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e", "cached": 0, "prompt": 35434, "tokens": 160, "ttft": 23.246930796, "prefill": 23.239967678, "wall": 27.06529931491241, "finish": "length"}} +{"i": 15, "got": {"sha": "1adac255ff32acc04618c1f9db94e931b1e9cb783c4e441a21a9ba32b740da6e", "cached": 0, "prompt": 35434, "tokens": 160, "ttft": 23.230126945, "prefill": 23.224191517, "wall": 30.4882349279942, "finish": "length"}} +{"i": 16, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 0, "prompt": 38152, "tokens": 160, "ttft": 24.941843347, "prefill": 24.93418424, "wall": 27.324622172978707, "finish": "length"}} +{"i": 17, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 0, "prompt": 38152, "tokens": 160, "ttft": 24.9041245, "prefill": 24.897083862, "wall": 32.18161984800827, "finish": "length"}} +{"i": 18, "got": {"sha": "7b1461bf71da95a2b74977aec25ecf7215b9f0fa044c82d2e1f4905ed66b798a", "cached": 0, "prompt": 2115, "tokens": 160, "ttft": 1.481300764, "prefill": 1.478996857, "wall": 5.783782757003792, "finish": "length"}} +{"i": 19, "got": {"sha": "d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666", "cached": 0, "prompt": 7412, "tokens": 90, "ttft": 4.958550845, "prefill": 4.956710796, "wall": 7.139850158942863, "finish": "tool_calls"}} +{"i": 20, "got": {"sha": "d0573eef060977d98da46af682ca71f30b35a155da5be22aa3baecca9ffc9666", "cached": 0, "prompt": 7412, "tokens": 90, "ttft": 4.920046886, "prefill": 4.918956541, "wall": 8.983381281956099, "finish": "tool_calls"}} +{"i": 21, "got": {"sha": "9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb", "cached": 0, "prompt": 7474, "tokens": 73, "ttft": 4.980821239, "prefill": 4.978943175, "wall": 6.557870409102179, "finish": "tool_calls"}} +{"i": 22, "got": {"sha": "9a2685e9905fa6973f8dbdf29d644d01cab9b75737d07335885b31808eaad9fb", "cached": 0, "prompt": 7474, "tokens": 73, "ttft": 4.956667008, "prefill": 4.955263443, "wall": 8.244590871967375, "finish": "tool_calls"}} +{"i": 23, "got": {"sha": "e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe", "cached": 0, "prompt": 12632, "tokens": 160, "ttft": 8.472915399, "prefill": 8.469282999, "wall": 12.759403006057255, "finish": "length"}} +{"i": 24, "got": {"sha": "e1f2511e60f396ff14ef731907b19803773915834a46d457301acf811d384dfe", "cached": 0, "prompt": 12632, "tokens": 160, "ttft": 8.452488536, "prefill": 8.450597143, "wall": 15.656728595960885, "finish": "length"}} +{"i": 25, "got": {"sha": "f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2", "cached": 0, "prompt": 12721, "tokens": 117, "ttft": 8.484791045, "prefill": 8.481465928, "wall": 11.221482279943302, "finish": "tool_calls"}} +{"i": 26, "got": {"sha": "f202ab6075aea8369fa69b3aa1d7872cf7a0444e28ea89b871f4593ef4609ca2", "cached": 0, "prompt": 12721, "tokens": 117, "ttft": 8.47596763, "prefill": 8.474037038, "wall": 13.74378959799651, "finish": "tool_calls"}} +{"i": 27, "got": {"sha": "7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8", "cached": 0, "prompt": 15383, "tokens": 137, "ttft": 10.131604108, "prefill": 10.127291894, "wall": 14.008070027921349, "finish": "tool_calls"}} +{"i": 28, "got": {"sha": "7695627844cdcc276b656d8e0c654fd9201331a6d8f02d4892bf4b04a5d549f8", "cached": 0, "prompt": 15383, "tokens": 137, "ttft": 10.084548065, "prefill": 10.081883609, "wall": 16.267391424975358, "finish": "tool_calls"}} +{"i": 29, "got": {"sha": "8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49", "cached": 0, "prompt": 17330, "tokens": 160, "ttft": 11.410527864, "prefill": 11.406242482, "wall": 15.458911090972833, "finish": "length"}} +{"i": 30, "got": {"sha": "8b15f58457e8e1e15a72bbbe2fbfbfa473b6503e6726a86003d24df79009de49", "cached": 0, "prompt": 17330, "tokens": 160, "ttft": 11.377691624, "prefill": 11.374987441, "wall": 18.600214232923463, "finish": "length"}} +{"i": 31, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 0, "prompt": 40901, "tokens": 160, "ttft": 26.654293517, "prefill": 26.645313214, "wall": 28.493615171988495, "finish": "length"}} +{"i": 32, "got": {"sha": "6fb6a6ab70d0834a329f5ce9263d1fddc174959ad7839578fbeb4a923bd285c3", "cached": 0, "prompt": 40901, "tokens": 160, "ttft": 26.600624457, "prefill": 26.594017359, "wall": 33.9103669479955, "finish": "length"}} +{"i": 33, "got": {"sha": "033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89", "cached": 0, "prompt": 1520, "tokens": 150, "ttft": 0.922774365, "prefill": 0.921150814, "wall": 4.698226289008744, "finish": "stop"}} +{"i": 34, "got": {"sha": "033a197da586db255fee4cdbf2d9ce5937da0137c9ffa99fe75ec35f893aae89", "cached": 0, "prompt": 1520, "tokens": 150, "ttft": 0.886860897, "prefill": 0.886178683, "wall": 7.4900314880069345, "finish": "stop"}} +{"i": 35, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4202, "tokens": 12, "ttft": 3.04261404, "prefill": 3.040071138, "wall": 3.2888425670098513, "finish": "stop"}} +{"i": 36, "got": {"sha": "a94919711db84742c30e2de91de5f8dcca3dcb8c55965a6c6ae08f7fb72592fa", "cached": 0, "prompt": 4202, "tokens": 12, "ttft": 3.030621741, "prefill": 3.02934157, "wall": 3.5415961580583826, "finish": "stop"}} +{"i": 38, "got": {"sha": "88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c", "cached": 0, "prompt": 9702, "tokens": 110, "ttft": 6.396371812, "prefill": 6.390369183, "wall": 9.148898824932985, "finish": "tool_calls"}} +{"i": 39, "got": {"sha": "88f98460968aa13c4baf13ac8cd3d60277e3cf4641b79f641b67398dda514d0c", "cached": 0, "prompt": 9702, "tokens": 110, "ttft": 6.341832898, "prefill": 6.340436885, "wall": 11.305321355932392, "finish": "tool_calls"}} +{"i": 40, "got": {"sha": "5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8", "cached": 0, "prompt": 15180, "tokens": 160, "ttft": 10.021166666, "prefill": 10.017358249, "wall": 14.31559655093588, "finish": "length"}} +{"i": 41, "got": {"sha": "5694b66a89958146e8f8e5a989c1bab8c538f7dad576c43f3cafd47675f84ab8", "cached": 0, "prompt": 15180, "tokens": 160, "ttft": 9.998888878, "prefill": 9.99626279, "wall": 17.212868075002916, "finish": "length"}} +{"i": 42, "got": {"sha": "9599ff0fa9b19bbb5cb5218a5b6f538de1146ce12dad36c729686f12f666389e", "cached": 0, "prompt": 18246, "tokens": 160, "ttft": 11.855061837, "prefill": 11.850043426, "wall": 31.68501882988494, "finish": "length"}} +{"i": 43, "got": {"sha": "025c4856c697674513547e9be3c33a84537e8a4fcb74e731ea09263af3e6d80d", "cached": 0, "prompt": 19808, "tokens": 141, "ttft": 26.466463677, "prefill": 25.736744865, "wall": 31.282331284950487, "finish": "tool_calls"}} diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/hostwatch-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/hostwatch-rank0.log new file mode 100644 index 0000000000..f0ad6ba7a2 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/hostwatch-rank0.log @@ -0,0 +1,452 @@ +11:05:30 114 +11:05:32 112 +11:05:34 112 +11:05:36 112 +11:05:38 112 +11:05:40 110 +11:05:42 110 +11:05:44 109 +11:05:46 109 +11:05:48 109 +11:05:50 107 +11:05:52 107 +11:05:54 106 +11:05:56 106 +11:05:58 105 +11:06:00 104 +11:06:02 104 +11:06:05 103 +11:06:07 103 +11:06:09 102 +11:06:11 102 +11:06:13 102 +11:06:15 101 +11:06:17 100 +11:06:19 100 +11:06:21 99 +11:06:23 99 +11:06:25 98 +11:06:27 97 +11:06:29 97 +11:06:31 97 +11:06:33 96 +11:06:35 96 +11:06:37 94 +11:06:39 94 +11:06:41 92 +11:06:44 92 +11:06:46 92 +11:06:48 92 +11:06:50 91 +11:06:52 91 +11:06:54 90 +11:06:56 90 +11:06:58 90 +11:07:00 88 +11:07:02 88 +11:07:04 87 +11:07:06 87 +11:07:08 87 +11:07:10 86 +11:07:12 86 +11:07:14 84 +11:07:16 84 +11:07:18 84 +11:07:21 83 +11:07:23 83 +11:07:25 81 +11:07:27 81 +11:07:29 80 +11:07:31 80 +11:07:33 80 +11:07:35 80 +11:07:37 78 +11:07:39 78 +11:07:41 77 +11:07:43 77 +11:07:45 77 +11:07:47 76 +11:07:49 76 +11:07:51 75 +11:07:53 74 +11:07:55 74 +11:07:58 74 +11:08:00 73 +11:08:02 73 +11:08:04 71 +11:08:06 71 +11:08:08 71 +11:08:10 70 +11:08:12 70 +11:08:14 69 +11:08:16 68 +11:08:18 68 +11:08:20 68 +11:08:22 66 +11:08:24 66 +11:08:26 65 +11:08:28 65 +11:08:30 64 +11:08:32 64 +11:08:34 64 +11:08:37 62 +11:08:39 62 +11:08:41 61 +11:08:43 61 +11:08:45 61 +11:08:47 61 +11:08:49 60 +11:08:51 59 +11:08:53 58 +11:08:55 58 +11:08:57 58 +11:08:59 57 +11:09:01 57 +11:09:03 57 +11:09:05 55 +11:09:07 55 +11:09:09 54 +11:09:11 54 +11:09:14 54 +11:09:16 52 +11:09:18 50 +11:09:20 50 +11:09:22 50 +11:09:24 50 +11:09:26 50 +11:09:28 39 +11:09:30 37 +11:09:32 37 +11:09:34 37 +11:09:36 37 +11:09:38 37 +11:09:40 37 +11:09:42 37 +11:09:44 37 +11:09:46 37 +11:09:49 37 +11:09:51 37 +11:09:53 37 +11:09:55 37 +11:09:57 37 +11:09:59 37 +11:10:01 37 +11:10:03 37 +11:10:05 37 +11:10:07 37 +11:10:09 37 +11:10:11 37 +11:10:13 37 +11:10:15 37 +11:10:17 37 +11:10:19 37 +11:10:22 37 +11:10:24 37 +11:10:26 37 +11:10:28 37 +11:10:30 37 +11:10:32 37 +11:10:34 37 +11:10:36 37 +11:10:38 37 +11:10:40 37 +11:10:42 37 +11:10:44 37 +11:10:46 37 +11:10:48 37 +11:10:50 37 +11:10:52 37 +11:10:54 37 +11:10:57 37 +11:10:59 37 +11:11:01 37 +11:11:03 37 +11:11:05 37 +11:11:07 37 +11:11:09 37 +11:11:11 37 +11:11:13 37 +11:11:15 37 +11:11:17 37 +11:11:19 37 +11:11:21 37 +11:11:23 37 +11:11:25 37 +11:11:27 37 +11:11:30 37 +11:11:32 37 +11:11:34 37 +11:11:36 37 +11:11:38 37 +11:11:40 37 +11:11:42 37 +11:11:44 37 +11:11:46 37 +11:11:48 37 +11:11:50 37 +11:11:52 37 +11:11:54 37 +11:11:56 37 +11:11:58 37 +11:12:01 37 +11:12:03 37 +11:12:05 37 +11:12:07 37 +11:12:09 37 +11:12:11 37 +11:12:13 37 +11:12:15 37 +11:12:17 37 +11:12:19 37 +11:12:21 37 +11:12:23 37 +11:12:25 37 +11:12:27 37 +11:12:29 37 +11:12:31 37 +11:12:33 37 +11:12:36 37 +11:12:38 37 +11:12:40 37 +11:12:42 37 +11:12:44 37 +11:12:46 37 +11:12:48 37 +11:12:50 37 +11:12:52 37 +11:12:54 37 +11:12:56 37 +11:12:58 37 +11:13:00 37 +11:13:02 37 +11:13:04 37 +11:13:07 37 +11:13:09 37 +11:13:11 37 +11:13:13 37 +11:13:15 37 +11:13:17 37 +11:13:19 37 +11:13:21 37 +11:13:23 37 +11:13:25 37 +11:13:27 37 +11:13:29 37 +11:13:31 37 +11:13:33 37 +11:13:35 37 +11:13:37 37 +11:13:40 37 +11:13:42 37 +11:13:44 37 +11:13:46 37 +11:13:48 37 +11:13:50 37 +11:13:52 37 +11:13:54 37 +11:13:56 37 +11:13:58 37 +11:14:00 37 +11:14:02 37 +11:14:04 37 +11:14:06 37 +11:14:08 37 +11:14:10 37 +11:14:13 37 +11:14:15 37 +11:14:17 37 +11:14:19 37 +11:14:21 37 +11:14:23 37 +11:14:25 37 +11:14:27 37 +11:14:29 37 +11:14:31 37 +11:14:33 37 +11:14:35 37 +11:14:37 37 +11:14:39 37 +11:14:41 37 +11:14:43 37 +11:14:46 37 +11:14:48 37 +11:14:50 37 +11:14:52 37 +11:14:54 37 +11:14:56 37 +11:14:58 37 +11:15:00 37 +11:15:02 37 +11:15:04 37 +11:15:06 37 +11:15:08 37 +11:15:10 37 +11:15:12 37 +11:15:14 37 +11:15:16 37 +11:15:19 37 +11:15:21 37 +11:15:23 37 +11:15:25 37 +11:15:27 37 +11:15:29 37 +11:15:31 37 +11:15:33 37 +11:15:35 37 +11:15:37 37 +11:15:39 37 +11:15:41 37 +11:15:43 37 +11:15:45 37 +11:15:47 37 +11:15:50 37 +11:15:52 37 +11:15:54 37 +11:15:56 37 +11:15:58 37 +11:16:00 37 +11:16:02 37 +11:16:04 37 +11:16:06 37 +11:16:08 37 +11:16:10 37 +11:16:12 37 +11:16:14 37 +11:16:16 37 +11:16:18 37 +11:16:20 37 +11:16:23 37 +11:16:25 37 +11:16:27 37 +11:16:29 37 +11:16:31 37 +11:16:33 37 +11:16:35 37 +11:16:37 37 +11:16:39 37 +11:16:41 37 +11:16:43 37 +11:16:45 37 +11:16:47 37 +11:16:49 37 +11:16:51 37 +11:16:53 37 +11:16:56 37 +11:16:58 37 +11:17:00 37 +11:17:02 37 +11:17:04 37 +11:17:06 37 +11:17:08 37 +11:17:10 37 +11:17:12 37 +11:17:14 37 +11:17:16 37 +11:17:18 37 +11:17:20 37 +11:17:22 37 +11:17:24 37 +11:17:26 37 +11:17:28 37 +11:17:31 37 +11:17:33 37 +11:17:35 37 +11:17:37 37 +11:17:39 37 +11:17:41 37 +11:17:43 37 +11:17:45 37 +11:17:47 37 +11:17:49 37 +11:17:51 37 +11:17:53 37 +11:17:55 37 +11:17:57 37 +11:17:59 37 +11:18:01 37 +11:18:04 37 +11:18:06 37 +11:18:08 37 +11:18:10 37 +11:18:12 37 +11:18:14 37 +11:18:16 37 +11:18:18 37 +11:18:20 37 +11:18:22 37 +11:18:24 37 +11:18:26 37 +11:18:28 37 +11:18:30 37 +11:18:32 37 +11:18:34 37 +11:18:37 37 +11:18:39 37 +11:18:41 37 +11:18:43 37 +11:18:45 37 +11:18:47 37 +11:18:49 37 +11:18:51 37 +11:18:53 37 +11:18:55 37 +11:18:57 37 +11:18:59 37 +11:19:01 37 +11:19:03 37 +11:19:05 37 +11:19:07 37 +11:19:10 37 +11:19:12 37 +11:19:14 37 +11:19:16 37 +11:19:18 37 +11:19:20 37 +11:19:22 37 +11:19:24 37 +11:19:26 37 +11:19:28 37 +11:19:30 37 +11:19:32 37 +11:19:34 37 +11:19:36 37 +11:19:38 37 +11:19:40 37 +11:19:43 37 +11:19:45 37 +11:19:47 37 +11:19:49 37 +11:19:51 37 +11:19:53 37 +11:19:55 37 +11:19:57 37 +11:19:59 37 +11:20:01 37 +11:20:03 37 +11:20:05 37 +11:20:07 37 +11:20:09 37 +11:20:11 37 +11:20:14 37 +11:20:16 37 +11:20:18 37 +11:20:20 37 +11:20:22 37 +11:20:24 37 +11:20:26 37 +11:20:28 37 +11:20:30 37 +11:20:32 37 +11:20:34 24 +11:20:36 24 +11:20:38 24 +11:20:40 24 +11:20:42 24 +11:20:44 24 +11:20:47 24 +11:20:49 24 +11:20:51 24 +11:20:53 24 +11:20:55 37 +11:20:57 61 +11:20:59 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/hostwatch-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/hostwatch-rank1.log new file mode 100644 index 0000000000..1a3318e27c --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/hostwatch-rank1.log @@ -0,0 +1,451 @@ +11:05:31 114 +11:05:33 113 +11:05:35 112 +11:05:37 112 +11:05:39 110 +11:05:41 110 +11:05:43 109 +11:05:45 108 +11:05:47 108 +11:05:49 108 +11:05:51 107 +11:05:53 107 +11:05:55 106 +11:05:57 106 +11:05:59 104 +11:06:01 104 +11:06:03 104 +11:06:05 103 +11:06:07 103 +11:06:10 102 +11:06:12 101 +11:06:14 102 +11:06:16 101 +11:06:18 100 +11:06:20 100 +11:06:22 98 +11:06:24 98 +11:06:26 97 +11:06:28 97 +11:06:30 97 +11:06:32 95 +11:06:34 95 +11:06:36 94 +11:06:38 94 +11:06:40 94 +11:06:42 92 +11:06:44 92 +11:06:47 92 +11:06:49 91 +11:06:51 91 +11:06:53 91 +11:06:55 90 +11:06:57 89 +11:06:59 88 +11:07:01 88 +11:07:03 87 +11:07:05 87 +11:07:07 86 +11:07:09 86 +11:07:11 86 +11:07:13 84 +11:07:15 84 +11:07:17 83 +11:07:19 82 +11:07:21 82 +11:07:24 81 +11:07:26 81 +11:07:28 80 +11:07:30 79 +11:07:32 79 +11:07:34 78 +11:07:36 78 +11:07:38 78 +11:07:40 76 +11:07:42 76 +11:07:44 77 +11:07:46 75 +11:07:48 75 +11:07:50 73 +11:07:52 73 +11:07:54 73 +11:07:56 72 +11:07:58 72 +11:08:00 72 +11:08:03 71 +11:08:05 71 +11:08:07 69 +11:08:09 69 +11:08:11 69 +11:08:13 68 +11:08:15 68 +11:08:17 66 +11:08:19 67 +11:08:21 66 +11:08:23 65 +11:08:25 65 +11:08:27 65 +11:08:29 64 +11:08:31 64 +11:08:33 62 +11:08:35 63 +11:08:37 62 +11:08:40 60 +11:08:42 60 +11:08:44 59 +11:08:46 59 +11:08:48 58 +11:08:50 57 +11:08:52 57 +11:08:54 57 +11:08:56 56 +11:08:58 56 +11:09:00 54 +11:09:02 55 +11:09:04 54 +11:09:06 53 +11:09:08 53 +11:09:10 53 +11:09:12 49 +11:09:14 50 +11:09:16 50 +11:09:19 50 +11:09:21 50 +11:09:23 49 +11:09:25 36 +11:09:27 36 +11:09:29 36 +11:09:31 36 +11:09:33 36 +11:09:35 36 +11:09:37 36 +11:09:39 36 +11:09:41 36 +11:09:43 37 +11:09:45 37 +11:09:47 37 +11:09:49 37 +11:09:52 37 +11:09:54 37 +11:09:56 37 +11:09:58 37 +11:10:00 37 +11:10:02 37 +11:10:04 37 +11:10:06 37 +11:10:08 37 +11:10:10 37 +11:10:12 37 +11:10:14 37 +11:10:16 37 +11:10:18 37 +11:10:20 37 +11:10:22 37 +11:10:25 37 +11:10:27 37 +11:10:29 37 +11:10:31 37 +11:10:33 37 +11:10:35 37 +11:10:37 37 +11:10:39 37 +11:10:41 37 +11:10:43 37 +11:10:45 37 +11:10:47 37 +11:10:49 37 +11:10:51 37 +11:10:53 37 +11:10:55 37 +11:10:58 37 +11:11:00 37 +11:11:02 37 +11:11:04 37 +11:11:06 37 +11:11:08 37 +11:11:10 37 +11:11:12 37 +11:11:14 37 +11:11:16 37 +11:11:18 37 +11:11:20 37 +11:11:22 37 +11:11:24 37 +11:11:26 37 +11:11:28 37 +11:11:30 37 +11:11:33 37 +11:11:35 37 +11:11:37 37 +11:11:39 37 +11:11:41 37 +11:11:43 37 +11:11:45 37 +11:11:47 37 +11:11:49 37 +11:11:51 37 +11:11:53 37 +11:11:55 37 +11:11:57 37 +11:11:59 37 +11:12:01 37 +11:12:03 37 +11:12:06 37 +11:12:08 37 +11:12:10 37 +11:12:12 37 +11:12:14 37 +11:12:16 37 +11:12:18 37 +11:12:20 37 +11:12:22 37 +11:12:24 37 +11:12:26 37 +11:12:28 37 +11:12:30 37 +11:12:32 37 +11:12:34 37 +11:12:36 37 +11:12:39 37 +11:12:41 37 +11:12:43 37 +11:12:45 37 +11:12:47 37 +11:12:49 37 +11:12:51 37 +11:12:53 37 +11:12:55 37 +11:12:57 37 +11:12:59 37 +11:13:01 37 +11:13:03 37 +11:13:05 37 +11:13:07 37 +11:13:09 37 +11:13:12 37 +11:13:14 37 +11:13:16 37 +11:13:18 37 +11:13:20 37 +11:13:22 37 +11:13:24 37 +11:13:26 37 +11:13:28 37 +11:13:30 37 +11:13:32 37 +11:13:34 37 +11:13:36 37 +11:13:38 37 +11:13:40 37 +11:13:42 37 +11:13:45 37 +11:13:47 37 +11:13:49 37 +11:13:51 37 +11:13:53 37 +11:13:55 37 +11:13:57 37 +11:13:59 37 +11:14:01 37 +11:14:03 37 +11:14:05 37 +11:14:07 37 +11:14:09 37 +11:14:11 37 +11:14:13 37 +11:14:15 37 +11:14:18 37 +11:14:20 37 +11:14:22 37 +11:14:24 37 +11:14:26 37 +11:14:28 37 +11:14:30 37 +11:14:32 37 +11:14:34 37 +11:14:36 37 +11:14:38 37 +11:14:40 37 +11:14:42 37 +11:14:44 37 +11:14:46 37 +11:14:48 37 +11:14:51 37 +11:14:53 37 +11:14:55 37 +11:14:57 37 +11:14:59 37 +11:15:01 37 +11:15:03 37 +11:15:05 37 +11:15:07 37 +11:15:09 37 +11:15:11 37 +11:15:13 37 +11:15:15 37 +11:15:17 37 +11:15:19 37 +11:15:21 37 +11:15:24 37 +11:15:26 37 +11:15:28 37 +11:15:30 37 +11:15:32 37 +11:15:34 37 +11:15:36 37 +11:15:38 37 +11:15:40 37 +11:15:42 37 +11:15:44 37 +11:15:46 37 +11:15:48 37 +11:15:50 37 +11:15:52 37 +11:15:54 37 +11:15:57 37 +11:15:59 37 +11:16:01 37 +11:16:03 37 +11:16:05 37 +11:16:07 37 +11:16:09 37 +11:16:11 37 +11:16:13 37 +11:16:15 37 +11:16:17 37 +11:16:19 37 +11:16:21 37 +11:16:23 37 +11:16:25 37 +11:16:28 37 +11:16:30 37 +11:16:32 37 +11:16:34 37 +11:16:36 37 +11:16:38 37 +11:16:40 37 +11:16:42 37 +11:16:44 37 +11:16:46 37 +11:16:48 37 +11:16:50 37 +11:16:52 37 +11:16:54 37 +11:16:56 37 +11:16:59 37 +11:17:01 37 +11:17:03 37 +11:17:05 37 +11:17:07 37 +11:17:09 37 +11:17:11 37 +11:17:13 37 +11:17:15 37 +11:17:17 37 +11:17:19 37 +11:17:21 37 +11:17:23 37 +11:17:25 37 +11:17:27 37 +11:17:29 37 +11:17:32 37 +11:17:34 37 +11:17:36 37 +11:17:38 37 +11:17:40 37 +11:17:42 37 +11:17:44 37 +11:17:46 37 +11:17:48 37 +11:17:50 37 +11:17:52 37 +11:17:54 37 +11:17:56 37 +11:17:58 37 +11:18:00 37 +11:18:03 37 +11:18:05 37 +11:18:07 37 +11:18:09 37 +11:18:11 37 +11:18:13 37 +11:18:15 37 +11:18:17 37 +11:18:19 37 +11:18:21 37 +11:18:23 37 +11:18:25 37 +11:18:27 37 +11:18:29 37 +11:18:31 37 +11:18:33 37 +11:18:36 37 +11:18:38 37 +11:18:40 37 +11:18:42 37 +11:18:44 37 +11:18:46 37 +11:18:48 37 +11:18:50 37 +11:18:52 37 +11:18:54 37 +11:18:56 37 +11:18:58 37 +11:19:00 37 +11:19:02 37 +11:19:04 37 +11:19:06 37 +11:19:08 37 +11:19:11 37 +11:19:13 37 +11:19:15 37 +11:19:17 37 +11:19:19 37 +11:19:21 37 +11:19:23 37 +11:19:25 37 +11:19:27 37 +11:19:29 37 +11:19:31 37 +11:19:33 37 +11:19:35 37 +11:19:37 37 +11:19:39 37 +11:19:42 37 +11:19:44 37 +11:19:46 37 +11:19:48 37 +11:19:50 37 +11:19:52 37 +11:19:54 37 +11:19:56 37 +11:19:58 37 +11:20:00 37 +11:20:02 37 +11:20:04 37 +11:20:06 37 +11:20:08 37 +11:20:10 37 +11:20:13 37 +11:20:15 37 +11:20:17 37 +11:20:19 37 +11:20:21 37 +11:20:23 37 +11:20:25 37 +11:20:27 37 +11:20:29 37 +11:20:31 37 +11:20:33 37 +11:20:35 24 +11:20:37 24 +11:20:39 24 +11:20:41 24 +11:20:43 24 +11:20:46 24 +11:20:48 24 +11:20:50 24 +11:20:52 24 +11:20:54 37 +11:20:56 37 +11:20:58 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/hostwatch-rank2.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/hostwatch-rank2.log new file mode 100644 index 0000000000..dc0795ff75 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/hostwatch-rank2.log @@ -0,0 +1,450 @@ +11:05:31 112 +11:05:33 112 +11:05:36 111 +11:05:38 110 +11:05:40 110 +11:05:42 109 +11:05:44 109 +11:05:46 109 +11:05:48 108 +11:05:50 108 +11:05:52 106 +11:05:54 106 +11:05:56 105 +11:05:58 105 +11:06:00 104 +11:06:02 104 +11:06:04 104 +11:06:06 102 +11:06:08 102 +11:06:10 101 +11:06:13 101 +11:06:15 101 +11:06:17 100 +11:06:19 100 +11:06:21 100 +11:06:23 99 +11:06:25 99 +11:06:27 97 +11:06:29 97 +11:06:31 97 +11:06:33 96 +11:06:35 96 +11:06:37 94 +11:06:39 94 +11:06:41 94 +11:06:43 93 +11:06:45 92 +11:06:47 92 +11:06:50 92 +11:06:52 90 +11:06:54 91 +11:06:56 90 +11:06:58 89 +11:07:00 88 +11:07:02 88 +11:07:04 88 +11:07:06 87 +11:07:08 87 +11:07:10 85 +11:07:12 85 +11:07:14 84 +11:07:16 84 +11:07:18 84 +11:07:20 83 +11:07:22 83 +11:07:24 82 +11:07:26 82 +11:07:29 82 +11:07:31 80 +11:07:33 80 +11:07:35 79 +11:07:37 79 +11:07:39 79 +11:07:41 78 +11:07:43 78 +11:07:45 76 +11:07:47 76 +11:07:49 75 +11:07:51 75 +11:07:53 75 +11:07:55 74 +11:07:57 74 +11:07:59 72 +11:08:01 72 +11:08:03 72 +11:08:06 71 +11:08:08 71 +11:08:10 70 +11:08:12 70 +11:08:14 70 +11:08:16 69 +11:08:18 68 +11:08:20 68 +11:08:22 67 +11:08:24 67 +11:08:26 67 +11:08:28 66 +11:08:30 66 +11:08:32 64 +11:08:34 64 +11:08:36 64 +11:08:38 63 +11:08:40 63 +11:08:43 62 +11:08:45 62 +11:08:47 60 +11:08:49 60 +11:08:51 59 +11:08:53 58 +11:08:55 58 +11:08:57 58 +11:08:59 56 +11:09:01 55 +11:09:03 55 +11:09:05 55 +11:09:07 55 +11:09:09 55 +11:09:11 41 +11:09:13 42 +11:09:15 42 +11:09:17 42 +11:09:20 42 +11:09:22 42 +11:09:24 42 +11:09:26 42 +11:09:28 42 +11:09:30 42 +11:09:32 42 +11:09:34 42 +11:09:36 42 +11:09:38 42 +11:09:40 42 +11:09:42 42 +11:09:44 42 +11:09:46 42 +11:09:48 42 +11:09:50 42 +11:09:53 42 +11:09:55 42 +11:09:57 42 +11:09:59 42 +11:10:01 42 +11:10:03 42 +11:10:05 42 +11:10:07 42 +11:10:09 42 +11:10:11 42 +11:10:14 42 +11:10:16 42 +11:10:18 42 +11:10:20 42 +11:10:22 42 +11:10:24 42 +11:10:26 42 +11:10:28 42 +11:10:30 42 +11:10:32 42 +11:10:34 42 +11:10:36 42 +11:10:38 42 +11:10:40 42 +11:10:42 42 +11:10:44 42 +11:10:47 42 +11:10:49 42 +11:10:51 42 +11:10:53 42 +11:10:55 42 +11:10:57 42 +11:10:59 42 +11:11:01 42 +11:11:03 42 +11:11:05 42 +11:11:07 42 +11:11:09 42 +11:11:11 42 +11:11:13 42 +11:11:15 42 +11:11:18 42 +11:11:20 42 +11:11:22 42 +11:11:24 42 +11:11:26 42 +11:11:28 42 +11:11:30 42 +11:11:32 42 +11:11:34 42 +11:11:36 42 +11:11:38 42 +11:11:40 42 +11:11:42 42 +11:11:44 42 +11:11:46 42 +11:11:48 42 +11:11:51 42 +11:11:53 42 +11:11:55 42 +11:11:57 42 +11:11:59 42 +11:12:01 42 +11:12:03 42 +11:12:05 42 +11:12:07 42 +11:12:09 42 +11:12:11 42 +11:12:13 42 +11:12:15 42 +11:12:17 42 +11:12:19 42 +11:12:21 42 +11:12:24 42 +11:12:26 42 +11:12:28 42 +11:12:30 42 +11:12:32 42 +11:12:34 42 +11:12:36 42 +11:12:38 42 +11:12:40 42 +11:12:42 42 +11:12:44 42 +11:12:46 42 +11:12:48 42 +11:12:50 42 +11:12:52 42 +11:12:54 42 +11:12:57 42 +11:12:59 42 +11:13:01 42 +11:13:03 42 +11:13:05 42 +11:13:07 42 +11:13:09 42 +11:13:11 42 +11:13:13 42 +11:13:15 42 +11:13:17 42 +11:13:19 42 +11:13:21 42 +11:13:23 42 +11:13:25 42 +11:13:27 42 +11:13:29 42 +11:13:32 42 +11:13:34 42 +11:13:36 42 +11:13:38 42 +11:13:40 42 +11:13:42 42 +11:13:44 42 +11:13:46 42 +11:13:48 42 +11:13:50 42 +11:13:52 42 +11:13:54 42 +11:13:56 42 +11:13:58 42 +11:14:00 42 +11:14:02 42 +11:14:05 42 +11:14:07 42 +11:14:09 42 +11:14:11 42 +11:14:13 42 +11:14:15 42 +11:14:17 42 +11:14:19 42 +11:14:21 42 +11:14:23 42 +11:14:25 42 +11:14:27 42 +11:14:29 42 +11:14:31 42 +11:14:33 42 +11:14:35 42 +11:14:38 42 +11:14:40 42 +11:14:42 42 +11:14:44 42 +11:14:46 42 +11:14:48 42 +11:14:50 42 +11:14:52 42 +11:14:54 42 +11:14:56 42 +11:14:58 42 +11:15:00 42 +11:15:02 42 +11:15:04 42 +11:15:06 42 +11:15:08 42 +11:15:10 42 +11:15:13 42 +11:15:15 42 +11:15:17 42 +11:15:19 42 +11:15:21 42 +11:15:23 42 +11:15:25 42 +11:15:27 42 +11:15:29 42 +11:15:31 42 +11:15:33 42 +11:15:35 42 +11:15:37 42 +11:15:39 42 +11:15:41 42 +11:15:44 42 +11:15:46 42 +11:15:48 42 +11:15:50 42 +11:15:52 42 +11:15:54 42 +11:15:56 42 +11:15:58 42 +11:16:00 42 +11:16:02 42 +11:16:04 42 +11:16:06 42 +11:16:08 42 +11:16:10 42 +11:16:12 42 +11:16:14 42 +11:16:17 42 +11:16:19 42 +11:16:21 42 +11:16:23 42 +11:16:25 42 +11:16:27 42 +11:16:29 42 +11:16:31 42 +11:16:33 42 +11:16:35 42 +11:16:37 42 +11:16:39 42 +11:16:41 42 +11:16:43 42 +11:16:45 42 +11:16:47 42 +11:16:50 42 +11:16:52 42 +11:16:54 42 +11:16:56 42 +11:16:58 42 +11:17:00 42 +11:17:02 42 +11:17:04 42 +11:17:06 42 +11:17:08 42 +11:17:10 42 +11:17:12 42 +11:17:14 42 +11:17:16 42 +11:17:18 42 +11:17:21 42 +11:17:23 42 +11:17:25 42 +11:17:27 42 +11:17:29 42 +11:17:31 42 +11:17:33 42 +11:17:35 42 +11:17:37 42 +11:17:39 42 +11:17:41 42 +11:17:43 42 +11:17:45 42 +11:17:47 42 +11:17:49 42 +11:17:51 42 +11:17:53 42 +11:17:56 42 +11:17:58 42 +11:18:00 42 +11:18:02 42 +11:18:04 42 +11:18:06 42 +11:18:08 42 +11:18:10 42 +11:18:12 42 +11:18:14 42 +11:18:16 42 +11:18:18 42 +11:18:20 42 +11:18:22 42 +11:18:24 42 +11:18:27 42 +11:18:29 42 +11:18:31 42 +11:18:33 42 +11:18:35 42 +11:18:37 42 +11:18:39 42 +11:18:41 42 +11:18:43 42 +11:18:45 42 +11:18:47 42 +11:18:49 42 +11:18:51 42 +11:18:53 42 +11:18:55 42 +11:18:57 42 +11:18:59 42 +11:19:02 42 +11:19:04 42 +11:19:06 42 +11:19:08 42 +11:19:10 42 +11:19:12 42 +11:19:14 42 +11:19:16 42 +11:19:18 42 +11:19:20 42 +11:19:22 42 +11:19:24 42 +11:19:26 42 +11:19:28 42 +11:19:30 42 +11:19:33 42 +11:19:35 42 +11:19:37 42 +11:19:39 42 +11:19:41 42 +11:19:43 42 +11:19:45 42 +11:19:47 42 +11:19:49 42 +11:19:51 42 +11:19:53 42 +11:19:55 42 +11:19:57 42 +11:19:59 42 +11:20:01 42 +11:20:04 42 +11:20:06 42 +11:20:08 42 +11:20:10 42 +11:20:12 42 +11:20:14 42 +11:20:16 42 +11:20:18 42 +11:20:20 42 +11:20:22 42 +11:20:24 42 +11:20:26 42 +11:20:28 42 +11:20:30 42 +11:20:32 42 +11:20:34 29 +11:20:37 29 +11:20:39 29 +11:20:41 29 +11:20:43 29 +11:20:45 29 +11:20:47 29 +11:20:49 29 +11:20:51 29 +11:20:53 42 +11:20:55 42 +11:20:57 done diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/server-rank0.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/server-rank0.log new file mode 100644 index 0000000000..bab84685a7 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/server-rank0.log @@ -0,0 +1,16644 @@ +[kvp-serve] 2026-10-23T02:05:21+00:00 rank 0 of 3 context 1048576 parallel auto binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=off TF_GLM_MM=unset args --prompt-cache-gib 0 +info: rank 0: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-a:7185:7185 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-a:7185:7185 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-a:7185:7185 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-a:7185:7185 [0] NCCL INFO cudaDriverVersion 13030 +host-a:7185:7185 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-a:7185:7185 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-a:7185:7185 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-a:7185:7185 [0] NCCL INFO Count not open libibverbs.so +host-a:7185:7185 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-a:7185:7185 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-a:7185:7185 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-a:7185:7185 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0xae035c0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:7185:7185 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0xae44cc0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-a:7185:7185 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0xae863c0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:7185:7185 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0xaec7ac0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-a:7185:7185 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:7185:7185 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:7185:7185 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:7185:7185 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-a:7185:7185 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-a:7185:7185 [0] NCCL INFO Initialized NET plugin IB +host-a:7185:7185 [0] NCCL INFO Assigned NET plugin IB to comm +host-a:7185:7185 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-a:7185:7185 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-a:7185:7185 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-a:7185:7185 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-a:7185:7185 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-a:7185:7185 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-a:7185:7185 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-a:7185:7185 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-a:7185:7185 [0] NCCL INFO Using network IB +host-a:7185:7185 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-a:7185:7185 [0] NCCL INFO [Rank 0] ncclCommInitRank comm 0x81cbf10 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9eea48d2e83cbf58 - Init START +host-a:7185:7185 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-a:7185:7185 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:7185:7185 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:7185:7185 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:7185:7185 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:7185:7185 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:7185:7185 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:7185:7185 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:7185:7185 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:7185:7185 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-a:7185:7185 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-a:7185:7185 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-a:7185:7185 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-a:7185:7185 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:7185:7185 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-a:7185:7185 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-a:7185:7185 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-a:7185:7185 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-a:7185:7185 [0] NCCL INFO Rank 0: 6 Net devices +host-a:7185:7185 [0] NCCL INFO Rank 0: 0 CollNet devices +host-a:7185:7185 [0] NCCL INFO Local Net device counts across ranks: min 6 max 6 +host-a:7185:7185 [0] NCCL INFO Local CollNet device counts across ranks: min 0 max 0 +host-a:7185:7185 [0] NCCL INFO comm 0x81cbf10 rank 0 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-a:7185:7185 [0] NCCL INFO Channel 00/04 : 0 1 2 +host-a:7185:7185 [0] NCCL INFO Channel 01/04 : 0 1 2 +host-a:7185:7185 [0] NCCL INFO Channel 02/04 : 0 1 2 +host-a:7185:7185 [0] NCCL INFO Channel 03/04 : 0 1 2 +host-a:7185:7185 [0] NCCL INFO Trees [0] 2/-1/-1->0->-1 [1] 2/-1/-1->0->-1 [2] 2/-1/-1->0->-1 [3] 2/-1/-1->0->-1 +host-a:7185:7185 [0] NCCL INFO P2P Chunksize set to 262144 +host-a:7185:7185 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-a:7185:7185 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-a:7185:7185 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-a:7185:7185 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-a:7185:7202 [0] NCCL INFO [Proxy Service] Device 0 CPU core 12 +host-a:7185:7202 [0] NCCL INFO proxy listening socket at <34391> +host-a:7185:7203 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 0 +host-a:7185:7204 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 12 +host-a:7185:7185 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:7185:7185 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:7185:7185 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [receive] via NET/IB/4 +host-a:7185:7185 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [receive] via NET/IB/5 +host-a:7185:7185 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:7185:7185 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:7185:7185 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [send] via NET/IB/4 +host-a:7185:7185 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [send] via NET/IB/5 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b44010000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b44054000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b4409c000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b440e0000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b44128000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b4416c000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b441b4000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6936 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6937 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6936 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b514e ctsFifoLkey=0x1b514e +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6936 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6937 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1b514e ctsFifoLkey=0x1b514e +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6937 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b4425d000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1914 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1915 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1916 pkey=0 pd=0xfc5b4409bdb0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2042 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2043 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2044 pkey=0 pd=0xfc5b4420ae60 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6936 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6937 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1917 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1918 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1919 pkey=0 pd=0xfc5b4409bdb0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6968 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6969 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6968 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c8585 ctsFifoLkey=0x1c8585 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6968 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6969 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c8585 ctsFifoLkey=0x1c8585 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6969 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2045 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2046 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2047 pkey=0 pd=0xfc5b4420ae60 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6938 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6939 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6938 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1bd5d2 ctsFifoLkey=0x1bd5d2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6938 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6939 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x1bd5d2 ctsFifoLkey=0x1bd5d2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6939 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6970 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6971 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6970 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6970 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6971 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1d2521 ctsFifoLkey=0x1d2521 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6971 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6968 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6969 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6938 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6939 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6970 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6971 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7185 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-a:7185:7185 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:7185:7185 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:7185:7185 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [send] via NET/IB/4 +host-a:7185:7185 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [send] via NET/IB/5 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b445b9000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b445fa000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b4463b000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1920 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1921 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1920 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1920 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1921 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x17904d ctsFifoLkey=0x17904d +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1921 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b446d1000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1920 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1921 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2048 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2049 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2048 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b4ad ctsFifoLkey=0x19b4ad +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2048 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2049 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b4ad ctsFifoLkey=0x19b4ad +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2049 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2048 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2049 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1922 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1923 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1922 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1922 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1923 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x178543 ctsFifoLkey=0x178543 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1923 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2050 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2051 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2050 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2050 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2051 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19b9b7 ctsFifoLkey=0x19b9b7 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2051 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1922 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1923 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2050 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2051 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7185 [0] NCCL INFO Connected all trees +host-a:7185:7185 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:7185:7185 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:7185:7185 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [receive] via NET/IB/4 +host-a:7185:7185 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [receive] via NET/IB/5 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b44822000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b44867000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b448ac000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b448f4000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6940 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6941 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6942 pkey=0 pd=0xfc5b44053e10 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6972 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6973 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6974 pkey=0 pd=0xfc5b44317c30 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6943 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6944 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6945 pkey=0 pd=0xfc5b44053e10 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6975 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6976 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6977 pkey=0 pd=0xfc5b44317c30 +host-a:7185:7185 [0] NCCL INFO Connected binomial trees +host-a:7185:7185 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-a:7185:7185 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-a:7185:7185 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-a:7185:7185 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-a:7185:7185 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-a:7185:7185 [0] NCCL INFO CC Off, workFifoBytes 1048576 +host-a:7185:7185 [0] NCCL INFO ncclCommInitRank comm 0x81cbf10 rank 0 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9eea48d2e83cbf58 - Init COMPLETE +host-a:7185:7185 [0] NCCL INFO Init timings - ncclCommInitRank: rank 0 nranks 3 total 2.53 (kernels 0.24, alloc 0.02, bootstrap 1.96, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.30, rest 0.00) +host-a:7185:7185 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:7185:7185 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:7185:7185 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [send] via NET/IB/4/Shared +host-a:7185:7185 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [send] via NET/IB/5/Shared +host-a:7185:7185 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [receive] via NET/IB/5/Shared +host-a:7185:7185 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [receive] via NET/IB/4/Shared +host-a:7185:7185 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [send] via NET/IB/5/Shared +host-a:7185:7185 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [send] via NET/IB/4/Shared +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b44b18000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b44b59000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b44b9f000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b44be3000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b44c2b000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b44c6f000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xfc5b44cb7000) +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6946 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6947 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6946 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x16f8f6 ctsFifoLkey=0x16f8f6 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6946 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 6947 mtu 3 GID 3 (0/10C290AFFFF0000) ctsFifoRkey=0x16f8f6 ctsFifoLkey=0x16f8f6 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 6947 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6978 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6979 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6978 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c2525 ctsFifoLkey=0x1c2525 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6978 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 6979 mtu 3 GID 3 (0/10C2A0AFFFF0000) ctsFifoRkey=0x1c2525 ctsFifoLkey=0x1c2525 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 6979 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2052 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2053 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2052 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c0bd ctsFifoLkey=0x19c0bd +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2052 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2053 mtu 3 GID 3 (0/10D2A0AFFFF0000) ctsFifoRkey=0x19c0bd ctsFifoLkey=0x19c0bd +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2053 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1924 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1925 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1924 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1924 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1925 mtu 3 GID 3 (0/10D290AFFFF0000) ctsFifoRkey=0x176321 ctsFifoLkey=0x176321 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1925 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xfc5b44e67000) +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6946 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 6947 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1926 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1927 pkey=0 pd=0xfc5b4409bdb0 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1928 pkey=0 pd=0xfc5b4409bdb0 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6978 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 6979 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2054 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2055 pkey=0 pd=0xfc5b4420ae60 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP createinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +[tensorfold] CUDA sm_121 device 0 (NVIDIA GB10, memory shared with the host): model 78.03 GiB; 2 streams at once, 12.88 GiB each at a 1048576-token window, of 24.4 GiB left after a 12.2 GiB reserve; prompts in passes of up to 2048 rows, no prompt cache; own glue kernels +[tensorfold] serving glm-tf at http://127.0.0.1:8097/v1 (sampling: temperature 1.0, top_p 0.95; drafts: on; context: 1048576; loaded in 246.6s) +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/models HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ed538ba28f8def3fab3197442926bd8d prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=29 accepted=32/43 ttft=5.106 decode=1.625 elapsed=6.731 queued=0.000 tok/s=36.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-19dc3c48105ab0900b2605dc616ed9c9 prompt=7759 cached=0 thinking=True effort=high tokens=61 sha=f0f55b35a304 finish=stop rounds=60 accepted=0/0 ttft=5.063 decode=2.740 elapsed=7.803 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-682de7c01b32c05c0add420ae0f671c9 prompt=15550 cached=0 thinking=True effort=high tokens=46 sha=539973cb8b13 finish=stop rounds=13 accepted=33/36 ttft=10.209 decode=0.862 elapsed=11.071 queued=0.000 tok/s=52.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1ec75c6cd8adf08cb19afdd5b763eb95 prompt=15550 cached=0 thinking=True effort=high tokens=46 sha=539973cb8b13 finish=stop rounds=45 accepted=0/0 ttft=10.275 decode=2.072 elapsed=12.347 queued=0.000 tok/s=21.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-28bf9e978f24fd1543a61dda40dbd607 prompt=19462 cached=0 thinking=True effort=high tokens=96 sha=8fc3c26162f1 finish=stop rounds=38 accepted=58/82 ttft=12.892 decode=2.329 elapsed=15.221 queued=0.000 tok/s=40.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-adc6aa8cdf583ee5971aeaac54c7405f prompt=19462 cached=0 thinking=True effort=high tokens=96 sha=8fc3c26162f1 finish=stop rounds=95 accepted=0/0 ttft=12.757 decode=4.440 elapsed=17.197 queued=0.000 tok/s=21.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2f2c4a26f7185e9dba39c688255056bc prompt=21892 cached=0 thinking=True effort=high tokens=113 sha=dedd451779f3 finish=stop rounds=61 accepted=52/80 ttft=14.368 decode=3.416 elapsed=17.784 queued=0.000 tok/s=32.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-95bee0785fbc2bbd781ab02550622f85 prompt=21892 cached=0 thinking=True effort=high tokens=113 sha=dedd451779f3 finish=stop rounds=112 accepted=0/0 ttft=14.327 decode=5.116 elapsed=19.443 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-b8520f0f489c7721a1c9325947f66574 prompt=24388 cached=0 thinking=True effort=high tokens=160 sha=8bcbd355dc04 finish=length rounds=81 accepted=79/121 ttft=15.830 decode=4.585 elapsed=20.415 queued=0.000 tok/s=34.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-86b82b5b05b409d7123c56d5c3141203 prompt=24388 cached=0 thinking=True effort=high tokens=160 sha=8bcbd355dc04 finish=length rounds=159 accepted=0/0 ttft=15.784 decode=7.261 elapsed=23.045 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-814a6e2422cff4516f17d6fe468b3200 prompt=26895 cached=0 thinking=True effort=high tokens=154 sha=c8e79afc5a10 finish=stop rounds=81 accepted=72/122 ttft=17.708 decode=4.576 elapsed=22.284 queued=0.000 tok/s=33.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-601f911484f6c79ade4a5bd2ad93f2a5 prompt=26895 cached=0 thinking=True effort=high tokens=154 sha=c8e79afc5a10 finish=stop rounds=153 accepted=0/0 ttft=17.725 decode=6.958 elapsed=24.683 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0447d771e9ba70ad3014170603482cca prompt=29798 cached=0 thinking=True effort=high tokens=136 sha=7629aa6bf5df finish=stop rounds=66 accepted=69/100 ttft=19.490 decode=3.729 elapsed=23.219 queued=0.000 tok/s=36.2 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-86bc6fac4451c7a49aeae8d6cf348395 prompt=29798 cached=0 thinking=True effort=high tokens=136 sha=7629aa6bf5df finish=stop rounds=135 accepted=0/0 ttft=19.446 decode=6.163 elapsed=25.609 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bc65340feca6b83705d1c14686675b25 prompt=35434 cached=0 thinking=True effort=high tokens=160 sha=1adac255ff32 finish=length rounds=60 accepted=99/148 ttft=23.247 decode=3.816 elapsed=27.063 queued=0.000 tok/s=41.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-417cf16ec70b648d5abaede052c8e9ca prompt=35434 cached=0 thinking=True effort=high tokens=160 sha=1adac255ff32 finish=length rounds=159 accepted=0/0 ttft=23.230 decode=7.256 elapsed=30.486 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-23892bea01252d4a0630d5ef3a30bb29 prompt=38152 cached=0 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=34 accepted=125/133 ttft=24.942 decode=2.380 elapsed=27.322 queued=0.000 tok/s=66.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-c2d141fa7956eafc867945061bef9dcf prompt=38152 cached=0 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=159 accepted=0/0 ttft=24.904 decode=7.275 elapsed=32.179 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-cc258b06982b7ecbe862ad53b3ef6a27 prompt=2115 cached=0 thinking=True effort=high tokens=160 sha=7b1461bf71da finish=length rounds=75 accepted=84/126 ttft=1.481 decode=4.301 elapsed=5.782 queued=0.000 tok/s=37.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7c43c1251d7067b47eb8b55764efafc1 prompt=7412 cached=0 thinking=True effort=high tokens=90 sha=d0573eef0609 finish=stop rounds=35 accepted=55/81 ttft=4.959 decode=2.180 elapsed=7.138 queued=0.000 tok/s=40.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b9bcb24a896510f5ea72e040d5693e9a prompt=7412 cached=0 thinking=True effort=high tokens=90 sha=d0573eef0609 finish=stop rounds=89 accepted=0/0 ttft=4.920 decode=4.062 elapsed=8.982 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3d3835037ab455ccb6b350118ae23a81 prompt=7474 cached=0 thinking=True effort=high tokens=73 sha=9a2685e9905f finish=stop rounds=24 accepted=49/65 ttft=4.981 decode=1.576 elapsed=6.556 queued=0.000 tok/s=45.7 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d4bd54cf398bc748970c850e687e9294 prompt=7474 cached=0 thinking=True effort=high tokens=73 sha=9a2685e9905f finish=stop rounds=72 accepted=0/0 ttft=4.957 decode=3.286 elapsed=8.243 queued=0.000 tok/s=21.9 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0c3ed028ea2cc1b88cc317235e36088f prompt=12632 cached=0 thinking=True effort=high tokens=160 sha=e1f2511e60f3 finish=length rounds=75 accepted=84/122 ttft=8.473 decode=4.285 elapsed=12.758 queued=0.000 tok/s=37.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0f94aebaf6fff1397f1e6929672d9dae prompt=12632 cached=0 thinking=True effort=high tokens=160 sha=e1f2511e60f3 finish=length rounds=159 accepted=0/0 ttft=8.452 decode=7.203 elapsed=15.655 queued=0.000 tok/s=22.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-95e0edbed91571a77607378864e2758c prompt=12721 cached=0 thinking=True effort=high tokens=117 sha=f202ab6075ae finish=stop rounds=44 accepted=72/103 ttft=8.485 decode=2.735 elapsed=11.220 queued=0.000 tok/s=42.4 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a75471d77f5387b3ac5fd62f36ed0946 prompt=12721 cached=0 thinking=True effort=high tokens=117 sha=f202ab6075ae finish=stop rounds=116 accepted=0/0 ttft=8.476 decode=5.266 elapsed=13.742 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-63f4beb564a4f71d0aa001569d8be536 prompt=15383 cached=0 thinking=True effort=high tokens=137 sha=7695627844cd finish=stop rounds=70 accepted=67/97 ttft=10.132 decode=3.875 elapsed=14.006 queued=0.000 tok/s=35.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-3e92414549deda97ea128d49066e86cb prompt=15383 cached=0 thinking=True effort=high tokens=137 sha=7695627844cd finish=stop rounds=136 accepted=0/0 ttft=10.085 decode=6.181 elapsed=16.266 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-9a231f896748a7bdf5024dab89139ace prompt=17330 cached=0 thinking=True effort=high tokens=160 sha=8b15f58457e8 finish=length rounds=69 accepted=90/124 ttft=11.411 decode=4.047 elapsed=15.457 queued=0.000 tok/s=39.3 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-3bdf6d21f7ab588351430dbe420057bc prompt=17330 cached=0 thinking=True effort=high tokens=160 sha=8b15f58457e8 finish=length rounds=159 accepted=0/0 ttft=11.378 decode=7.221 elapsed=18.598 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-bcf4a237af2127a6a37cd0055f61ee5a prompt=40901 cached=0 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=22 accepted=137/137 ttft=26.654 decode=1.837 elapsed=28.491 queued=0.000 tok/s=86.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-cb112b12ba93dff80feade0c4ba3a72e prompt=40901 cached=0 thinking=True effort=high tokens=160 sha=6fb6a6ab70d0 finish=length rounds=159 accepted=0/0 ttft=26.601 decode=7.307 elapsed=33.908 queued=0.000 tok/s=21.8 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-4c2e7ab6d4cb09387f5b1604146b7167 prompt=1520 cached=0 thinking=False effort=none tokens=150 sha=033a197da586 finish=stop rounds=64 accepted=85/131 ttft=0.923 decode=3.774 elapsed=4.697 queued=0.000 tok/s=39.5 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9d99b2a10b54f21f1cd9c7390329fb10 prompt=1520 cached=0 thinking=False effort=none tokens=150 sha=033a197da586 finish=stop rounds=149 accepted=0/0 ttft=0.887 decode=6.602 elapsed=7.489 queued=0.000 tok/s=22.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-97f8f775f92d83bef10e59f70460afe1 prompt=4202 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=4 accepted=8/9 ttft=3.043 decode=0.245 elapsed=3.287 queued=0.000 tok/s=45.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-25c2df27294dbdf8716779201d22e3ab prompt=4202 cached=0 thinking=False effort=none tokens=12 sha=a94919711db8 finish=stop rounds=11 accepted=0/0 ttft=3.031 decode=0.510 elapsed=3.540 queued=0.000 tok/s=21.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9aac441933db34c8cf7706df42ef9286 prompt=9702 cached=0 thinking=True effort=high tokens=110 sha=88f98460968a finish=stop rounds=47 accepted=63/86 ttft=6.396 decode=2.751 elapsed=9.147 queued=0.000 tok/s=39.6 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-70914d984c4e11fd53c3b0efec5313b5 prompt=9702 cached=0 thinking=True effort=high tokens=110 sha=88f98460968a finish=stop rounds=109 accepted=0/0 ttft=6.342 decode=4.962 elapsed=11.304 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-5c75214f935520374f2de9cb84332631 prompt=15180 cached=0 thinking=True effort=high tokens=160 sha=5694b66a8995 finish=length rounds=76 accepted=83/115 ttft=10.021 decode=4.293 elapsed=14.314 queued=0.000 tok/s=37.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-492a16f0f41f6f5eb9dedef53ee011fb prompt=15180 cached=0 thinking=True effort=high tokens=160 sha=5694b66a8995 finish=length rounds=159 accepted=0/0 ttft=9.999 decode=7.212 elapsed=17.211 queued=0.000 tok/s=22.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c172e462d9ffdf4264ed7b9def94c84b prompt=19808 cached=0 thinking=True effort=high tokens=141 sha=025c4856c697 finish=stop rounds=48 accepted=93/143 ttft=26.466 decode=4.813 elapsed=31.280 queued=0.725 tok/s=29.1 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] warning: a reply reached max_tokens while still thinking, so its content is empty and its text is all in reasoning_content; raise max_tokens, or send chat_template_kwargs {"enable_thinking": false} (server: --no-thinking) +[tensorfold] done chatcmpl-4b9c8c00adcda669ed3a26632d7db2b9 prompt=18246 cached=0 thinking=True effort=high tokens=160 sha=9599ff0fa9b1 finish=length rounds=64 accepted=95/171 ttft=11.855 decode=19.828 elapsed=31.683 queued=0.000 tok/s=8.0 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] shutting down (SIGTERM): draining: new requests get 503; 0 request(s) in progress may finish within 20.0 s +[tensorfold] shutting down: drained +[tensorfold] shutting down: stopped +d: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2056 pkey=0 pd=0xfc5b4420ae60 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2052 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2053 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6980 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6981 pkey=0 pd=0xfc5b44317c30 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=6982 pkey=0 pd=0xfc5b44317c30 +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1924 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1925 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-a:7185:7202 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6948 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6949 pkey=0 pd=0xfc5b44053e10 oooRq=0 +host-a:7185:7202 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=6950 pkey=0 pd=0xfc5b44053e10 +host-a:7185:7185 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/server-rank1.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/server-rank1.log new file mode 100644 index 0000000000..39a30e3187 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/server-rank1.log @@ -0,0 +1,5056 @@ +[kvp-serve] 2026-10-23T02:05:21+00:00 rank 1 of 3 context 1048576 parallel auto binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=off TF_GLM_MM=unset args --prompt-cache-gib 0 +info: rank 1: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-b:6279:6279 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-b:6279:6279 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-b:6279:6279 [0] NCCL INFO cudaDriverVersion 13030 +host-b:6279:6279 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-b:6279:6279 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-b:6279:6279 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-b:6279:6279 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-b:6279:6279 [0] NCCL INFO Count not open libibverbs.so +host-b:6279:6279 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-b:6279:6279 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-b:6279:6279 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-b:6279:6279 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x394882a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:6279:6279 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x394c99a0 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-b:6279:6279 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x3950b0a0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:6279:6279 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x3954c7a0 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-b:6279:6279 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:6279:6279 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:6279:6279 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:6279:6279 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-b:6279:6279 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-b:6279:6279 [0] NCCL INFO Initialized NET plugin IB +host-b:6279:6279 [0] NCCL INFO Assigned NET plugin IB to comm +host-b:6279:6279 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-b:6279:6279 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-b:6279:6279 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-b:6279:6279 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-b:6279:6279 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-b:6279:6279 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-b:6279:6279 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-b:6279:6279 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-b:6279:6279 [0] NCCL INFO Using network IB +host-b:6279:6279 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-b:6279:6279 [0] NCCL INFO [Rank 1] ncclCommInitRank comm 0x36850bf0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9eea48d2e83cbf58 - Init START +host-b:6279:6279 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-b:6279:6279 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:6279:6279 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:6279:6279 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:6279:6279 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:6279:6279 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:6279:6279 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:6279:6279 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:6279:6279 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:6279:6279 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-b:6279:6279 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-b:6279:6279 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-b:6279:6279 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-b:6279:6279 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:6279:6279 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-b:6279:6279 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-b:6279:6279 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-b:6279:6279 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-b:6279:6279 [0] NCCL INFO Rank 1: 6 Net devices +host-b:6279:6279 [0] NCCL INFO Rank 1: 0 CollNet devices +host-b:6279:6279 [0] NCCL INFO comm 0x36850bf0 rank 1 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-b:6279:6279 [0] NCCL INFO Trees [0] -1/-1/-1->1->2 [1] -1/-1/-1->1->2 [2] -1/-1/-1->1->2 [3] -1/-1/-1->1->2 +host-b:6279:6279 [0] NCCL INFO P2P Chunksize set to 262144 +host-b:6279:6279 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-b:6279:6279 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-b:6279:6279 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-b:6279:6279 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-b:6279:6294 [0] NCCL INFO [Proxy Service] Device 0 CPU core 13 +host-b:6279:6294 [0] NCCL INFO proxy listening socket at <47369> +host-b:6279:6295 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 3 +host-b:6279:6279 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:6279:6296 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 13 +host-b:6279:6279 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:6279:6279 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[0] [receive] via NET/IB/4 +host-b:6279:6279 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[0] [receive] via NET/IB/5 +host-b:6279:6279 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:6279:6279 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:6279:6279 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [send] via NET/IB/4 +host-b:6279:6279 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [send] via NET/IB/5 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f4010000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f4054000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f409c000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2170 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2171 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2170 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2170 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2171 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1bdad7 ctsFifoLkey=0x1bdad7 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2171 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6714 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6715 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6716 pkey=0 pd=0xedf7f40f2420 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2298 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2299 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2298 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d958b ctsFifoLkey=0x1d958b +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2298 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2299 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1d958b ctsFifoLkey=0x1d958b +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2299 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f422e000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f4276000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f42ba000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f4302000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f4346000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2170 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2171 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2298 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2299 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6746 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6747 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6748 pkey=0 pd=0xedf7f40df4c0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2172 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2173 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2172 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b7977 ctsFifoLkey=0x1b7977 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2172 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2173 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b7977 ctsFifoLkey=0x1b7977 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2173 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6717 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6718 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6719 pkey=0 pd=0xedf7f40f2420 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2300 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2301 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2300 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1dcecc ctsFifoLkey=0x1dcecc +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2300 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2301 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1dcecc ctsFifoLkey=0x1dcecc +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2301 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6749 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6750 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6751 pkey=0 pd=0xedf7f40df4c0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2172 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2173 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2300 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2301 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6279 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-b:6279:6279 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:6279:6279 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:6279:6279 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [receive] via NET/IB/4 +host-b:6279:6279 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [receive] via NET/IB/5 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f45b5000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f45fa000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f463f000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f4684000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2174 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2175 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2176 pkey=0 pd=0xedf7f4053590 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2302 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2303 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2304 pkey=0 pd=0xedf7f40f2f90 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2177 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2178 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2179 pkey=0 pd=0xedf7f4053590 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2305 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2306 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2307 pkey=0 pd=0xedf7f40f2f90 +host-b:6279:6279 [0] NCCL INFO Connected all trees +host-b:6279:6279 [0] NCCL INFO Channel 00/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:6279:6279 [0] NCCL INFO Channel 01/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:6279:6279 [0] NCCL INFO Channel 02/0 : 1[0] -> 0[0] [send] via NET/IB/4 +host-b:6279:6279 [0] NCCL INFO Channel 03/0 : 1[0] -> 0[0] [send] via NET/IB/5 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f48a8000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f48e9000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6720 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6721 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6720 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17d896 ctsFifoLkey=0x17d896 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6720 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6721 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x17d896 ctsFifoLkey=0x17d896 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6721 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6752 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6753 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6752 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6752 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6753 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19edea ctsFifoLkey=0x19edea +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6753 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f49e0000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6720 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6721 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6752 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6753 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6722 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6723 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6722 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1826e3 ctsFifoLkey=0x1826e3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6722 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6723 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x1826e3 ctsFifoLkey=0x1826e3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6723 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f4a7b000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6754 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6755 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6754 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19e6e5 ctsFifoLkey=0x19e6e5 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6754 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6755 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x19e6e5 ctsFifoLkey=0x19e6e5 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6755 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6722 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6723 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6754 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6755 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6279 [0] NCCL INFO Connected binomial trees +host-b:6279:6279 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-b:6279:6279 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-b:6279:6279 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-b:6279:6279 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-b:6279:6279 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-b:6279:6279 [0] NCCL INFO ncclCommInitRank comm 0x36850bf0 rank 1 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9eea48d2e83cbf58 - Init COMPLETE +host-b:6279:6279 [0] NCCL INFO Init timings - ncclCommInitRank: rank 1 nranks 3 total 1.53 (kernels 0.24, alloc 0.03, bootstrap 0.94, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.30, rest 0.00) +host-b:6279:6279 [0] NCCL INFO Channel 02/1 : 0[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:6279:6279 [0] NCCL INFO Channel 03/1 : 0[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:6279:6279 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [send] via NET/IB/4/Shared +host-b:6279:6279 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [send] via NET/IB/5/Shared +host-b:6279:6279 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [receive] via NET/IB/5/Shared +host-b:6279:6279 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [receive] via NET/IB/4/Shared +host-b:6279:6279 [0] NCCL INFO Channel 01/1 : 1[0] -> 0[0] [send] via NET/IB/5/Shared +host-b:6279:6279 [0] NCCL INFO Channel 02/1 : 1[0] -> 0[0] [send] via NET/IB/4/Shared +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f4b1a000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f4b5b000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f4ba0000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f4be4000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f4c2c000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f4c70000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xedf7f4cb8000) +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xedf7f4cfc000) +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6724 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6725 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6726 pkey=0 pd=0xedf7f40f2420 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2308 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2309 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2308 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1decea ctsFifoLkey=0x1decea +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2308 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2309 mtu 3 GID 3 (0/1172A0AFFFF0000) ctsFifoRkey=0x1decea ctsFifoLkey=0x1decea +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2309 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6756 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6757 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6758 pkey=0 pd=0xedf7f40df4c0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6759 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=6760 pkey=0 pd=0xedf7f40df4c0 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6759 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x183a37 ctsFifoLkey=0x183a37 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6759 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 6760 mtu 3 GID 3 (0/20C2A0AFFFF0000) ctsFifoRkey=0x183a37 ctsFifoLkey=0x183a37 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 6760 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6727 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=6728 pkey=0 pd=0xedf7f40f2420 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6727 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x126121 ctsFifoLkey=0x126121 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6727 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 6728 mtu 3 GID 3 (0/20C290AFFFF0000) ctsFifoRkey=0x126121 ctsFifoLkey=0x126121 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 6728 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2180 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2181 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2180 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8c8b ctsFifoLkey=0x1b8c8b +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2180 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2181 mtu 3 GID 3 (0/117290AFFFF0000) ctsFifoRkey=0x1b8c8b ctsFifoLkey=0x1b8c8b +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2181 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6759 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 6760 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2180 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2181 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2308 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2309 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2310 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2311 pkey=0 pd=0xedf7f40f2f90 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2312 pkey=0 pd=0xedf7f40f2f90 +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6727 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 6728 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-b:6279:6294 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2182 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 1: rank 0 ended; leaving +ateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2183 pkey=0 pd=0xedf7f4053590 oooRq=0 +host-b:6279:6294 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2184 pkey=0 pd=0xedf7f4053590 +host-b:6279:6279 [0] NCCL INFO Symmetric VA size=124GB diff --git a/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/server-rank2.log b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/server-rank2.log new file mode 100644 index 0000000000..817324d007 --- /dev/null +++ b/docs/receipts/glm-kv-pages-8e705835/tp3/serve/r6off/server-rank2.log @@ -0,0 +1,3141 @@ +[kvp-serve] 2026-10-23T02:05:20+00:00 rank 2 of 3 context 1048576 parallel auto binary tensorfold-native eed66a8b83e10eb3 env TF_KV_PAGES=off TF_GLM_MM=unset args --prompt-cache-gib 0 +info: rank 2: TF_GLM_NCCL_ID_FILE /work/nccl-id; NCCL_COMM_ID :29587 leaves NCCL's environment +host-c:6242:6242 [0] NCCL INFO ncclOsDlopen(libnccl-env.so) failed: libnccl-env.so: cannot open shared object file: No such file or directory +host-c:6242:6242 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so +host-c:6242:6242 [0] NCCL INFO cudaDriverVersion 13030 +host-c:6242:6242 [0] NCCL INFO NCCL_SOCKET_IFNAME set to =tp3host0 +host-c:6242:6242 [0] NCCL INFO NCCL version 2.30.7+cuda13.3 +host-c:6242:6242 [0] NCCL INFO NCCL git version HEAD a5187fa2e +host-c:6242:6242 [0] NCCL INFO NCCL_NET_PLUGIN set by environment to none +host-c:6242:6242 [0] NCCL INFO Count not open libibverbs.so +host-c:6242:6242 [0] NCCL INFO dlvsym failed on mlx5dv_get_data_direct_sysfs_path - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_get_data_direct_sysfs_path, version MLX5_1.25 version MLX5_1.25 +host-c:6242:6242 [0] NCCL INFO dlvsym failed on mlx5dv_reg_dmabuf_mr - /lib/aarch64-linux-gnu/libmlx5.so.1: undefined symbol: mlx5dv_reg_dmabuf_mr, version MLX5_1.25 version MLX5_1.25 +host-c:6242:6242 [0] NCCL INFO NCCL_IB_HCA set to =rocep1s0f0:1,rocep1s0f1:1,roceP2p1s0f0:1,roceP2p1s0f1:1 +host-c:6242:6242 [0] NCCL INFO NET/IB: [0] rocep1s0f0:uverbs0:1/RoCE provider=Mlx5 speed=200000 context=0x2e1d1280 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:6242:6242 [0] NCCL INFO NET/IB: [1] rocep1s0f1:uverbs1:1/RoCE provider=Mlx5 speed=200000 context=0x2e212980 pciPath=/sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0 ar=0 oooRqSize=0 +host-c:6242:6242 [0] NCCL INFO NET/IB: [2] roceP2p1s0f0:uverbs2:1/RoCE provider=Mlx5 speed=200000 context=0x2e254080 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:6242:6242 [0] NCCL INFO NET/IB: [3] roceP2p1s0f1:uverbs3:1/RoCE provider=Mlx5 speed=200000 context=0x2e295780 pciPath=/sys/devices/pci0002:00/0002:00:00.0/0002:01:00.0 ar=0 oooRqSize=0 +host-c:6242:6242 [0] NCCL INFO NET/IB : Made virtual device [0] name=rocep1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:6242:6242 [0] NCCL INFO NET/IB : Made virtual device [1] name=rocep1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:6242:6242 [0] NCCL INFO NET/IB : Made virtual device [2] name=roceP2p1s0f0 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:6242:6242 [0] NCCL INFO NET/IB : Made virtual device [3] name=roceP2p1s0f1 speed=200000 ndevs=1 rail=-1 plane=-1 +host-c:6242:6242 [0] NCCL INFO NET/IB : Using [0]rocep1s0f0:1/RoCE [1]rocep1s0f1:1/RoCE [2]roceP2p1s0f0:1/RoCE [3]roceP2p1s0f1:1/RoCE [RO]; OOB tp3host0:<0> +host-c:6242:6242 [0] NCCL INFO Initialized NET plugin IB +host-c:6242:6242 [0] NCCL INFO Assigned NET plugin IB to comm +host-c:6242:6242 [0] NCCL INFO ncclOsDlopen(libnccl-rma.so) failed: libnccl-rma.so: cannot open shared object file: No such file or directory +host-c:6242:6242 [0] NCCL INFO RMA/Plugin: Could not find: libnccl-rma.so +host-c:6242:6242 [0] NCCL INFO RMA/Plugin: Failed to load external plugin libnccl-rma.so, dlHandle: (nil), ncclRma: (nil) +host-c:6242:6242 [0] NCCL INFO RMA/Plugin: Assigned plugin RMA_IB_PROXY to comm +host-c:6242:6242 [0] NCCL INFO ncclOsDlopen(libnccl-gin.so) failed: libnccl-gin.so: cannot open shared object file: No such file or directory +host-c:6242:6242 [0] NCCL INFO GIN/Plugin: Could not find: libnccl-gin.so +host-c:6242:6242 [0] NCCL INFO GIN/Plugin: Failed to load external plugin libnccl-gin.so, dlHandle: (nil), ncclGin: (nil) +host-c:6242:6242 [0] NCCL INFO GIN/Plugin: Assigned plugin GIN_IB_GDAKI type 3 to comm +host-c:6242:6242 [0] NCCL INFO Using network IB +host-c:6242:6242 [0] NCCL INFO Loaded NVML from libnvidia-ml.so.1 +host-c:6242:6242 [0] NCCL INFO [Rank 2] ncclCommInitRank comm 0x2b599bd0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9eea48d2e83cbf58 - Init START +host-c:6242:6242 [0] NCCL INFO RAS client listening socket at 127.0.0.1<28028> +host-c:6242:6242 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:6242:6242 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:6242:6242 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:6242:6242 [0] NCCL INFO NET/GIN_IB_GDAKI : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:6242:6242 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:6242:6242 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:6242:6242 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:6242:6242 [0] NCCL INFO NET/RMA_IB_PROXY : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:6242:6242 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 0 'rocep1s0f0' +host-c:6242:6242 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 1 'rocep1s0f1' +host-c:6242:6242 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 2 'roceP2p1s0f0' +host-c:6242:6242 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 3 'roceP2p1s0f1' +host-c:6242:6242 [0] NCCL INFO NET/IB : Made virtual device [4] name=rocep1s0f0+rocep1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:6242:6242 [0] NCCL INFO NET/IB : Made virtual device [5] name=roceP2p1s0f0+roceP2p1s0f1 speed=400000 ndevs=2 rail=-1 plane=16385 +host-c:6242:6242 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 4 'rocep1s0f0+rocep1s0f1' +host-c:6242:6242 [0] NCCL INFO NET/IB : GPU Direct RDMA Disabled for HCA 5 'roceP2p1s0f0+roceP2p1s0f1' +host-c:6242:6242 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is empty, ignoring. (GPU affinity = ; CPU affinity = 0-4,10-14). +host-c:6242:6242 [0] NCCL INFO Rank 2: 6 Net devices +host-c:6242:6242 [0] NCCL INFO Rank 2: 0 CollNet devices +host-c:6242:6242 [0] NCCL INFO comm 0x2b599bd0 rank 2 nRanks 3 nNodes 3 localRanks 1 localRank 0 MNNVL 0 +host-c:6242:6242 [0] NCCL INFO Trees [0] 1/-1/-1->2->0 [1] 1/-1/-1->2->0 [2] 1/-1/-1->2->0 [3] 1/-1/-1->2->0 +host-c:6242:6242 [0] NCCL INFO P2P Chunksize set to 262144 +host-c:6242:6242 [0] NCCL INFO Tuning P2P operations with maxP2pPeers = 3 +host-c:6242:6242 [0] NCCL INFO ncclOsDlopen(libnccl-profiler.so) failed: libnccl-profiler.so: cannot open shared object file: No such file or directory +host-c:6242:6242 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so +host-c:6242:6242 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 +host-c:6242:6258 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 2 +host-c:6242:6257 [0] NCCL INFO [Proxy Service] Device 0 CPU core 10 +host-c:6242:6257 [0] NCCL INFO proxy listening socket at <48569> +host-c:6242:6242 [0] NCCL INFO Channel 00/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:6242:6259 [0] NCCL INFO [Proxy Progress] Device 0 CPU core 14 +host-c:6242:6242 [0] NCCL INFO Channel 01/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:6242:6242 [0] NCCL INFO Channel 02/0 : 1[0] -> 2[0] [receive] via NET/IB/4 +host-c:6242:6242 [0] NCCL INFO Channel 03/0 : 1[0] -> 2[0] [receive] via NET/IB/5 +host-c:6242:6242 [0] NCCL INFO Channel 00/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:6242:6242 [0] NCCL INFO Channel 01/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:6242:6242 [0] NCCL INFO Channel 02/0 : 2[0] -> 0[0] [send] via NET/IB/4 +host-c:6242:6242 [0] NCCL INFO Channel 03/0 : 2[0] -> 0[0] [send] via NET/IB/5 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f70010000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f70054000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f7009c000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f700e0000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f70128000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f7016c000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f701b4000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f701f8000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2170 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2171 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2170 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcfcc ctsFifoLkey=0x1bcfcc +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2170 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2171 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bcfcc ctsFifoLkey=0x1bcfcc +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2171 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2298 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2299 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2298 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2298 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2299 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3635 ctsFifoLkey=0x1d3635 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2299 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2172 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2173 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2172 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2172 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2173 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1bafac ctsFifoLkey=0x1bafac +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2173 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1914 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1915 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1916 pkey=0 pd=0xf66f70283ed0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2300 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2301 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2300 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3f41 ctsFifoLkey=0x1d3f41 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2300 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2301 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d3f41 ctsFifoLkey=0x1d3f41 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2301 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2042 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2043 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2044 pkey=0 pd=0xf66f703b1930 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2170 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2171 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2298 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2299 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2172 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2173 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2300 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2301 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1917 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1918 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1919 pkey=0 pd=0xf66f70283ed0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2045 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2046 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2047 pkey=0 pd=0xf66f703b1930 +host-c:6242:6242 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 0 +host-c:6242:6242 [0] NCCL INFO Channel 00/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:6242:6242 [0] NCCL INFO Channel 01/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:6242:6242 [0] NCCL INFO Channel 02/0 : 0[0] -> 2[0] [receive] via NET/IB/4 +host-c:6242:6242 [0] NCCL INFO Channel 03/0 : 0[0] -> 2[0] [receive] via NET/IB/5 +host-c:6242:6242 [0] NCCL INFO Channel 00/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:6242:6242 [0] NCCL INFO Channel 01/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:6242:6242 [0] NCCL INFO Channel 02/0 : 2[0] -> 1[0] [send] via NET/IB/4 +host-c:6242:6242 [0] NCCL INFO Channel 03/0 : 2[0] -> 1[0] [send] via NET/IB/5 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f705ad000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f705ee000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f70633000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1920 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1921 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1920 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1920 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1921 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17febb ctsFifoLkey=0x17febb +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1921 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2174 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2175 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2176 pkey=0 pd=0xf66f70053eb0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2048 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2049 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2048 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2048 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2049 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x19e9e6 ctsFifoLkey=0x19e9e6 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2049 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f707a6000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1920 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1921 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2048 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2049 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2302 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2303 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2304 pkey=0 pd=0xf66f7009bf70 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f70860000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f708a4000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f708ec000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1922 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1923 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1922 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1804c2 ctsFifoLkey=0x1804c2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1922 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1923 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x1804c2 ctsFifoLkey=0x1804c2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1923 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2050 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2051 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2050 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197a79 ctsFifoLkey=0x197a79 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2050 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2051 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x197a79 ctsFifoLkey=0x197a79 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2051 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f709e6000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2177 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2178 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2179 pkey=0 pd=0xf66f70053eb0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2305 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2306 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2307 pkey=0 pd=0xf66f7009bf70 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1922 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1923 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2050 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2051 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6242 [0] NCCL INFO Connected all trees +host-c:6242:6242 [0] NCCL INFO Connected binomial trees +host-c:6242:6242 [0] NCCL INFO ncclOsDlopen(libnccl-tuner.so) failed: libnccl-tuner.so: cannot open shared object file: No such file or directory +host-c:6242:6242 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so +host-c:6242:6242 [0] NCCL INFO threadThresholds 8/8/64 | 24/8/64 | 512 | 512 +host-c:6242:6242 [0] NCCL INFO 4 coll channels, 4 collnet channels, 0 nvls channels, 4 p2p channels, 2 p2p channels per peer +host-c:6242:6242 [0] NCCL INFO Symmetric memory is not supported. cuMemEnable 0, globalGinSupport 0, cuMemGdrSupport 0 +host-c:6242:6242 [0] NCCL INFO ncclCommInitRank comm 0x2b599bd0 rank 2 nranks 3 cudaDev 0 nvmlDev 0 busId f01000 commId 0x9eea48d2e83cbf58 - Init COMPLETE +host-c:6242:6242 [0] NCCL INFO Init timings - ncclCommInitRank: rank 2 nranks 3 total 0.53 (kernels 0.24, alloc 0.03, bootstrap 0.01, allgathers 0.00, topo 0.01, graphs 0.00, connections 0.24, rest 0.00) +host-c:6242:6242 [0] NCCL INFO Channel 02/1 : 1[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:6242:6242 [0] NCCL INFO Channel 03/1 : 1[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:6242:6242 [0] NCCL INFO Channel 02/1 : 2[0] -> 0[0] [send] via NET/IB/4/Shared +host-c:6242:6242 [0] NCCL INFO Channel 03/1 : 2[0] -> 0[0] [send] via NET/IB/5/Shared +host-c:6242:6242 [0] NCCL INFO Channel 01/1 : 0[0] -> 2[0] [receive] via NET/IB/5/Shared +host-c:6242:6242 [0] NCCL INFO Channel 02/1 : 0[0] -> 2[0] [receive] via NET/IB/4/Shared +host-c:6242:6242 [0] NCCL INFO Channel 01/1 : 2[0] -> 1[0] [send] via NET/IB/5/Shared +host-c:6242:6242 [0] NCCL INFO Channel 02/1 : 2[0] -> 1[0] [send] via NET/IB/4/Shared +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f70b1e000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f70b5f000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f70ba4000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2180 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2181 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2180 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1ba09d ctsFifoLkey=0x1ba09d +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2180 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 1 IbDev 1 Port 1 qp_num 2181 mtu 3 GID 3 (0/20D290AFFFF0000) ctsFifoRkey=0x1ba09d ctsFifoLkey=0x1ba09d +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 1 Port 1 qp_num 2181 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f70c40000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f70c88000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f70ccc000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the send communicator (comm=0xf66f70d14000) +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbResiliencyInit: Resiliency is disabled on the recv communicator (comm=0xf66f70d58000) +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbRecvCommInit: Receive work requests will be posted on-demand +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2308 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2309 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2308 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d150c ctsFifoLkey=0x1d150c +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2308 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 3 IbDev 3 Port 1 qp_num 2309 mtu 3 GID 3 (0/20D2A0AFFFF0000) ctsFifoRkey=0x1d150c ctsFifoLkey=0x1d150c +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 3 Port 1 qp_num 2309 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2310 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2311 pkey=0 pd=0xf66f7009bf70 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=3 devName=roceP2p1s0f1 ndevs=4 nmdevs=6 qp_num=2312 pkey=0 pd=0xf66f7009bf70 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2182 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2183 pkey=0 pd=0xf66f70053eb0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=1 devName=rocep1s0f1 ndevs=4 nmdevs=6 qp_num=2184 pkey=0 pd=0xf66f70053eb0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 1 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2180 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 1 Port 1 qp_num 2181 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1924 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1925 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1926 pkey=0 pd=0xf66f70283ed0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 3 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2308 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 3 Port 1 qp_num 2309 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2052 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2053 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbReceiverQpsCreateToRts: Flush QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2054 pkey=0 pd=0xf66f703b1930 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2055 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=2 devName=roceP2p1s0f0 ndevs=4 nmdevs=6 qp_num=2056 pkey=0 pd=0xf66f703b1930 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2055 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x195448 ctsFifoLkey=0x195448 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2055 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 2 IbDev 2 Port 1 qp_num 2056 mtu 3 GID 3 (0/2172A0AFFFF0000) ctsFifoRkey=0x195448 ctsFifoLkey=0x195448 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 2 Port 1 qp_num 2056 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1927 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsCreate: QP created: port=1 dev=0 devName=rocep1s0f0 ndevs=4 nmdevs=6 qp_num=1928 pkey=0 pd=0xf66f70283ed0 oooRq=0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1927 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17d08d ctsFifoLkey=0x17d08d +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1927 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: NCCL Dev 0 IbDev 0 Port 1 qp_num 1928 mtu 3 GID 3 (0/217290AFFFF0000) ctsFifoRkey=0x17d08d ctsFifoLkey=0x17d08d +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbConnectImpl: IbDev 0 Port 1 qp_num 1928 query_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 5 with dev 2 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2055 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 2 Port 1 qp_num 2056 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NEinfo: checkpoint digest: 0.0 GiB read in 0.1 s, 33/33 cached files +info: rank 2: rank 0 ended; leaving +T/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: Subnet-aware routing: overriding dev 4 with dev 0 +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1927 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6257 [0] NCCL INFO NET/IB: ncclIbSenderQpsToRts: Set ECE: IbDev 0 Port 1 qp_num 1928 set_ece={supported=1, vendor_id=0x15b3, options=0x30000002, comp_mask=0x0} +host-c:6242:6242 [0] NCCL INFO Symmetric VA size=124GB diff --git a/zig/glm_snapshot_host.zig b/zig/glm_snapshot_host.zig index 5765b4dab6..f74ed6e880 100644 --- a/zig/glm_snapshot_host.zig +++ b/zig/glm_snapshot_host.zig @@ -358,3 +358,161 @@ test "GLM snapshots: FP8 prefix rows and separate scales restore for every rank try t.expectEqual(allocs, rec.allocs); }; } + +/// A paged sequence of the fixture's shape (a DSA and a KDA layer), its caches on the pages at `kv`, `ids` mapped. +fn pagedSeq(d: *const cuda.Driver, f: *const Fixture, kv: u64, page: u64, ids: []const u32) !glm.forward.State { + var w: glm.weights.Weights = undefined; + w.config = f.e.c; + w.share = f.e.share; + var layers: [2]glm.weights.Layer = undefined; + layers[0].mixer = .{ .mla = undefined }; + layers[1].mixer = .{ .kda = undefined }; + w.layers = &layers; + var st = try glm.forward.State.initPaged(t.allocator, d, &w, 16448, .{ .kv = kv, .pages = glm.pages.tableLen(16448), .page_bytes = page }); + st.pages = ids; + st.capacity = ids.len * 512; + return st; +} + +/// Row `r`'s first byte in a cache of `width`-byte rows, `per` rows a page when the cache is on pages. +fn rowAt(st: *const glm.forward.State, x: glm.dsa_block.Caches, base: u64, r: usize, width: usize, per: usize) u64 { + return if (x.pt == 0) base + r * width else base + @as(u64, st.pages[r / per]) * x.ps + (r % per) * width; +} + +fn same(a: u64, b: u64, len: usize) !void { + try t.expectEqualSlices(u8, rec.bytes(a, len), rec.bytes(b, len)); +} + +/// Every field a prefix at `at` reads (the MTP layer's `head` rows), in `a` and in `b`, from either layout. +fn expectSamePrefix(c: glm.config.Config, a: *const glm.forward.State, b: *const glm.forward.State, at: usize, head: ?usize) !void { + const kv = a.kda_layers * 8 * a.heads * 128; + try same(a.pos_dev, b.pos_dev, 4); + try same(a.conv, b.conv, a.kda_layers * 3 * a.conv_c * 2); + try same(a.rec, b.rec, 2 * a.kda_layers * a.rec_layer); + try same(a.proj, b.proj, a.kda_layers * a.proj_layer * 2); + try same(a.save_k, b.save_k, kv * 4); + try same(a.save_v, b.save_v, kv * 2); + try same(a.save_g, b.save_g, kv * 4); + try same(a.save_b, b.save_b, a.kda_layers * 8 * a.heads * 4); + for (0..a.caches.len + 1) |j| { + const n = if (j < a.caches.len) at else head orelse break; + const x = if (j < a.caches.len) a.caches[j] else a.mtp_caches; + const y = if (j < b.caches.len) b.caches[j] else b.mtp_caches; + const lat = c.kv_lora * 2; + for (0..n) |r| try same(rowAt(a, x, x.lc, r, lat, 512), rowAt(b, y, y.lc, r, lat, 512), lat); + for (0..2) |k| try same(a.rings[2 * j + k], b.rings[2 * j + k], @min(n, a.ring_rows) * a.ring_row); + for (0..@min(n / 4 + 2, x.pk_rows)) |p| try same(rowAt(a, x, x.pk, p, c.i_dim * 2, 128), rowAt(b, y, y.pk, p, c.i_dim * 2, 128), c.i_dim * 2); + } + try same(a.mtp_pos_dev, b.mtp_pos_dev, 4); + try same(a.prompt_hidden, b.prompt_hidden, c.hidden * 2); +} + +test "GLM snapshots: a paged sequence's prefix keeps and resumes through its pages in row order, to and from a window" { + var d = rec.init(); + for ([_]bool{ false, true }) |mtp| { + var f: Fixture = undefined; + try f.init(&d, 0, 2, mtp); + defer f.deinit(); + const c = f.e.c; + const page = 2 * (512 * c.kv_lora * 2 + 128 * c.i_dim * 2); // the DSA layer's part and the MTP layer's + const ids = [_]u32{ 5, 2, 9, 0, 7, 1, 3, 8, 4, 6 }; + var kv = try cuda.DeviceBuffer.alloc(&d, ids.len * page); + defer kv.free(); + var paged = f.seq; + paged.st = try pagedSeq(&d, &f, kv.ptr, page, &ids); + defer paged.st.deinit(); + const window = &f.seq; + try f.e.prepareSnapshots(3 * f.e.stateBytes(4096)); + for ([_]usize{ 511, 1537, 4096 }) |at| { + const head: ?usize = if (mtp) at + 7 else null; + // a window's prefix onto the pages: the paged arena's table stays its own + f.e.cur = window; + try f.mark(at, head orelse at, @intCast(at % 251)); + const id = try f.e.keepState(); + @memset(rec.bytes(paged.st.arena.buf.ptr, paged.st.arena.buf.len), 0xE9); + @memset(rec.bytes(kv.ptr, kv.len), 0xE9); + f.e.cur = &paged; + try f.e.restoreState(id); + try expectSamePrefix(c, &window.st, &paged.st, at, head); + try t.expectEqual(at, paged.st.pos); + for (rec.bytes(paged.st.table, ids.len * 4)) |b| try t.expectEqual(@as(u8, 0xE9), b); + f.e.dropState(id); + f.e.dropped.clearRetainingCapacity(); + // the pages' prefix back onto a window + const back = try f.e.keepState(); + @memset(rec.bytes(window.st.arena.buf.ptr, window.st.arena.buf.len), 0x5C); + f.e.cur = window; + try f.e.restoreState(back); + try expectSamePrefix(c, &paged.st, &window.st, at, head); + try t.expectEqual(at, window.st.pos); + f.e.dropState(back); + f.e.dropped.clearRetainingCapacity(); + } + f.e.cur = window; + } +} + +test "GLM snapshots: a paged part's piece from inside a page's span keeps its offset, on the page the order names" { + const ids = [_]u32{ 3, 0, 2 }; + const plan: glm.snapshots.layout.Plan = .{ .ids = &ids }; + const part: glm.snapshots.layout.Part = .{ .live = 1 << 20, .len = 250, .span = 100, .page = 4096 }; + // a snapshot page that ends inside a span leaves the rest of that span to the next piece + const a = plan.piece(part, 130); + try t.expectEqual(@as(u64, (1 << 20) + 30), a.live); + try t.expectEqual(@as(usize, 70), a.len); + const b = plan.piece(part, 230); + try t.expectEqual(@as(u64, (1 << 20) + 2 * 4096 + 30), b.live); + try t.expectEqual(@as(usize, 20), b.len); +} + +test "GLM snapshots: while a paged stream is current, a prefix's size is the engine's own sequence's, past its pages" { + var d = rec.init(); + var f: Fixture = undefined; + try f.init(&d, 0, 2, true); + defer f.deinit(); + f.e.seqs = .empty; + defer f.e.seqs.deinit(t.allocator); + try f.e.seqs.append(t.allocator, &f.seq); + const c = f.e.c; + const page = 2 * (512 * c.kv_lora * 2 + 128 * c.i_dim * 2); + const ids = [_]u32{ 1, 0 }; + var kv = try cuda.DeviceBuffer.alloc(&d, ids.len * page); + defer kv.free(); + var paged = f.seq; + paged.st = try pagedSeq(&d, &f, kv.ptr, page, &ids); + defer paged.st.deinit(); + const own = f.e.stateBytes(4096); + try t.expect(own < std.math.maxInt(u64)); + f.e.cur = &paged; + defer f.e.cur = &f.seq; + try t.expectEqual(own, f.e.stateBytes(4096)); +} + +test "GLM snapshots: with the engine's own sequence on pages too, a prefix's size past a stream's pages is a window's" { + var d = rec.init(); + var f: Fixture = undefined; + try f.init(&d, 0, 2, true); + defer f.deinit(); + const ats = [_]usize{ 511, 4096, 16000 }; + var sizes: [ats.len]u64 = undefined; + for (ats, &sizes) |at, *s| s.* = f.e.stateBytes(at); // the fixture's sequence holds a whole window + const c = f.e.c; + const page = 2 * (512 * c.kv_lora * 2 + 128 * c.i_dim * 2); + const ids = [_]u32{ 1, 0 }; + var kv = try cuda.DeviceBuffer.alloc(&d, ids.len * page); + defer kv.free(); + var paged = f.seq; + paged.st = try pagedSeq(&d, &f, kv.ptr, page, &ids); + defer paged.st.deinit(); + var own = f.seq; // the engine's own, on pages and holding none: no whole window anywhere + own.st = try pagedSeq(&d, &f, kv.ptr, page, &.{}); + defer own.st.deinit(); + f.e.seqs = .empty; + defer f.e.seqs.deinit(t.allocator); + try f.e.seqs.append(t.allocator, &own); + defer f.e.cur = &f.seq; + for ([_]*@TypeOf(f.seq){ &paged, &own }) |cur| { + f.e.cur = cur; + for (ats, sizes) |at, s| try t.expectEqual(s, f.e.stateBytes(at)); + } +} diff --git a/zig/kernels/cuda/glm_glue_attn.cuh b/zig/kernels/cuda/glm_glue_attn.cuh index a0c16a6d23..f61b4a4796 100644 --- a/zig/kernels/cuda/glm_glue_attn.cuh +++ b/zig/kernels/cuda/glm_glue_attn.cuh @@ -54,7 +54,7 @@ constexpr int wide_smem() { template __device__ void chunks_wide(const __nv_bfloat16* qa, const __nv_bfloat16* lc, const int* pos, const int* tok, const int* cnt, float* po, float* pm, float* pl, int64_t rows, int64_t heads, int64_t w, - float scale, int sparse, const float* scales = nullptr) { + float scale, int sparse, const int* pt, int64_t pstep, const float* scales = nullptr) { constexpr int MT = HB / 16; extern __shared__ __align__(16) unsigned char smem[]; __nv_bfloat16* qs = reinterpret_cast<__nv_bfloat16*>(smem); @@ -67,6 +67,8 @@ __device__ void chunks_wide(const __nv_bfloat16* qa, const __nv_bfloat16* lc, co const int64_t r = blockIdx.x, h0 = (int64_t)blockIdx.y * HB, c = blockIdx.z; const int64_t n = sparse ? cnt[r] : 0, limit = sparse ? 0 : (int64_t)pos[0] + r, start = c * CH; const bool live = sparse ? start < n : start <= limit; + // a dense chunk's 512 keys are one page, its rows from that page's; a sparse key's row comes as the token list has it + const __nv_bfloat16* keys_at = !FP8 && pt && !sparse && live ? lc + ((int64_t)pt[c] * (pstep / 2) - start * LW) : lc; auto load_kv = [&](int tile) { __nv_bfloat16* dst = kvs + (tile & 1) * KT * LW; const int64_t first = start + tile * KT; @@ -87,7 +89,7 @@ __device__ void chunks_wide(const __nv_bfloat16* qa, const __nv_bfloat16* lc, co dst[wide(key, u) + j + 1] = __float2bfloat16_rn(__fmul_rn(v.y, row_scale)); } } else { - cp16(dst + wide(key, u), lc + src * LW + u * 8, ok); + cp16(dst + wide(key, u), keys_at + src * LW + u * 8, ok); } } }; @@ -221,14 +223,16 @@ __device__ void chunks_wide(const __nv_bfloat16* qa, const __nv_bfloat16* lc, co // The 16-head form (decode windows) and 32-head form (prompt chunks), each in glm_attn::wide_smem() bytes of smem. extern "C" __global__ void __launch_bounds__(256) tf_glm_chunks_16(const __nv_bfloat16* qa, const __nv_bfloat16* lc, const int* pos, const int* tok, const int* cnt, - float* po, float* pm, float* pl, int64_t rows, int64_t heads, int64_t w, float scale, int sparse) { - glm_attn::chunks_wide<16>(qa, lc, pos, tok, cnt, po, pm, pl, rows, heads, w, scale, sparse); + float* po, float* pm, float* pl, int64_t rows, int64_t heads, int64_t w, float scale, int sparse, + const int* pt, int64_t pstep) { + glm_attn::chunks_wide<16>(qa, lc, pos, tok, cnt, po, pm, pl, rows, heads, w, scale, sparse, pt, pstep); } extern "C" __global__ void __launch_bounds__(256) tf_glm_chunks_32(const __nv_bfloat16* qa, const __nv_bfloat16* lc, const int* pos, const int* tok, const int* cnt, - float* po, float* pm, float* pl, int64_t rows, int64_t heads, int64_t w, float scale, int sparse) { - glm_attn::chunks_wide<32>(qa, lc, pos, tok, cnt, po, pm, pl, rows, heads, w, scale, sparse); + float* po, float* pm, float* pl, int64_t rows, int64_t heads, int64_t w, float scale, int sparse, + const int* pt, int64_t pstep) { + glm_attn::chunks_wide<32>(qa, lc, pos, tok, cnt, po, pm, pl, rows, heads, w, scale, sparse, pt, pstep); } // E4M3 cache variants reconstruct the existing BF16 shared tile before the unchanged attention math. @@ -237,14 +241,14 @@ extern "C" __global__ void __launch_bounds__(256) const int* tok, const int* cnt, float* po, float* pm, float* pl, int64_t rows, int64_t heads, int64_t w, float scale, int sparse) { glm_attn::chunks_wide<16, true>(qa, reinterpret_cast(lc), pos, tok, cnt, - po, pm, pl, rows, heads, w, scale, sparse, scales); + po, pm, pl, rows, heads, w, scale, sparse, nullptr, 0, scales); } extern "C" __global__ void __launch_bounds__(256) tf_glm_chunks_32_fp8(const __nv_bfloat16* qa, const unsigned char* lc, const float* scales, const int* pos, const int* tok, const int* cnt, float* po, float* pm, float* pl, int64_t rows, int64_t heads, int64_t w, float scale, int sparse) { glm_attn::chunks_wide<32, true>(qa, reinterpret_cast(lc), pos, tok, cnt, - po, pm, pl, rows, heads, w, scale, sparse, scales); + po, pm, pl, rows, heads, w, scale, sparse, nullptr, 0, scales); } // latent._merge, block (row, head), 128 threads: chunks merged in order, o / l by div.full; skips sparse count-0 rows. @@ -279,7 +283,7 @@ extern "C" __global__ void __launch_bounds__(128) extern "C" __global__ void __launch_bounds__(128) tf_glm_scores(const __nv_bfloat16* qi, const __nv_bfloat16* wts, int64_t w_stride, const __nv_bfloat16* pk, float* out, const int* pos, int64_t np, int64_t nb, float scale, float wscale, int64_t heads, - int loop) { + int loop, const int* pt, int64_t ps) { constexpr int D = 128, HP = 32, BP = 64; __shared__ __align__(16) __nv_bfloat16 sm[4 * 2 * 16 * D]; // q's 32 rows, then each warp's two 16-pool buffers auto at = [](int row, int u) { return row * D + ((u ^ (row & 7)) << 3); }; @@ -296,14 +300,15 @@ extern "C" __global__ void __launch_bounds__(128) const int64_t hh = g + 8 * i; wv[i] = hh < heads ? glm_triton::fmul(__bfloat162float(wts[r * w_stride + hh]), wscale) : 0.0f; } - // A block's pools for this warp: rows p0 .. p0 + 15 of pk, 16 chunks of 16 bytes each, 8 a lane. + // A block's pools for this warp: rows p0 .. p0 + 15 of pk (16 | p0: one page), 16 chunks of 16 bytes, 8 a lane. auto stage = [&](int buf, int64_t p0) { + const __nv_bfloat16* src = pk + glm_pages::at<128>(pt, ps / 2, p0 < npool ? p0 : 0, D); #pragma unroll for (int t = 0; t < 8; ++t) { const int idx = lane + t * 32, row = idx / (D / 8), u = idx % (D / 8); const int64_t p = p0 + row; const bool ok = p < npool; - glm_attn::cp16(sm + (warp * 2 + buf) * 16 * D + at(row, u), pk + (ok ? p : 0) * D + u * 8, ok); + glm_attn::cp16(sm + (warp * 2 + buf) * 16 * D + at(row, u), src + (ok ? row : 0) * D + u * 8, ok); } }; const int64_t first = (int64_t)blockIdx.y * loop; diff --git a/zig/kernels/cuda/glm_glue_index.cuh b/zig/kernels/cuda/glm_glue_index.cuh index ba1241cdec..f0cd03c1f9 100644 --- a/zig/kernels/cuda/glm_glue_index.cuh +++ b/zig/kernels/cuda/glm_glue_index.cuh @@ -37,10 +37,11 @@ extern "C" __global__ void __launch_bounds__(32) // sparse._pool_keys, a warp a completed pool: softmax of its 4 gate rows (+ ape) times bf16 keys, summed in order. extern "C" __global__ void __launch_bounds__(32) tf_glm_pool_keys(const __nv_bfloat16* ik, const __nv_bfloat16* ig, const __nv_bfloat16* ape, __nv_bfloat16* pk, - const int* pos, int64_t rows) { + const int* pos, int64_t rows, const int* pt, int64_t ps) { constexpr int D = 128; const int64_t P = pos[0], p = P / 4 + blockIdx.x; if (4 * p + 3 > P + rows - 1) return; + __nv_bfloat16* out = pk + glm_pages::at<128>(pt, ps / 2, p, D); // a page holds 512 rows' 128 pools for (int j = 0; j < 4; ++j) { const int d = threadIdx.x * 4 + j; float l[4], e[4]; @@ -53,7 +54,7 @@ extern "C" __global__ void __launch_bounds__(32) const float tq = rbf(__fmul_rn(rbf(div_full(e[q], s)), bf(ik[(4 * p + q) * D + d]))); t = q == 0 ? tq : t + tq; } - pk[p * D + d] = __float2bfloat16_rn(t); + out[d] = __float2bfloat16_rn(t); } } diff --git a/zig/kernels/cuda/glm_glue_latent.cuh b/zig/kernels/cuda/glm_glue_latent.cuh index a93830680f..2abdd9610d 100644 --- a/zig/kernels/cuda/glm_glue_latent.cuh +++ b/zig/kernels/cuda/glm_glue_latent.cuh @@ -66,11 +66,21 @@ extern "C" __global__ void __launch_bounds__(128) } } -// latent._lat_write (BF16 cache): row r of `lat` into cache slot pos + r. +namespace glm_pages { + +// Row r's first element in a cache of `width`-element rows: in order (pt null), or row r % P of page pt[r / P]. +template +__device__ __forceinline__ int64_t at(const int* pt, int64_t page_elems, int64_t r, int64_t width) { + return pt ? (int64_t)pt[r / P] * page_elems + (r % P) * width : r * width; +} + +} // namespace glm_pages + +// latent._lat_write (BF16 cache): row r of `lat` into cache slot pos + r (on pages, through the stream's table). extern "C" __global__ void tf_glm_lat_write(const __nv_bfloat16* lat, int64_t lat_stride, __nv_bfloat16* cache, - const int* pos, int64_t lw) { - const int64_t r = blockIdx.x, at = (int64_t)pos[0] + r; - for (int64_t k = threadIdx.x; k < lw; k += blockDim.x) cache[at * lw + k] = lat[r * lat_stride + k]; + const int* pos, int64_t lw, const int* pt, int64_t ps) { + const int64_t r = blockIdx.x, at = glm_pages::at<512>(pt, ps / 2, (int64_t)pos[0] + r, lw); + for (int64_t k = threadIdx.x; k < lw; k += blockDim.x) cache[at + k] = lat[r * lat_stride + k]; } #include "kv_fp8.cuh" diff --git a/zig/kernels/cuda/glm_ops.cu b/zig/kernels/cuda/glm_ops.cu index 7805cd23a6..adbf1abf14 100644 --- a/zig/kernels/cuda/glm_ops.cu +++ b/zig/kernels/cuda/glm_ops.cu @@ -3,19 +3,23 @@ #include #include -// sparse.select_tokens' assembly, a block a row: the K pools' 4 tokens each, then the last pool's visible tail. +constexpr long long PAGE = 512; // rows a KV page holds + +// sparse.select_tokens' assembly, a block a row: K pools' 4 tokens, then the tail; on pages each token's row in them. extern "C" __global__ void tf_glm_dsa_tokens(const long long* __restrict__ pools, const int* __restrict__ pos, int K, - int* __restrict__ tokens, int* __restrict__ counts) { + int* __restrict__ tokens, int* __restrict__ counts, + const int* __restrict__ pt, long long page_rows) { const int r = blockIdx.x; const long long q = static_cast(*pos) + r; const long long npool = (q + 1) / 4; const int width = 4 * K + 3; int* row = tokens + static_cast(r) * width; + auto at = [&](long long t) { return static_cast(pt ? pt[t / PAGE] * page_rows + t % PAGE : t); }; for (int i = threadIdx.x; i < 4 * K; i += blockDim.x) - row[i] = static_cast(pools[static_cast(r) * K + i / 4] * 4 + i % 4); + row[i] = at(pools[static_cast(r) * K + i / 4] * 4 + i % 4); if (threadIdx.x < 3) { const long long t = npool * 4 + threadIdx.x; - row[4 * K + threadIdx.x] = t <= q ? static_cast(t) : -1; + row[4 * K + threadIdx.x] = t <= q ? at(t) : -1; } if (threadIdx.x == 0) { int tail = 0; diff --git a/zig/src/core/contract/glm_cuda_records.zig b/zig/src/core/contract/glm_cuda_records.zig index 025f5f9022..bd2681ed51 100644 --- a/zig/src/core/contract/glm_cuda_records.zig +++ b/zig/src/core/contract/glm_cuda_records.zig @@ -1,4 +1,4 @@ -//! GLM-5.3-Flash's packed matmul and DFlash2 launches on CUDA, read by kernel-contract-limits from every SM's image. +//! GLM-5.3-Flash's packed matmul, DFlash2 and KV page launches, read by kernel-contract-limits from every SM's image. const registry = @import("registry.zig"); /// nvcc 13.0's usage, the compiler of the release fatbins. @@ -36,6 +36,12 @@ pub const cuda = [_]registry.CudaRecord{ .{ .launch = .{ .name = "df_top16_part", .symbol = "df_top16_part", .image = "glm_dflash", .threads = 256, .min_sm = 80 }, .usage = .{ .{ .regs = 32, .static = 32864, .bound = 256 }, .{ .regs = 32, .static = 32864, .bound = 256 }, .{ .regs = 32, .static = 32864, .bound = 256 }, .{ .regs = 30, .static = 32864, .bound = 256 }, .{ .regs = 30, .static = 32864, .bound = 256 } } }, .{ .launch = .{ .name = "df_top16_merge", .symbol = "df_top16_merge", .image = "glm_dflash", .threads = 256, .min_sm = 80 }, .usage = .{ .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 } } }, .{ .launch = .{ .name = "df_gsum", .symbol = "df_gsum", .image = "glm_dflash", .threads = 128, .min_sm = 80 }, .usage = .{ .{ .regs = 30, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 }, .{ .regs = 38, .static = 0, .bound = 0 } } }, + .{ .launch = .{ .name = "glm_glue tf_glm_lat_write", .symbol = "tf_glm_lat_write", .image = "glm_glue", .threads = 128, .min_sm = 80 }, .usage = .{ .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 }, .{ .regs = 36, .static = 0, .bound = 0 } } }, + .{ .launch = .{ .name = "glm_glue tf_glm_pool_keys", .symbol = "tf_glm_pool_keys", .image = "glm_glue", .threads = 32, .min_sm = 80 }, .usage = .{ .{ .regs = 25, .static = 0, .bound = 32 }, .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 26, .static = 0, .bound = 32 }, .{ .regs = 24, .static = 0, .bound = 32 }, .{ .regs = 24, .static = 0, .bound = 32 } } }, + .{ .launch = .{ .name = "glm_glue tf_glm_scores", .symbol = "tf_glm_scores", .image = "glm_glue", .threads = 128, .min_sm = 80 }, .usage = .{ .{ .regs = 126, .static = 32768, .bound = 128 }, .{ .regs = 131, .static = 32768, .bound = 128 }, .{ .regs = 131, .static = 32768, .bound = 128 }, .{ .regs = 155, .static = 32768, .bound = 128 }, .{ .regs = 155, .static = 32768, .bound = 128 } } }, + .{ .launch = .{ .name = "glm_glue tf_glm_chunks_16", .symbol = "tf_glm_chunks_16", .image = "glm_glue", .threads = 256, .dynamic = 83328, .opt_in = true, .min_sm = 80 }, .usage = .{ .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 64, .static = 0, .bound = 256 }, .{ .regs = 123, .static = 0, .bound = 256 }, .{ .regs = 123, .static = 0, .bound = 256 } } }, + .{ .launch = .{ .name = "glm_glue tf_glm_chunks_32", .symbol = "tf_glm_chunks_32", .image = "glm_glue", .threads = 256, .dynamic = 101120, .opt_in = true, .min_sm = 80 }, .usage = .{ .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 130, .static = 0, .bound = 256 }, .{ .regs = 130, .static = 0, .bound = 256 } } }, + .{ .launch = .{ .name = "glm_ops tf_glm_dsa_tokens", .symbol = "tf_glm_dsa_tokens", .image = "glm_ops", .threads = 256, .min_sm = 80 }, .usage = .{ .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 }, .{ .regs = 24, .static = 0, .bound = 0 } } }, }; /// nvcc 13.2's usage where it differs from 13.0's. @@ -47,3 +53,11 @@ pub const cuda_132 = [_]registry.Override{ .{ .name = "df_ctx_kv", .usage = .{ .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 30, .static = 8, .bound = 0 }, .{ .regs = 28, .static = 8, .bound = 0 }, .{ .regs = 28, .static = 8, .bound = 0 } } }, .{ .name = "df_top16_merge", .usage = .{ .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 23, .static = 8416, .bound = 256 }, .{ .regs = 28, .static = 8416, .bound = 256 }, .{ .regs = 28, .static = 8416, .bound = 256 } } }, }; + +/// nvcc 13.3's usage where it differs from 13.0's. +pub const cuda_133 = [_]registry.Override{ + .{ .name = "glm_glue tf_glm_lat_write", .usage = .{ .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 13, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 }, .{ .regs = 34, .static = 0, .bound = 0 } } }, + .{ .name = "glm_glue tf_glm_scores", .usage = .{ .{ .regs = 122, .static = 32768, .bound = 128 }, .{ .regs = 129, .static = 32768, .bound = 128 }, .{ .regs = 129, .static = 32768, .bound = 128 }, .{ .regs = 157, .static = 32768, .bound = 128 }, .{ .regs = 157, .static = 32768, .bound = 128 } } }, + .{ .name = "glm_glue tf_glm_chunks_32", .usage = .{ .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 150, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 }, .{ .regs = 128, .static = 0, .bound = 256 } } }, + .{ .name = "glm_ops tf_glm_dsa_tokens", .usage = .{ .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 25, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 }, .{ .regs = 22, .static = 0, .bound = 0 } } }, +}; diff --git a/zig/src/core/contract/registry.zig b/zig/src/core/contract/registry.zig index 13946b7fd9..1ff92cd267 100644 --- a/zig/src/core/contract/registry.zig +++ b/zig/src/core/contract/registry.zig @@ -308,7 +308,7 @@ pub const cuda_sets = [_]CudaSet{ .{ .release = .{ .major = 13, .minor = 3 }, .usage = &([_]Override{ .{ .name = "router tf_nemo_topk", .usage = .{ .{ .regs = 94, .static = 0, .bound = 32 }, .{ .regs = 95, .static = 0, .bound = 32 }, .{ .regs = 95, .static = 0, .bound = 32 }, .{ .regs = 96, .static = 0, .bound = 32 }, .{ .regs = 96, .static = 0, .bound = 32 } } }, .{ .name = "flashnext_batch fn_qsa_select", .usage = .{ .{ .regs = 32, .static = 1168, .bound = 1024 }, .{ .regs = 50, .static = 1168, .bound = 1024 }, .{ .regs = 50, .static = 1168, .bound = 1024 }, .{ .regs = 44, .static = 1168, .bound = 1024 }, .{ .regs = 44, .static = 1168, .bound = 1024 } } }, - } ++ @import("flashnext_tile_records.zig").cuda_133) }, + } ++ @import("flashnext_tile_records.zig").cuda_133 ++ @import("glm_cuda_records.zig").cuda_133) }, }; pub fn newer(x: limits.Nvcc, y: limits.Nvcc) bool { return x.major > y.major or (x.major == y.major and x.minor > y.minor); diff --git a/zig/src/core/kv_pool.zig b/zig/src/core/kv_pool.zig index 8e9d55c923..59130bfd52 100644 --- a/zig/src/core/kv_pool.zig +++ b/zig/src/core/kv_pool.zig @@ -10,7 +10,7 @@ pub const Layout = struct { token_bytes: u64, /// Tokens a page holds. page_tokens: u32, - /// Bytes of one kept snapshot's slot, outside the pages. + /// Bytes of one kept snapshot's slot, outside the pages (0: no slots, the family's kept states are copies alone). snap_bytes: u64, pub fn pageBytes(l: Layout) u64 { @@ -35,7 +35,7 @@ pub const most_snaps = 64; pub const Budget = struct { /// What the device grants past the weights. room: u64, - /// The engine's own full-window sequence, counted once. + /// The engine's own sequence, counted once: its full window, or its fixed part when its rows are pages. engine: u64, /// Streams asked for, the engine's own among them. lanes: u32, @@ -45,8 +45,10 @@ pub const Budget = struct { cache: u64 = 0, /// Snapshot slots (null: half the cache's bytes in slots); most_snaps at most. snaps: ?usize = null, - /// Rows one stream holds at most (0: no bound): pages past what every other stream holds at once stay unmade. + /// Rows one stream holds at most (0: no bound): pages past what the paged streams hold at once stay unmade. window: u64 = 0, + /// The engine's own rows are pages too: a window of them comes first, so one stream can still reach it. + engine_paged: bool = false, }; /// The streams served, the cache's slots and copied-KV bytes, and the pages left for the streams. @@ -54,14 +56,16 @@ pub const Plan = struct { lanes: u32, snaps: usize, copied: u64, pages: usize }; /// The prompt cache's split: slots for its snapshots, the rest for their copied KV rows (no slot: no cache). pub fn split(cache: u64, snap_bytes: u64, named: ?usize) struct { snaps: usize, copied: u64 } { + if (snap_bytes == 0) return .{ .snaps = 0, .copied = cache }; // a family without slots: its states are all copies const snaps = @min(most_snaps, named orelse cache / 2 / snap_bytes, cache / snap_bytes); return .{ .snaps = @intCast(snaps), .copied = if (snaps > 0) cache - snaps * snap_bytes else 0 }; } /// The pool as room - engine - (lanes - 1) x fixed - snaps x snap_bytes - copied, a page a stream beside it at least. pub fn plan(l: Layout, b: Budget) error{ NoStream, TooMany }!Plan { - if (b.engine > b.room or b.lanes == 0) return error.NoStream; - var left = b.room - b.engine; + const own = ownPages(l, b); + if (b.engine + own > b.room or b.lanes == 0) return error.NoStream; + var left = b.room - b.engine - own; const extra: u64 = b.lanes - 1; const each = l.fixed_bytes + l.pageBytes(); const fit = @min(extra, left / each); @@ -69,18 +73,24 @@ pub fn plan(l: Layout, b: Budget) error{ NoStream, TooMany }!Plan { left -= fit * l.fixed_bytes; const c = split(@min(b.cache, left - fit * l.pageBytes()), l.snap_bytes, b.snaps); left -= c.snaps * l.snap_bytes + c.copied; - const most: u64 = if (b.window > 0) fit * l.pages(b.window) else std.math.maxInt(u64); - return .{ .lanes = @intCast(1 + fit), .snaps = c.snaps, .copied = c.copied, .pages = @intCast(@min(left / l.pageBytes(), most)) }; + const most: u64 = if (b.window > 0) (fit + @intFromBool(b.engine_paged)) * l.pages(b.window) else std.math.maxInt(u64); + return .{ .lanes = @intCast(1 + fit), .snaps = c.snaps, .copied = c.copied, .pages = @intCast(@min((left + own) / l.pageBytes(), most)) }; } /// Whether `b.cache` fits whole in what `plan` leaves it: the room past the engine and a fixed part and page a stream. pub fn holds(l: Layout, b: Budget) bool { - if (b.engine > b.room or b.lanes == 0) return false; - const left = b.room - b.engine; + const own = ownPages(l, b); + if (b.engine + own > b.room or b.lanes == 0) return false; + const left = b.room - b.engine - own; const each = l.fixed_bytes + l.pageBytes(); return left - @min(@as(u64, b.lanes - 1), left / each) * each >= b.cache; } +/// The bytes of the engine's own window as pages, set aside first when its rows are pages (0 when it has a window). +fn ownPages(l: Layout, b: Budget) u64 { + return if (b.engine_paged) l.pages(b.window) * l.pageBytes() else 0; +} + /// A stream's reservation: pages promised but not yet taken, and the pages it holds in token order (its table). pub const Lease = struct { promised: usize = 0, diff --git a/zig/src/core/kv_pool_test.zig b/zig/src/core/kv_pool_test.zig index 47196e1856..02ddff9f48 100644 --- a/zig/src/core/kv_pool_test.zig +++ b/zig/src/core/kv_pool_test.zig @@ -52,6 +52,53 @@ test "the plan: the room pays the engine's own sequence, the other streams' fixe try expectEqual(@as(usize, kv.most_snaps), kv.split(16 * g, 48 * mib, 1000).snaps); } +test "a family without snapshot slots (snap_bytes 0) keeps its whole cache as copied rows beside the pages" { + const g: u64 = 1 << 30; + // GLM-5.3-Flash on CUDA at TP=2: KDA states and rings fixed a stream, 13,056 bytes a token, kept states copied + const copies: kv.Layout = .{ .fixed_bytes = 180 * mib, .token_bytes = 13_056, .page_tokens = 512, .snap_bytes = 0 }; + const s = kv.split(16 * g, 0, null); + try expectEqual(@as(usize, 0), s.snaps); + try expectEqual(16 * g, s.copied); + try expectEqual(16 * g, kv.split(16 * g, 0, 4).copied); // --keep names no slots here + const p = try kv.plan(copies, .{ .room = 60 * g, .engine = 0, .lanes = 4, .cache = 16 * g }); + try expectEqual(@as(usize, 0), p.snaps); + try expectEqual(16 * g, p.copied); + try expectEqual(@as(usize, @intCast((44 * g - 3 * 180 * mib) / copies.pageBytes())), p.pages); +} + +test "the engine's own sequence on pages: a window of them comes first, then the others' fixed parts, the cache, the rest" { + const g: u64 = 1 << 30; + // GLM-5.3-Flash on CUDA at TP=3, a 1M window and a verify window; the engine's own fixed part is in the model + const glm: kv.Layout = .{ .fixed_bytes = 180 * mib, .token_bytes = 13_056, .page_tokens = 512, .snap_bytes = 0 }; + const window: u64 = 1_048_576 + 8; + const floor = glm.pages(window) * glm.pageBytes(); + try expectEqual(@as(usize, 2049), glm.pages(window)); // one request at --context + const p = try kv.plan(glm, .{ .room = 40 * g, .engine = 0, .lanes = 4, .cache = 16 * g, .window = window, .engine_paged = true }); + try expectEqual(@as(u32, 4), p.lanes); + try expectEqual(16 * g, p.copied); + try expectEqual(@as(usize, @intCast((24 * g - 3 * 180 * mib) / glm.pageBytes())), p.pages); + // the window's pages before the cache: the cache gets what they and a page a stream leave + const each = 180 * mib + glm.pageBytes(); + const tight = try kv.plan(glm, .{ .room = floor + 3 * each + g, .engine = 0, .lanes = 4, .cache = 16 * g, .window = window, .engine_paged = true }); + try expectEqual(g, tight.copied); + try expectEqual(@as(usize, 2049 + 3), tight.pages); + // a room under one window's pages serves no stream; at it, the engine's own alone + try std.testing.expectError(error.NoStream, kv.plan(glm, .{ .room = floor - 1, .engine = 0, .lanes = 4, .window = window, .engine_paged = true })); + const alone = try kv.plan(glm, .{ .room = floor + each - 1, .engine = 0, .lanes = 4, .window = window, .engine_paged = true }); + try expectEqual(@as(u32, 1), alone.lanes); + try expectEqual(@as(usize, 2049), alone.pages); + try std.testing.expectError(error.TooMany, kv.plan(glm, .{ .room = floor + each - 1, .engine = 0, .lanes = 4, .fixed = true, .window = window, .engine_paged = true })); + // no more pages than every stream holds at once, the engine's own among them + const bounded = try kv.plan(glm, .{ .room = 100 * g, .engine = 0, .lanes = 2, .window = 8192 + 8, .engine_paged = true }); + try expectEqual(@as(usize, 2 * 17), bounded.pages); + // what holds is what plan gives the cache whole, past the window's pages + const room = floor + 3 * each + 16 * g; + try expect(kv.holds(glm, .{ .room = room, .engine = 0, .lanes = 4, .cache = 16 * g, .window = window, .engine_paged = true })); + try expect(!kv.holds(glm, .{ .room = room - 1, .engine = 0, .lanes = 4, .cache = 16 * g, .window = window, .engine_paged = true })); + try expect(!kv.holds(glm, .{ .room = floor - 1, .engine = 0, .lanes = 1, .window = window, .engine_paged = true })); + try expectEqual(16 * g, (try kv.plan(glm, .{ .room = room, .engine = 0, .lanes = 4, .cache = 16 * g, .window = window, .engine_paged = true })).copied); +} + test "a cache holds when the room past the engine and every stream's fixed part and page is at least its bytes" { const g: u64 = 1 << 30; const streams = 3 * (64 * mib + nemotron.pageBytes()); diff --git a/zig/src/families/glm/cuda.zig b/zig/src/families/glm/cuda.zig index 7fe9c1a198..ddfcf34c28 100644 --- a/zig/src/families/glm/cuda.zig +++ b/zig/src/families/glm/cuda.zig @@ -29,6 +29,7 @@ pub const sampler = @import("cuda_sampler.zig"); pub const prompt_sink = @import("cuda_prompt_sink.zig"); pub const prompt_logprobs = @import("cuda_prompt_logprobs.zig"); pub const engine = @import("cuda_engine.zig"); +pub const pages = @import("cuda_pages.zig"); pub const follow = @import("cuda_follow.zig"); pub const lanes = @import("cuda_lanes.zig"); pub const native = @import("cuda_native.zig"); @@ -41,6 +42,7 @@ test { _ = @import("cuda_prof.zig"); _ = @import("cuda_load_owner.zig"); _ = @import("cuda_images_interleave_test.zig"); + _ = @import("cuda_pages_test.zig"); _ = experts; _ = exl3; _ = vision; diff --git a/zig/src/families/glm/cuda_dsa_block.zig b/zig/src/families/glm/cuda_dsa_block.zig index 1d5779cf29..e79ed33822 100644 --- a/zig/src/families/glm/cuda_dsa_block.zig +++ b/zig/src/families/glm/cuda_dsa_block.zig @@ -35,7 +35,16 @@ pub const Layer = struct { }; /// The caches: latent bf16 [capacity][lw], index keys/gates bf16 [capacity][128], pooled keys bf16 [pk_rows][128]. -pub const Caches = struct { lc: u64, ls: u64 = 0, ik: u64, ig: u64, pk: u64, pk_rows: usize }; +pub const Caches = struct { + lc: u64, + ls: u64 = 0, + ik: u64, + ig: u64, + pk: u64, + pk_rows: usize, + pt: u64 = 0, // on pages: the stream's table (i32 page ids), lc and pk then this cache's place in page 0; 0: rows + ps: u64 = 0, // on pages: bytes from one page to the next +}; /// The block's scratch for up to `rows` rows. pub const Scratch = struct { @@ -124,11 +133,13 @@ pub fn attend(k: Kernels, stream: cuda.Stream, l: Layer, c: Caches, s: Scratch, v.qa += r0 * l.heads * l.lw * 2; v.ol += r0 * l.heads * l.lw * 2; v.qi += r0 * ix.heads * ix.d * 2; + if (c.ps % (l.lw * 2) != 0) return error.LatentShape; // pages whole latent rows apart (the token lists' unit) + const t: latent.Table = .{ .pt = c.pt, .ps = c.ps, .page_rows = c.ps / (l.lw * 2) }; var stamp = prof.span(k.prof, stream, .cache_write); - try latent.latentWriteFormat(set, stream, v.lat, l.lw, c.lc, c.ls, pos, rows, l.lw); + try latent.latentWriteFormat(set, stream, v.lat, l.lw, c.lc, c.ls, t, pos, rows, l.lw); stamp.end(); stamp = prof.span(k.prof, stream, .index_update); - try sparse.indexUpdate(set, stream, v.ikr, kw_rows, v.igr, ix.ln_w, ix.ln_b, ix.ape, c.ik, c.ig, c.pk, pos, rows); + try sparse.indexUpdate(set, stream, v.ikr, kw_rows, v.igr, ix.ln_w, ix.ln_b, ix.ape, c.ik, c.ig, c.pk, t, pos, rows); stamp.end(); const scale: f32 = @floatCast(std.math.pow(f64, @floatFromInt(l.qk), -0.5)); if (host_pos < l.dense_limit) { @@ -137,10 +148,10 @@ pub fn attend(k: Kernels, stream: cuda.Stream, l: Layer, c: Caches, s: Scratch, stamp.end(); } if (sparseRows(l, host_pos, rows)) { - try sparse.selectTokensProfile(set, k.ops, stream, v.qi, v.ikr + ix.d * 2, kw_rows, c.pk, pos, host_pos, rows, ix.heads, ix.d, c.pk_rows - 2, v.select, v.tokens, v.counts, k.prof); + try sparse.selectTokensProfile(set, k.ops, stream, v.qi, v.ikr + ix.d * 2, kw_rows, c.pk, t, pos, host_pos, rows, ix.heads, ix.d, c.pk_rows - 2, v.select, v.tokens, v.counts, k.prof); stamp = prof.span(k.prof, stream, .mla); defer stamp.end(); - try latent.sparseAttentionFormat(set, stream, v.qa, c.lc, c.ls, v.tokens, v.counts, sparse.width, v.partials, v.ol, rows, l.heads, l.lw, scale); + try latent.sparseAttentionFormat(set, stream, v.qa, c.lc, c.ls, t, v.tokens, v.counts, sparse.width, v.partials, v.ol, rows, l.heads, l.lw, scale); } } @@ -160,6 +171,6 @@ fn denseAttention(k: Kernels, stream: cuda.Stream, l: Layer, c: Caches, s: Scrat at = s.at; } const off = r0 * l.heads * l.lw * 2; - try latent.attentionFormat(k.set, stream, s.qa + off, c.lc, c.ls, at, s.partials, s.ol + off, s.dummy, n, l.heads, l.lw, nch, scale, hb); + try latent.attentionFormat(k.set, stream, s.qa + off, c.lc, c.ls, .{ .pt = c.pt, .ps = c.ps }, at, s.partials, s.ol + off, s.dummy, n, l.heads, l.lw, nch, scale, hb); } } diff --git a/zig/src/families/glm/cuda_engine.zig b/zig/src/families/glm/cuda_engine.zig index 3e9da65961..a15f5cc181 100644 --- a/zig/src/families/glm/cuda_engine.zig +++ b/zig/src/families/glm/cuda_engine.zig @@ -17,6 +17,7 @@ const tile = @import("cuda_tile.zig"); const mm_mod = @import("cuda_mm.zig"); const dflash = @import("dflash.zig"); const prof_mod = @import("cuda_prof.zig"); +const pages_mod = @import("cuda_pages.zig"); pub const Options = @import("cuda_engine_types.zig").Options; /// Rank 0's own word on its request (cancelled or not); the other ranks have none and hear rank 0's. @@ -91,6 +92,7 @@ pub const Engine = struct { prompt_tried: bool = false, // that pass came: without buffers after it, the passes take the window's prompt_out: prompt_lp.Output, // a head pass's logprob words, then its picks, host-mapped (rank 0 reads them) structured: bool = false, // a structured output step: every rank's draws gather whole rows, rank 0 masks them + pages: ?*pages_mod.Pages = null, // the other streams' KV pages and sequences (cuda_pages), once the server attached them /// Opens rank `rank` of `world` over `comm`; the ranks agree on settings (cuda_ranks) before the weights load. pub fn init(gpa: std.mem.Allocator, io: std.Io, d: *const cuda.Driver, ctx: *const cuda.Context, checkpoint: []const u8, lib: *const cuda.nccl.Library, comm: cuda.nccl.Comm, rank: usize, world: usize, o: Options) !*Engine { @@ -165,7 +167,7 @@ pub const Engine = struct { e.m.mm.prompt_fixed_k = o.settings.prompt_fixed_k; e.m.tiled = o.settings.tiled; if (o.settings.tiled) e.m.packs = &e.packs; - try e.bind(0); + if (o.settings.pages) try pages_mod.ownSequence(e) else try e.bind(0); // on pages: no window, its pool places it errdefer { e.free(0); e.seqs.deinit(gpa); @@ -220,6 +222,7 @@ pub const Engine = struct { e.bm.deinit(); e.bd.deinit(); e.bp.deinit(); + if (e.pages) |p| p.deinit(); for (0..e.seqs.items.len) |i| e.free(i); e.seqs.deinit(e.gpa); e.w.deinit(); @@ -257,10 +260,11 @@ pub const Engine = struct { if (e.images) |v| try v.activate(slot); } - /// Frees slot `slot`'s sequence (slot 0 stays: the first stream's, made at init) and binds slot 0. + /// Frees slot `slot`'s sequence (slot 0 stays, and a paged one gives its pages back) and binds slot 0. pub fn release(e: *Engine, slot: u32) void { if (e.images) |v| v.release(slot); - if (slot != 0) e.free(slot); + const paged = if (e.pages) |p| p.paged(slot) else false; + if (paged) e.pages.?.close(slot) else if (slot != 0) e.free(slot); e.cur = e.seqs.items[0].?; e.slot = 0; } diff --git a/zig/src/families/glm/cuda_follow.zig b/zig/src/families/glm/cuda_follow.zig index b936feb740..771ea8d147 100644 --- a/zig/src/families/glm/cuda_follow.zig +++ b/zig/src/families/glm/cuda_follow.zig @@ -8,9 +8,10 @@ const vcfg = @import("vision_config.zig"); const sampler = @import("cuda_sampler.zig"); const prompt_wire = @import("cuda_prompt_wire.zig"); const dflash = @import("dflash.zig"); +const pages_mod = @import("cuda_pages.zig"); const Engine = engine.Engine; -pub const Kind = enum(u32) { begin = 1, prefill, verify, keep, draft, end, drop, release, chunk, shared, image, block, many, redraw, idle, snapshots }; +pub const Kind = enum(u32) { begin = 1, prefill, verify, keep, draft, end, drop, release, chunk, shared, image, block, many, redraw, idle, snapshots, pool }; /// One step as it crosses: 64 bytes, then `n` u32 ids. pub const Header = extern struct { @@ -170,6 +171,11 @@ pub const Leader = struct { } pub fn begin(x: Leader, s: ?sampler.Sampling) !void { + return x.beginOn(s, &.{}); + } + + /// `begin` on a paged sequence: the page ids it maps (rank 0's lease) go with the step, to every rank's table. + pub fn beginOn(x: Leader, s: ?sampler.Sampling, pages: []const u32) !void { try x.flush(); const c = sampler.check(s); var h: Header = .{ .kind = @backingInt(Kind.begin) }; @@ -181,7 +187,7 @@ pub const Leader = struct { h.min_p = v.min_p; h.top_k = v.top_k; } - try x.send(h, &.{}); + try x.send(h, pages); try x.e.begin(c); } @@ -191,6 +197,11 @@ pub const Leader = struct { try @import("cuda_snapshots.zig").agree(x.e, budget); } + /// The other ranks make rank 0's pool: `streams` paged sequences and `pages` pages, each page's bytes checked. + pub fn pool(x: Leader, streams: u32, pages: usize, page_bytes: u64) !void { + try x.send(.{ .kind = @backingInt(Kind.pool), .depth = streams, .a = pages, .seed = page_bytes }, &.{}); + } + /// The prompt pass's image rows (rank 0 encoded them for the bound slot) to every rank: spans, then the rows. pub fn images(x: Leader, v: *const images_mod.Images) !void { try x.flush(); @@ -304,11 +315,18 @@ pub fn follow(e: *Engine, link: *Link) !void { const step = try link.receive(); const h = step.h; const kind: Kind = @fromBackingInt(@intCast(h.kind)); - if (kind != .end and kind != .drop and kind != .shared and kind != .block and kind != .many and kind != .idle and kind != .snapshots) try e.bind(h.slot); + if (kind != .end and kind != .drop and kind != .shared and kind != .block and kind != .many and kind != .idle and kind != .snapshots and kind != .pool) try e.bind(h.slot); e.prompt_logprobs = if (promptRows(h)) .{} else null; e.structured = structuredStep(h); // rank 0 alone masks them switch (kind) { - .begin => try e.beginFollowing(if (h.flags & 1 != 0) .{ .seed = h.seed, .temperature = h.temperature, .top_p = h.top_p, .min_p = h.min_p, .top_k = h.top_k } else null), + .begin => { + if (step.ids.len > 0) try (e.pages orelse return error.BadStep).map(h.slot, step.ids); + try e.beginFollowing(if (h.flags & 1 != 0) .{ .seed = h.seed, .temperature = h.temperature, .top_p = h.top_p, .min_p = h.min_p, .top_k = h.top_k } else null); + }, + .pool => { + if (e.pages != null) return error.BadStep; + e.pages = try pages_mod.Pages.follow(e.gpa, e, h.depth, @intCast(h.a), h.seed); + }, .prefill => { const ids = try e.gpa.dupe(u32, step.ids); defer e.gpa.free(ids); diff --git a/zig/src/families/glm/cuda_kv8_test.zig b/zig/src/families/glm/cuda_kv8_test.zig index 01d7210c01..285660b1fa 100644 --- a/zig/src/families/glm/cuda_kv8_test.zig +++ b/zig/src/families/glm/cuda_kv8_test.zig @@ -109,25 +109,43 @@ test "GLM FP8 KV: real launchers preserve BF16 ABI and pass FP8 scale pointers f const set: glue.Set = .{ .glue = &ks }; const stream: cuda.Stream = .{ .d = &d, .handle = null }; Launch.n = 0; - try latent.latentWriteFormat(set, stream, 100, 512, 200, 0, 300, 3, 512); - try latent.latentWriteFormat(set, stream, 100, 512, 200, 400, 300, 3, 512); + try latent.latentWriteFormat(set, stream, 100, 512, 200, 0, .{}, 300, 3, 512); + try latent.latentWriteFormat(set, stream, 100, 512, 200, 400, .{}, 300, 3, 512); try t.expectEqualSlices(usize, &.{ 1, 2 }, Launch.ids[0..2]); try t.expectEqualSlices(u32, &.{ 128, 512 }, Launch.blocks[0..2]); try t.expectEqual(@as(u64, 400), Launch.scales[1]); const p: latent.Partials = .{ .po = 1000, .pm = 2000, .pl = 3000 }; for ([_]usize{ 16, 32 }) |hb| for ([_]u64{ 0, 400 }) |scale| { Launch.n = 0; - try latent.attentionFormat(set, stream, 100, 200, scale, 300, p, 500, 600, 3, 32, 512, 1, 0.125, hb); + try latent.attentionFormat(set, stream, 100, 200, scale, .{}, 300, p, 500, 600, 3, 32, 512, 1, 0.125, hb); try t.expectEqual(@as(usize, if (hb == 16) 3 else 5) + @intFromBool(scale != 0), Launch.ids[0]); try t.expectEqual(scale, Launch.scales[0]); try t.expectEqual(@as(usize, 7), Launch.ids[1]); }; Launch.n = 0; - try latent.sparseAttentionFormat(set, stream, 100, 200, 400, 500, 600, 2051, p, 700, 3, 32, 512, 0.125); + try latent.sparseAttentionFormat(set, stream, 100, 200, 400, .{}, 500, 600, 2051, p, 700, 3, 32, 512, 0.125); try t.expectEqual(@as(usize, 4), Launch.ids[0]); try t.expectEqual(@as(u64, 400), Launch.scales[0]); } +test "GLM FP8 KV: an FP8 cache takes no page table, so its writes and reads refuse one before they launch" { + const glue = @import("cuda_glue.zig"); + const latent = @import("cuda_latent.zig"); + var d: cuda.Driver = undefined; + d.api.cuLaunchKernel = Launch.run; + var ks: glue.Kernels = undefined; + inline for (@typeInfo(glue.Kernels).@"struct".field_names) |name| @field(ks, name) = .{ .d = &d, .handle = @ptrFromInt(10) }; + const set: glue.Set = .{ .glue = &ks }; + const stream: cuda.Stream = .{ .d = &d, .handle = null }; + const paged: latent.Table = .{ .pt = 900, .ps = 6_684_672, .page_rows = 6_528 }; + const p: latent.Partials = .{ .po = 1000, .pm = 2000, .pl = 3000 }; + Launch.n = 0; + try t.expectError(error.Fp8OnPages, latent.latentWriteFormat(set, stream, 100, 512, 200, 400, paged, 300, 3, 512)); + try t.expectError(error.Fp8OnPages, latent.attentionFormat(set, stream, 100, 200, 400, paged, 300, p, 500, 600, 3, 32, 512, 1, 0.125, 16)); + try t.expectError(error.Fp8OnPages, latent.sparseAttentionFormat(set, stream, 100, 200, 400, paged, 500, 600, 2051, p, 700, 3, 32, 512, 0.125)); + try t.expectEqual(@as(usize, 0), Launch.n); +} + const Ops = struct { fn function(out: *cuda.abi.Function, _: cuda.abi.Module, name: [*:0]const u8) callconv(.c) c_int { out.* = @ptrFromInt(if (std.mem.eql(u8, std.mem.span(name), "tf_glm_dsa_tokens")) @as(usize, 15) else 14); diff --git a/zig/src/families/glm/cuda_lanes.zig b/zig/src/families/glm/cuda_lanes.zig index 29d344a7ee..c18ef215ec 100644 --- a/zig/src/families/glm/cuda_lanes.zig +++ b/zig/src/families/glm/cuda_lanes.zig @@ -248,17 +248,41 @@ pub const Lanes = struct { fn opening(self: *Lanes, s: *lanes.Stream) !?engine.Resume { const ids = s.prompt(); if (ids.len + s.max_new + max_rows >= self.x.e.capacity) return error.PromptTooLong; + const pages = try self.seat(s, ids.len, s.max_new); + if (self.x.e.images) |v| v.clear(); // no earlier request's rows splice into this one + try self.x.beginOn(s.sampling, pages); + s.cached = 0; + s.prompt_rows.clearRetainingCapacity(); + return if (s.prompt_logprobs != null) null else self.resumeFrom(s, ids.len); // prompt rows need every row + } + + /// The stream's slot, its pages, then its sequence: a slot past the paged ones waits before binding makes a window. + pub fn seat(self: *Lanes, s: *const lanes.Stream, prompt: usize, max_new: usize) ![]const u32 { const l = try self.claim(s); l.pending_rows = null; l.held_n = 0; l.fill = null; l.first = null; + const pages = self.pagesFor(l.slot, prompt, max_new) catch |err| { + _ = self.lanes.remove(s); // nothing held or made: a LaneMemory request waits for a stream to end + return err; + }; try self.x.bind(l.slot); - if (self.x.e.images) |v| v.clear(); // no earlier request's rows splice into this one - try self.x.begin(s.sampling); - s.cached = 0; - s.prompt_rows.clearRetainingCapacity(); - return if (s.prompt_logprobs != null) null else self.resumeFrom(s, ids.len); // prompt rows need every row + return pages; + } + + /// A slot's pages for a prompt and its reply (none without a pool); LaneMemory: the pool lacks them now. + fn pagesFor(self: *Lanes, slot: u32, prompt: usize, max_new: usize) ![]const u32 { + const p = self.x.e.pages orelse return &.{}; + if (!p.paged(slot)) return error.LaneMemory; // every paged sequence is taken + p.close(slot); // a stream that opens again gives its earlier pages back first + return p.open(slot, prompt, max_new); + } + + /// The lane host's admission: with a pool, a paged sequence and the request's pages are free. + pub fn fits(self: *const Lanes, prompt: usize, max_tokens: usize) bool { + const p = self.x.e.pages orelse return true; + return p.fits(prompt, max_tokens); } /// A prompt pass with images: rank 0's tower encodes those past `from`, then every rank holds their rows. @@ -479,7 +503,7 @@ pub const Lanes = struct { return true; } - /// The stream left the rounds: its sequence is freed on every rank (slot 0 stays for the next stream). + /// The stream left the rounds: its sequence is freed, or its pages given back, on every rank (slot 0 stays). fn releaseFn(ptr: *anyopaque, s: *lanes.Stream) void { const self = of(ptr); const kv = self.lanes.fetchRemove(s) orelse return; diff --git a/zig/src/families/glm/cuda_latent.zig b/zig/src/families/glm/cuda_latent.zig index c9ca154092..8cc7b2eeb9 100644 --- a/zig/src/families/glm/cuda_latent.zig +++ b/zig/src/families/glm/cuda_latent.zig @@ -53,18 +53,23 @@ pub fn cacheBytes(capacity: usize, lw: usize, kv8: bool) !usize { return std.math.mul(usize, capacity, row); } -pub fn latentWriteFormat(set: glue.Set, stream: cuda.Stream, lat: u64, lat_stride: usize, cache: u64, scales: u64, pos: u64, rows: usize, lw: usize) !void { - if (scales == 0) return latentWrite(set, stream, lat, lat_stride, cache, pos, rows, lw); +/// On pages: the stream's table (i32 page ids) and the step from page to page in bytes and latent rows; 0: in order. +pub const Table = struct { pt: u64 = 0, ps: u64 = 0, page_rows: u64 = 0 }; + +/// BF16 rows through `t`, or E4M3 rows and their scales (scales nonzero), which take no pages. +pub fn latentWriteFormat(set: glue.Set, stream: cuda.Stream, lat: u64, lat_stride: usize, cache: u64, scales: u64, t: Table, pos: u64, rows: usize, lw: usize) !void { + if (scales == 0) return latentWrite(set, stream, lat, lat_stride, cache, t, pos, rows, lw); if (lw != 512) return error.LatentShape; + if (t.pt != 0) return error.Fp8OnPages; var a: cuda.Args = .{}; inline for (.{ lat, @as(i64, @intCast(lat_stride)), cache, scales, pos }) |v| a.add(v); try cuda.launch.launch(set.glue.lat_write_fp8, .{ .grid = .{ .x = @intCast(rows) }, .block = .{ .x = 512 } }, stream, &a); } /// latent.latent_write: rows of `lat` (stride `lat_stride`) into slots pos .. pos + rows - 1 (pos an i32 on device). -pub fn latentWrite(set: glue.Set, stream: cuda.Stream, lat: u64, lat_stride: usize, cache: u64, pos: u64, rows: usize, lw: usize) !void { +pub fn latentWrite(set: glue.Set, stream: cuda.Stream, lat: u64, lat_stride: usize, cache: u64, t: Table, pos: u64, rows: usize, lw: usize) !void { var a: cuda.Args = .{}; - inline for (.{ lat, @as(i64, @intCast(lat_stride)), cache, pos, @as(i64, @intCast(lw)) }) |v| a.add(v); + inline for (.{ lat, @as(i64, @intCast(lat_stride)), cache, pos, @as(i64, @intCast(lw)), t.pt, @as(i64, @intCast(t.ps)) }) |v| a.add(v); try cuda.launch.launch(set.glue.lat_write, .{ .grid = .{ .x = @intCast(rows) }, .block = .{ .x = 128 } }, stream, &a); } @@ -72,18 +77,19 @@ pub fn latentWrite(set: glue.Set, stream: cuda.Stream, lat: u64, lat_stride: usi pub const Partials = struct { po: u64, pm: u64, pl: u64 }; /// latent.attention: causal qa over the cache in `nch` 512-key chunks into out bf16; `dummy`: the merge's count i32. -pub fn attention(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, pos: u64, p: Partials, out: u64, dummy: u64, rows: usize, heads: usize, lw: usize, nch: usize, scale: f32, hb: usize) !void { - return attentionFormat(set, stream, qa, cache, 0, pos, p, out, dummy, rows, heads, lw, nch, scale, hb); +pub fn attention(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, t: Table, pos: u64, p: Partials, out: u64, dummy: u64, rows: usize, heads: usize, lw: usize, nch: usize, scale: f32, hb: usize) !void { + return attentionFormat(set, stream, qa, cache, 0, t, pos, p, out, dummy, rows, heads, lw, nch, scale, hb); } -pub fn attentionFormat(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, scales: u64, pos: u64, p: Partials, out: u64, dummy: u64, rows: usize, heads: usize, lw: usize, nch: usize, scale: f32, hb: usize) !void { - try chunks(set, stream, qa, cache, scales, pos, 0, 0, p, rows, heads, lw, nch, 0, scale, hb, false); +pub fn attentionFormat(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, scales: u64, t: Table, pos: u64, p: Partials, out: u64, dummy: u64, rows: usize, heads: usize, lw: usize, nch: usize, scale: f32, hb: usize) !void { + try chunks(set, stream, qa, cache, scales, t, pos, 0, 0, p, rows, heads, lw, nch, 0, scale, hb, false); try merge(set, stream, p, out, dummy, rows, heads, lw, nch, false); } /// latent._dense_chunks (sparse false: keys through pos + r) and _sparse_chunks (row r's `counts` tokens of `tokens`). -fn chunks(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, scales: u64, pos: u64, tokens: u64, counts: u64, p: Partials, rows: usize, heads: usize, lw: usize, nch: usize, w: usize, scale: f32, hb: usize, sparse: bool) !void { +fn chunks(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, scales: u64, t: Table, pos: u64, tokens: u64, counts: u64, p: Partials, rows: usize, heads: usize, lw: usize, nch: usize, w: usize, scale: f32, hb: usize, sparse: bool) !void { if (lw != 512) return error.LatentShape; + if (scales != 0 and t.pt != 0) return error.Fp8OnPages; const f = switch (hb) { heads_tile => if (scales == 0) set.glue.chunks_16 else set.glue.chunks_16_fp8, heads_tile_wide => if (scales == 0) set.glue.chunks_32 else set.glue.chunks_32_fp8, @@ -94,18 +100,22 @@ fn chunks(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, scales: u64, a.add(cache); if (scales != 0) a.add(scales); inline for (.{ pos, tokens, counts, p.po, p.pm, p.pl, @as(i64, @intCast(rows)), @as(i64, @intCast(heads)), @as(i64, @intCast(w)), scale, @as(c_int, @intFromBool(sparse)) }) |v| a.add(v); + if (scales == 0) { + a.add(t.pt); + a.add(@as(i64, @intCast(t.ps))); + } const shared = if (hb == heads_tile_wide) glue.chunksSmem(heads_tile_wide) else glue.chunksSmem(heads_tile); try cuda.launch.launch(f, .{ .grid = .{ .x = @intCast(rows), .y = cdiv(heads, hb), .z = @intCast(nch) }, .block = .{ .x = 256 }, .shared = shared }, stream, &a); } /// latent.sparse_attention: rows with counts > 0 over their selected tokens (i32 [rows][w]); others left alone. -pub fn sparseAttention(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, tokens: u64, counts: u64, w: usize, p: Partials, out: u64, rows: usize, heads: usize, lw: usize, scale: f32) !void { - return sparseAttentionFormat(set, stream, qa, cache, 0, tokens, counts, w, p, out, rows, heads, lw, scale); +pub fn sparseAttention(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, t: Table, tokens: u64, counts: u64, w: usize, p: Partials, out: u64, rows: usize, heads: usize, lw: usize, scale: f32) !void { + return sparseAttentionFormat(set, stream, qa, cache, 0, t, tokens, counts, w, p, out, rows, heads, lw, scale); } -pub fn sparseAttentionFormat(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, scales: u64, tokens: u64, counts: u64, w: usize, p: Partials, out: u64, rows: usize, heads: usize, lw: usize, scale: f32) !void { +pub fn sparseAttentionFormat(set: glue.Set, stream: cuda.Stream, qa: u64, cache: u64, scales: u64, t: Table, tokens: u64, counts: u64, w: usize, p: Partials, out: u64, rows: usize, heads: usize, lw: usize, scale: f32) !void { const nch = chunksFor(w); - try chunks(set, stream, qa, cache, scales, 0, tokens, counts, p, rows, heads, lw, nch, w, scale, headBlock(rows), true); + try chunks(set, stream, qa, cache, scales, t, 0, tokens, counts, p, rows, heads, lw, nch, w, scale, headBlock(rows), true); try merge(set, stream, p, out, counts, rows, heads, lw, nch, true); } diff --git a/zig/src/families/glm/cuda_native.zig b/zig/src/families/glm/cuda_native.zig index d5c66b3b0b..9193344061 100644 --- a/zig/src/families/glm/cuda_native.zig +++ b/zig/src/families/glm/cuda_native.zig @@ -13,6 +13,8 @@ const config = @import("config.zig"); const shard = @import("cuda_shard.zig"); const dflash = @import("dflash.zig"); const dflash_load = @import("dflash_load.zig"); +const pages_mod = @import("cuda_pages.zig"); +const kv = @import("core").kv_pool; pub const model_type = "glm5_next"; pub const formats: []const []const u8 = &.{"modelopt-nvfp4"}; @@ -59,8 +61,24 @@ pub const Loaded = struct { target: *engine.Engine, // the prompt cache keeps this engine's states (snap_*) peer_room: ?Peer = null, // rank 0: the server admits streams and kept states by this when it is smaller vision: ?*images_mod.Images = null, // rank 0's tower for the server's image offer + layout: ?kv.Layout = null, // every stream's memory as one pool of KV pages (null: a window each, by TF_KV_PAGES) + engine_paged: bool = false, // with a layout, the engine's own sequence takes pages too: the plan sets a window aside }; +/// The pool behind every stream: the engine's own sequence and `streams` more on `pages` KV pages, on every rank. +pub fn attach(p: *anyopaque, streams: u32, pages: usize) !void { + const own: *Owned = @ptrCast(@alignCast(p)); + const l = own.layout orelse return error.NoPages; + try own.lanes.x.pool(streams + 1, pages, l.pageBytes()); + own.e.pages = try pages_mod.Pages.init(own.gpa, own.e, l, streams + 1, pages, true); +} + +/// Whether a request's caches fit now (the lane host's admission, on its engine thread). +pub fn fits(p: *anyopaque, prompt: u32, max_tokens: u32) bool { + const own: *Owned = @ptrCast(@alignCast(p)); + return own.lanes.fits(prompt, max_tokens); +} + /// This rank's place: TF_GLM_RANK of TF_GLM_WORLD (both, or neither for a refusal that names them). pub const Place = struct { rank: usize, world: usize }; @@ -93,6 +111,9 @@ pub fn capacity(context: usize) usize { return std.mem.alignForward(usize, context + fw.window_rows, 64); } +/// A request's last verify window: the plan's window holds that many rows past the context, as `capacity` does. +pub const window_rows: u32 = fw.window_rows; + const snapshots = @import("cuda_snapshots.zig"); const Owned = struct { @@ -101,6 +122,7 @@ const Owned = struct { e: *engine.Engine, link: follow.Link, lanes: glm_lanes.Lanes, + layout: ?kv.Layout, }; const env = @import("sys").env; @@ -133,6 +155,16 @@ pub fn roceBytes(text: ?[]const u8) !usize { return std.math.mul(usize, n, 1024) catch return error.BadRoceCap; } +/// TF_KV_PAGES: every sequence on pages, the engine's own too (on, all, any other value), or a window each (off, 0). +fn paging(text: ?[]const u8, world: usize, kv8: bool) !bool { + const v = text orelse "auto"; + // unset or auto pages from TP=3 (two GB10s' room past the weights is under the prompt cache); FP8 keeps windows + if (std.mem.eql(u8, v, "auto")) return world >= 3 and !kv8; + if (std.mem.eql(u8, v, "0") or std.mem.eql(u8, v, "off")) return false; + if (kv8) return error.Fp8OnPages; + return true; +} + pub fn open(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: []const u8, kernels: ?[]const u8, o: Options) !Loaded { _ = kernels; // our own kernels: a captured set is not used const p = try place(); @@ -144,6 +176,7 @@ pub fn open(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: [ return err; }; set.kv8 = o.kv8; + set.pages = try paging(if (std.c.getenv("TF_KV_PAGES")) |v| std.mem.span(v) else null, p.world, o.kv8); // every rank // the ranks' lines before NCCL (cuda/tp_watch.zig): a rank that fails, refuses or dies stops every rank if (p.world > 1) try cuda.tp_watch.start(io, p.rank, p.world); return openRank(gpa, io, ctx, dir, o, p, set, roce_bytes) catch |err| { @@ -220,17 +253,21 @@ fn openRank(gpa: std.mem.Allocator, io: std.Io, ctx: *const cuda.Context, dir: [ }.get; own.link.asleep = cuda.tp_watch.open(); // the other ranks begin asleep (follow): the first step wakes them own.lanes = .{ .x = .{ .e = own.e, .link = &own.link }, .whole = off("TF_GLM_PROMPT_STEPS"), .alone = off("TF_GLM_SHARED") }; + const extra: u64 = (if (o.drafter != null) dflash.ring_bytes else 0) + if (own.e.images) |v| v.streamBytes() else @as(usize, 0); + own.layout = if (set.pages) try pages_mod.layout(gpa, own.e, extra) else null; return .{ .backend = own.lanes.backend(), .facts = own.lanes.facts(), .rows = glm_lanes.max_rows, - .stream_bytes = own.e.seqBytes() + (if (o.drafter != null) dflash.ring_bytes else 0) + if (own.e.images) |v| v.streamBytes() else @as(usize, 0), // each stream past the first makes its own sequence on every rank + .stream_bytes = own.e.seqBytes() + extra, // each stream past the first makes its own sequence on every rank .free_streams = 1, // the first is the sequence the engine made .ctx = own, .deinit = close, .target = own.e, .peer_room = peer, .vision = own.e.images, + .layout = own.layout, + .engine_paged = set.pages, }; } @@ -315,6 +352,7 @@ pub fn explain(_: ?*anyopaque, err: anyerror) ?[]const u8 { error.NoRank => "GLM-5.3-Flash runs at TP over hosts: set TF_GLM_RANK and TF_GLM_WORLD on every host (WORLD 1: one GPU, with TF_GLM_LAYERS for a thin model)", error.NoCommId => "GLM-5.3-Flash's ranks meet over NCCL: with TF_GLM_WORLD 2 or more, set NCCL_COMM_ID to rank 0's address:port on every host", error.RankMismatch => "the ranks' settings differ (each rank's log names the one): start them alike", + error.Fp8OnPages => "--kv-cache fp8 keeps a window per stream: unset TF_KV_PAGES, or set it to auto or off", error.TooManyRanks => std.fmt.comptimePrint( "GLM-5.3-Flash on CUDA runs on at most {d} ranks: set TF_GLM_WORLD to {d} or fewer", .{ ranks.max_world, ranks.max_world }, @@ -331,12 +369,48 @@ test "a place inside its world of at most 16 ranks serves; a larger world is ref try std.testing.expect(explain(null, error.TooManyRanks) != null); } +test "TF_KV_PAGES: auto pages from three ranks, off and 0 give windows, all and other values page like on" { + for ([_]usize{ 1, 2 }) |w| { + try std.testing.expect(!try paging(null, w, false)); + try std.testing.expect(!try paging("auto", w, false)); + } + for ([_]usize{ 3, 16 }) |w| { + try std.testing.expect(try paging(null, w, false)); + try std.testing.expect(try paging("auto", w, false)); + } + for ([_]usize{ 1, 2, 3 }) |w| { + try std.testing.expect(try paging("on", w, false)); + try std.testing.expect(try paging("1", w, false)); + try std.testing.expect(!try paging("off", w, false)); + try std.testing.expect(!try paging("0", w, false)); + try std.testing.expect(try paging("all", w, false)); + } +} + +test "TF_KV_PAGES with --kv-cache fp8: auto and off keep windows at every rank count, on and all are refused by name" { + for ([_]usize{ 1, 2, 3, 16 }) |w| { + try std.testing.expect(!try paging(null, w, true)); + try std.testing.expect(!try paging("auto", w, true)); + try std.testing.expect(!try paging("off", w, true)); + try std.testing.expect(!try paging("0", w, true)); + for ([_][]const u8{ "on", "1", "all" }) |v| try std.testing.expectError(error.Fp8OnPages, paging(v, w, true)); + } + try std.testing.expect(explain(null, error.Fp8OnPages) != null); +} + test "capacity keeps a verify window past the context and a multiple of 64" { try std.testing.expectEqual(@as(usize, 8256), capacity(8192)); try std.testing.expectEqual(@as(usize, 330560), capacity(330529)); try std.testing.expect(capacity(330529) % 64 == 0); } +test "the plan's window, the context and a verify window, takes the pages of a stream's table at that capacity" { + const l: kv.Layout = .{ .fixed_bytes = 0, .token_bytes = 1, .page_tokens = @import("cuda_state.zig").page_rows, .snap_bytes = 0 }; + for ([_]usize{ 8192, 8191, 245_760, 524_288 - 9, 524_288 - 8, 524_288 - 7, 1_048_576 }) |context| { + try std.testing.expectEqual(pages_mod.tableLen(capacity(context)), l.pages(context + window_rows)); + } +} + // Enough words for every offered frame pair and image, even with a small text context. pub fn linkCapacity(cap: usize, media: ?images_mod.Limits) !usize { const lim = media orelse return cap; diff --git a/zig/src/families/glm/cuda_pages.zig b/zig/src/families/glm/cuda_pages.zig new file mode 100644 index 0000000000..696c471d39 --- /dev/null +++ b/zig/src/families/glm/cuda_pages.zig @@ -0,0 +1,203 @@ +//! GLM-5.3-Flash's KV pages on CUDA: the core pool's pages in one device arena, and the paged sequences that map them. +const std = @import("std"); +const cuda = @import("cuda"); +const kv = @import("core").kv_pool; +const fw = @import("cuda_forward.zig"); +const state = @import("cuda_state.zig"); +const types = @import("cuda_engine_types.zig"); +const Engine = @import("cuda_engine.zig").Engine; + +/// Table entries a sequence of `capacity` rows takes. +pub fn tableLen(capacity: usize) usize { + return (capacity + state.page_rows - 1) / state.page_rows; +} + +/// What a stream of engine `e` holds: its fixed part and table (and `extra`, its other buffers), 13,056 bytes a token. +pub fn layout(gpa: std.mem.Allocator, e: *const Engine, extra: u64) !kv.Layout { + const fixed = try state.State.pagedBytes(gpa, &e.w, e.capacity, tableLen(e.capacity)); + const page = state.pageBytes(&e.w); + return .{ .fixed_bytes = fixed + extra, .token_bytes = page / state.page_rows, .page_tokens = state.page_rows, .snap_bytes = 0 }; +} + +/// A request's need: its prompt, its reply and one last verify window. +pub fn need(l: kv.Layout, prompt: usize, max_tokens: usize) usize { + return l.need(prompt, max_tokens, fw.window_rows); +} + +/// The engine's own sequence (slot 0) when it pages, bound: its fixed part and table, no rows until its pool places it. +pub fn ownSequence(e: *Engine) !void { + if (e.seqs.items.len == 0) try e.seqs.append(e.gpa, null); + if (e.seqs.items[0] != null) return error.SlotTaken; + const q = try e.gpa.create(types.Seq); + errdefer e.gpa.destroy(q); + q.* = .{ .st = try fw.State.initPaged(e.gpa, e.d, &e.w, e.capacity, on(e, 0, state.pageBytes(&e.w))) }; + e.seqs.items[0] = q; + e.cur = q; + e.slot = 0; +} + +/// A sequence's place on the pages at `kv`: a table as long as a stream's window. +fn on(e: *const Engine, kv_at: u64, page: u64) state.Paged { + return .{ .kv = kv_at, .pages = tableLen(e.capacity), .page_bytes = page }; +} + +pub const Pages = struct { + gpa: std.mem.Allocator, + e: *Engine, + layout: kv.Layout, + pool: kv.Pool, // rank 0 takes and gives back pages; the other ranks map the ids it sends + arena: ?cuda.DeviceBuffer, + leases: []kv.Lease, // slot i's: rank 0's lease, or the ids another rank mapped + used: []bool, + leader: bool, + poison: bool, // TF_KV_POISON=1: a stream's pages start as NaN, so a row read before its write shows in the reply + + /// `streams` sequences from slot 0 on `pages` pages: the engine's own (`ownSequence`) placed, the others made now. + pub fn init(gpa: std.mem.Allocator, e: *Engine, l: kv.Layout, streams: usize, pages: usize, leader: bool) !*Pages { + const own = if (e.seqs.items.len > 0) e.seqs.items[0] else null; + if (own == null or own.?.st.table == 0) return error.OwnNotPaged; // slot 0 holds a window, or none was made + const p = try gpa.create(Pages); + errdefer gpa.destroy(p); + const page = l.pageBytes(); + p.* = .{ .gpa = gpa, .e = e, .layout = l, .pool = try kv.Pool.init(gpa, pages, page), .arena = null, .leases = &.{}, .used = &.{}, .leader = leader, .poison = poisoned() }; + errdefer p.pool.deinit(); + if (pages > 0) p.arena = try cuda.DeviceBuffer.alloc(e.d, pages * page); + errdefer if (p.arena) |*a| a.free(); + p.leases = try gpa.alloc(kv.Lease, streams); + errdefer gpa.free(p.leases); + @memset(p.leases, .{}); + p.used = try gpa.alloc(bool, streams); + errdefer gpa.free(p.used); + @memset(p.used, false); + if (e.seqs.items.len < streams) try e.seqs.appendNTimes(e.gpa, null, streams - e.seqs.items.len); + const seqs = e.seqs.items[1..@max(streams, 1)]; + var made: usize = 0; + errdefer for (seqs[0..made]) |*q| { + q.*.?.st.deinit(); + e.gpa.destroy(q.*.?); + q.* = null; + }; + const at = on(e, if (p.arena) |a| a.ptr else 0, page); + for (seqs) |*q| { + if (q.* != null) return error.SlotTaken; // attach comes before any stream + const s = try e.gpa.create(types.Seq); + errdefer e.gpa.destroy(s); + s.* = .{ .st = try fw.State.initPaged(e.gpa, e.d, &e.w, e.capacity, at) }; + q.* = s; + made += 1; + } + try e.stream.synchronize(); + own.?.st.place(&e.w, at); // last: a failure before it leaves the engine's own as it was + return p; + } + + /// Another rank: the pool rank 0's `pool` step names; a page of other bytes than rank 0's is error.RankMismatch. + pub fn follow(gpa: std.mem.Allocator, e: *Engine, streams: usize, pages: usize, page_bytes: u64) !*Pages { + const l = try layout(gpa, e, 0); + if (l.pageBytes() != page_bytes) { + if (!@import("builtin").is_test) std.log.err("rank {d}: a KV page is {d} bytes here and {d} on rank 0", .{ e.share.rank, l.pageBytes(), page_bytes }); + return error.RankMismatch; + } + return init(gpa, e, l, streams, pages, false); + } + + fn poisoned() bool { + const v = std.c.getenv("TF_KV_POISON") orelse return false; + return std.mem.eql(u8, std.mem.span(v), "1"); + } + + /// The pool and the pages' arena; the paged sequences are the engine's (it frees every slot). + pub fn deinit(p: *Pages) void { + p.e.stream.synchronize() catch {}; + for (p.leases, p.used) |*l, u| { + if (p.leader and u) p.pool.finish(l) else l.ids.deinit(p.gpa); + } + p.gpa.free(p.leases); + p.gpa.free(p.used); + if (p.arena) |*a| a.free(); + p.pool.deinit(); + p.gpa.destroy(p); + } + + /// Whether `slot` is one of the paged sequences. + pub fn paged(p: *const Pages, slot: u32) bool { + return slot < p.used.len; + } + + /// Whether a request's pages fit now and a paged sequence is free for it. + pub fn fits(p: *const Pages, prompt: usize, max_tokens: usize) bool { + for (p.used) |u| if (!u) return p.pool.fits(need(p.layout, prompt, max_tokens)); + return false; + } + + /// Rank 0: paged slot `slot` takes the pages of a prompt, its reply and a last window; LaneMemory: not yet. + pub fn open(p: *Pages, slot: u32, prompt: usize, max_tokens: usize) ![]const u32 { + if (!p.paged(slot) or p.used[slot]) return error.SlotTaken; + const n = need(p.layout, prompt, max_tokens); + if (n > tableLen(p.e.capacity)) return error.PromptTooLong; + const l = &p.leases[slot]; + try p.pool.promise(l, n); + errdefer p.pool.finish(l); + var up: Upload = .{ .p = p, .slot = slot }; + try p.pool.cover(l, n, .{ .ctx = &up, .write = Upload.write }); + try p.mapped(slot, l.ids.items); + return l.ids.items; + } + + /// Another rank: paged slot `slot` maps rank 0's page ids (sent with the stream's begin step). + pub fn map(p: *Pages, slot: u32, ids: []const u32) !void { + if (!p.paged(slot) or ids.len == 0 or ids.len > tableLen(p.e.capacity)) return error.BadStep; + for (ids) |id| if (id >= p.pool.total()) return error.BadStep; + const l = &p.leases[slot]; + l.ids.clearRetainingCapacity(); + try l.ids.appendSlice(p.gpa, ids); + var up: Upload = .{ .p = p, .slot = slot }; + try Upload.write(&up, l.ids.items, 0); + try p.mapped(slot, l.ids.items); + } + + /// The slot's sequence reaches its pages' rows (NaN first under TF_KV_POISON). + fn mapped(p: *Pages, slot: u32, ids: []const u32) !void { + const st = &p.e.seqs.items[slot].?.st; + const page = p.layout.pageBytes(); + if (p.poison) for (ids) |id| try p.e.d.check(p.e.d.api.cuMemsetD32Async(p.arena.?.ptr + id * page, 0xFFFF_FFFF, page / 4, p.e.stream.handle), "cuMemsetD32Async"); + st.pages = ids; + st.capacity = @min(ids.len * state.page_rows, p.e.capacity); + p.used[slot] = true; + } + + /// The stream on paged slot `slot` ended: its sequence reaches no rows, and rank 0's lease gives its pages back. + pub fn close(p: *Pages, slot: u32) void { + if (!p.paged(slot) or !p.used[slot]) return; + const st = &p.e.seqs.items[slot].?.st; + st.pages = &.{}; + st.capacity = 0; + p.used[slot] = false; + if (!p.leader) return p.leases[slot].ids.clearRetainingCapacity(); + p.pool.finish(&p.leases[slot]); + for (p.used) |u| if (u) return; + p.report(); + } + + /// The idle server's line: nothing promised, every page free (the copy store holds none). + fn report(p: *const Pages) void { + if (@import("builtin").is_test) return; + const c = p.pool.census(); + const ok = p.pool.idle() and c.free == c.total; + std.log.info("kv pool idle{s}: promised {d}, free {d} + store {d} == total {d}", .{ if (ok) "" else " (UNEVEN)", c.promised, c.free, c.kept, c.total }); + } + + /// A stream's table entries written to its sequence's device table (blocking: the stream waits for the copy). + const Upload = struct { + p: *Pages, + slot: u32, + + fn write(ctx: *anyopaque, ids: []const u32, from: usize) anyerror!void { + const u: *Upload = @ptrCast(@alignCast(ctx)); + const e = u.p.e; + const st = &e.seqs.items[u.slot].?.st; + const bytes = std.mem.sliceAsBytes(ids[from..]); + try e.d.check(e.d.api.cuMemcpyHtoD_v2(st.table + from * 4, bytes.ptr, bytes.len), "cuMemcpyHtoD"); + } + }; +}; diff --git a/zig/src/families/glm/cuda_pages_test.zig b/zig/src/families/glm/cuda_pages_test.zig new file mode 100644 index 0000000000..7acda05fb6 --- /dev/null +++ b/zig/src/families/glm/cuda_pages_test.zig @@ -0,0 +1,279 @@ +//! GLM's KV pages on a driver of addresses alone: tables in row order, leases given back, a follower's ids checked. +const std = @import("std"); +const cuda = @import("cuda"); +const weights = @import("cuda_weights.zig"); +const shard = @import("cuda_shard.zig"); +const pages_mod = @import("cuda_pages.zig"); +const Engine = @import("cuda_engine.zig").Engine; +const Lanes = @import("cuda_lanes.zig").Lanes; +const lanes = @import("lanes"); +const expect = std.testing.expect; +const expectEqual = std.testing.expectEqual; + +/// The driver: each allocation a fresh address, the page tables written to it kept, the NaN fills counted. +const F = struct { + var next: u64 = 1 << 30; + var live: usize = 0; + var fills: usize = 0; + var at: [8]u64 = @splat(0); // a table's address, and its words below + var words: [8][32]u32 = undefined; + fn alloc(out: *u64, n: usize) callconv(.c) cuda.abi.Result { + out.* = next; + next += std.mem.alignForward(u64, n + 1, 1 << 20); + live += 1; + return 0; + } + fn free(_: u64) callconv(.c) cuda.abi.Result { + live -= 1; + return 0; + } + fn clear(_: u64, _: u8, _: usize) callconv(.c) cuda.abi.Result { + return 0; + } + fn fill(_: u64, v: c_uint, _: usize, _: cuda.abi.Stream) callconv(.c) cuda.abi.Result { + fills += @intFromBool(v == 0xFFFF_FFFF); + return 0; + } + fn put(dst: u64, src: ?*const anyopaque, n: usize) callconv(.c) cuda.abi.Result { + const ids: [*]const u32 = @ptrCast(@alignCast(src.?)); + for (&at, &words) |*a, *w| if (a.* == 0 or (dst >= a.* and dst < a.* + 128)) { + if (a.* == 0) a.* = dst; + const from = (dst - a.*) / 4; + @memcpy(w[from..][0 .. n / 4], ids[0 .. n / 4]); + return 0; + }; + return 1; + } + fn wait(_: cuda.abi.Stream) callconv(.c) cuda.abi.Result { + return 0; + } + /// The first `n` words written to the table at `t`. + fn table(t: u64, n: usize) []const u32 { + for (at, &words) |a, *w| if (a == t) return w[0..n]; + return &.{}; + } +}; + +/// An engine with what the pages read: a driver, TP=2's shapes on a KDA, a DSA and a KDA layer, 4,096 rows. +const Fake = struct { + d: cuda.Driver, + layers: [3]weights.Layer, + e: Engine, + + fn init(f: *Fake, gpa: std.mem.Allocator) !void { + F.live = 0; + F.fills = 0; + F.at = @splat(0); + f.d = undefined; + f.d.api.cuMemAlloc_v2 = F.alloc; + f.d.api.cuMemFree_v2 = F.free; + f.d.api.cuMemsetD8_v2 = F.clear; + f.d.api.cuMemsetD32Async = F.fill; + f.d.api.cuMemcpyHtoD_v2 = F.put; + f.d.api.cuStreamSynchronize = F.wait; + for (&f.layers, [_]bool{ false, true, false }) |*l, mla| l.mixer = if (mla) .{ .mla = undefined } else .{ .kda = undefined }; + f.e = undefined; + f.e.gpa = gpa; + f.e.d = &f.d; + f.e.stream = .{ .d = &f.d, .handle = null }; + f.e.capacity = 4096; + f.e.seqs = .empty; + f.e.pages = null; + f.e.images = null; + f.e.settings = .{}; + f.e.w = undefined; + f.e.w.config = .{}; + f.e.w.share = try shard.Share.of(&f.e.w.config, 0, 2); + f.e.w.layers = &f.layers; + } + + /// The pages first (if any), then every slot's sequence, as the engine frees them. + fn deinit(f: *Fake, p: ?*pages_mod.Pages) void { + if (p) |q| q.deinit(); + for (f.e.seqs.items) |q| if (q) |s| { + s.st.deinit(); + f.e.gpa.destroy(s); + }; + f.e.seqs.deinit(f.e.gpa); + } + + /// The engine's own sequence as `Engine.init` makes it when TF_KV_PAGES pages, then the pool beside it. + fn pool(f: *Fake, l: anytype, streams: usize, pages: usize, leader: bool) !*pages_mod.Pages { + try pages_mod.ownSequence(&f.e); + return pages_mod.Pages.init(f.e.gpa, &f.e, l, streams, pages, leader); + } +}; + +test "a paged stream takes its request's pages, its table holds them in row order, and its end gives them back" { + var f: Fake = undefined; + try f.init(std.testing.allocator); + const l = try pages_mod.layout(std.testing.allocator, &f.e, 0); + try expectEqual(@as(u64, 2 * 1088), l.token_bytes); // a DSA layer and the MTP layer: 1,024 + 64 bytes each + const p = try f.pool(l, 3, 8, true); + defer f.deinit(p); + try expect(p.paged(0) and p.paged(1) and p.paged(2) and !p.paged(3)); + try expect(p.fits(600, 100)); + try std.testing.expectEqualSlices(u32, &.{ 0, 1 }, try p.open(1, 600, 100)); // 708 rows: 2 pages + const st1 = &f.e.seqs.items[1].?.st; + try std.testing.expectEqualSlices(u32, &.{ 0, 1 }, F.table(st1.table, 2)); + try expectEqual(@as(usize, 1024), st1.capacity); + try expect(!p.fits(3000, 100)); // 7 pages of the 6 left + try std.testing.expectError(error.LaneMemory, p.open(2, 3000, 100)); + try expectEqual(@as(usize, 0), f.e.seqs.items[2].?.st.capacity); + try std.testing.expectEqualSlices(u32, &.{ 2, 3, 4, 5, 6 }, try p.open(2, 2000, 100)); + p.close(1); + try expectEqual(@as(usize, 0), st1.capacity); + try std.testing.expectEqualSlices(u32, &.{ 1, 0 }, try p.open(1, 600, 100)); // the last page given back, first + try std.testing.expectEqualSlices(u32, &.{ 1, 0 }, F.table(st1.table, 2)); + p.close(1); + p.close(2); + try expect(p.pool.idle() and p.pool.census().free == 8); + try std.testing.expectError(error.PromptTooLong, p.open(1, 4096, 0)); // past the window's table +} + +test "the engine's own sequence holds no rows before its pool, then sits on the pool's pages and takes them like the others" { + var f: Fake = undefined; + try f.init(std.testing.allocator); + const l = try pages_mod.layout(std.testing.allocator, &f.e, 0); + try pages_mod.ownSequence(&f.e); + const own = f.e.seqs.items[0].?; + try expect(f.e.cur == own and f.e.slot == 0 and own.st.table != 0 and own.st.capacity == 0); // no window made + const live = F.live; + const p = try pages_mod.Pages.init(std.testing.allocator, &f.e, l, 2, 8, true); + defer f.deinit(p); + try expect(f.e.cur == own and f.e.seqs.items[0].? == own); // the same sequence: it is placed, not made again + try expectEqual(live + 2, F.live); // the pages' arena and slot 1's sequence + try expectEqual(p.arena.?.ptr, own.st.caches[0].lc); // its DSA layer's part of page 0 starts the arena + try std.testing.expectEqualSlices(u32, &.{ 0, 1 }, try p.open(0, 600, 100)); + try std.testing.expectEqualSlices(u32, &.{ 0, 1 }, F.table(own.st.table, 2)); + try expectEqual(@as(usize, 1024), own.st.capacity); + p.close(0); + try expect(own.st.capacity == 0 and p.pool.idle() and p.pool.census().free == 8); +} + +test "a pool is refused while the engine's own sequence holds a window or was never made, and nothing is made for it" { + var f: Fake = undefined; + try f.init(std.testing.allocator); + defer f.deinit(null); + const l = try pages_mod.layout(std.testing.allocator, &f.e, 0); + try std.testing.expectError(error.OwnNotPaged, pages_mod.Pages.init(std.testing.allocator, &f.e, l, 2, 8, true)); + try f.e.bind(0); // a window, as without TF_KV_PAGES + try std.testing.expectError(error.OwnNotPaged, pages_mod.Pages.init(std.testing.allocator, &f.e, l, 2, 8, true)); + try expectEqual(@as(usize, 1), F.live); // the window alone + try expectEqual(@as(usize, 1), f.e.seqs.items.len); +} + +test "a follower maps rank 0's ids, and refuses ids past its pool or a slot that is not paged" { + var f: Fake = undefined; + try f.init(std.testing.allocator); + const l = try pages_mod.layout(std.testing.allocator, &f.e, 0); + const p = try f.pool(l, 3, 8, false); + defer f.deinit(p); + try p.map(2, &.{ 5, 3, 7 }); + try std.testing.expectEqualSlices(u32, &.{ 5, 3, 7 }, F.table(f.e.seqs.items[2].?.st.table, 3)); + try expectEqual(@as(usize, 1536), f.e.seqs.items[2].?.st.capacity); + try std.testing.expectError(error.BadStep, p.map(1, &.{8})); + try std.testing.expectError(error.BadStep, p.map(3, &.{1})); + p.close(2); + try expectEqual(@as(usize, 0), f.e.seqs.items[2].?.st.capacity); + try expect(p.pool.idle()); // a follower's pool lends nothing +} + +test "a follower refuses rank 0's pool when its KV page has other bytes, and makes no pool" { + var f: Fake = undefined; + try f.init(std.testing.allocator); + const l = try pages_mod.layout(std.testing.allocator, &f.e, 0); + try pages_mod.ownSequence(&f.e); + const more = l.pageBytes() + 1088 * 512; // rank 0 with one more paged layer + try std.testing.expectError(error.RankMismatch, pages_mod.Pages.follow(std.testing.allocator, &f.e, 2, 8, more)); + try expectEqual(@as(usize, 1), F.live); // the engine's own sequence alone: no arena, no paged sequence + const p = try pages_mod.Pages.follow(std.testing.allocator, &f.e, 2, 8, l.pageBytes()); + defer f.deinit(p); + try expect(!p.leader and p.pool.total() == 8); +} + +test "TF_KV_PAGES: a stream past the paged sequences waits for one, and its slot is never made a whole window" { + var f: Fake = undefined; + try f.init(std.testing.allocator); + const l = try pages_mod.layout(std.testing.allocator, &f.e, 0); + const p = try f.pool(l, 2, 8, true); + defer f.deinit(p); + f.e.pages = p; + var ln: Lanes = .{ .x = .{ .e = &f.e, .link = undefined } }; + defer ln.lanes.deinit(f.e.gpa); + var streams: [3]lanes.Stream = undefined; // as many at once as lanes: the engine's own sequence and one more + try expectEqual(@as(usize, 2), (try ln.seat(&streams[0], 600, 100)).len); + try expectEqual(@as(u32, 0), ln.lanes.get(&streams[0]).?.slot); // the engine's own takes pages too + try expectEqual(@as(usize, 2), (try ln.seat(&streams[1], 600, 100)).len); + const live = F.live; + try std.testing.expectError(error.LaneMemory, ln.seat(&streams[2], 600, 100)); // slot 2: past the paged ones + try expectEqual(live, F.live); // nothing allocated for it + try expectEqual(@as(usize, 2), f.e.seqs.items.len); + try expectEqual(@as(usize, 2), ln.lanes.count()); + _ = ln.lanes.remove(&streams[0]); // a stream ends: the one that waited takes its paged sequence + p.close(0); + _ = try ln.seat(&streams[2], 600, 100); + try expectEqual(@as(u32, 0), ln.lanes.get(&streams[2]).?.slot); + try expectEqual(@as(u32, 0), f.e.slot); +} + +test "TF_KV_PAGES: the lane host admits a request by the pool alone, the engine's own sequence free or not" { + var f: Fake = undefined; + try f.init(std.testing.allocator); + const l = try pages_mod.layout(std.testing.allocator, &f.e, 0); + const p = try f.pool(l, 2, 8, true); + defer f.deinit(p); + f.e.pages = p; + var ln: Lanes = .{ .x = .{ .e = &f.e, .link = undefined } }; + defer ln.lanes.deinit(f.e.gpa); + try expect(ln.fits(3000, 100)); // 7 of the 8 pages + var streams: [2]lanes.Stream = undefined; + _ = try ln.seat(&streams[0], 600, 100); // slot 0: 2 pages + _ = try ln.seat(&streams[1], 2000, 100); // slot 1: 5 pages + _ = ln.lanes.remove(&streams[0]); + p.close(0); + try expect(!ln.fits(3000, 100)); // the engine's own is free, but only 3 pages are + try expect(ln.fits(1000, 100)); +} + +test "TF_KV_POISON: every page a stream takes is filled with NaN before its rows are written" { + var f: Fake = undefined; + try f.init(std.testing.allocator); + const l = try pages_mod.layout(std.testing.allocator, &f.e, 0); + const p = try f.pool(l, 1, 8, true); + defer f.deinit(p); + p.poison = true; + _ = try p.open(0, 1200, 100); + try expectEqual(@as(usize, 3), F.fills); +} + +fn made(gpa: std.mem.Allocator) !void { + var f: Fake = undefined; + try f.init(gpa); + defer f.e.seqs.deinit(gpa); + const l = try pages_mod.layout(gpa, &f.e, 0); + try pages_mod.ownSequence(&f.e); + const own = f.e.seqs.items[0].?; + defer { // the engine frees its own sequence, its pool made or not + own.st.deinit(); + gpa.destroy(own); + } + const p = try pages_mod.Pages.init(gpa, &f.e, l, 3, 8, true); + p.deinit(); + for (f.e.seqs.items[1..]) |q| if (q) |s| { + s.st.deinit(); + gpa.destroy(s); + }; +} + +test "every host allocation failure while making the pages leaves no device buffer behind" { + try std.testing.checkAllAllocationFailures(std.testing.allocator, struct { + fn run(gpa: std.mem.Allocator) !void { + made(gpa) catch |err| { + try expectEqual(@as(usize, 0), F.live); + return err; + }; + try expectEqual(@as(usize, 0), F.live); + } + }.run, .{}); +} diff --git a/zig/src/families/glm/cuda_ranks.zig b/zig/src/families/glm/cuda_ranks.zig index 04bd50ad5e..7300deb65a 100644 --- a/zig/src/families/glm/cuda_ranks.zig +++ b/zig/src/families/glm/cuda_ranks.zig @@ -8,6 +8,7 @@ const Sha256 = std.crypto.hash.sha2.Sha256; /// The forward's knobs a rank reads from its environment (the Python family's names and defaults). pub const Settings = struct { kv8: bool = false, // --kv-cache fp8, supplied by the native options rather than an environment override + pages: bool = false, // TF_KV_PAGES as `open` resolves it (with the world and --kv-cache): every sequence on pages prompt_fixed_k: bool = false, // TF_GLM_PROMPT_K: one (default) or shape (experimental fixed K slices) tiled: bool = true, // TF_GLM_MM: 1 (the bf16 linears on glm_mm, tiled at load) or 0 split: bool = true, // TF_GLM_PREFILL_REDUCE: split (each rank glues its own rows), ring or gather @@ -99,10 +100,11 @@ pub fn environment(name: [:0]const u8) ?[]const u8 { /// What every rank must agree on, as bytes the communicator carries. pub const Fingerprint = extern struct { - layout: u32 = 11, // this struct's version + layout: u32 = 12, // this struct's version world: u32, rank: u32, kv8: u32 = 0, + pages: u32 = 0, // TF_KV_PAGES: the engine's own sequence is made on pages at init, so every rank pages alike prompt_fixed_k: u32, tiled: u32, split: u32, // TF_GLM_PREFILL_REDUCE: 0 gather, 1 split, 2 ring @@ -128,6 +130,7 @@ pub const Fingerprint = extern struct { rank, world, kv8, + pages, prompt_fixed_k, tiled, split, @@ -161,6 +164,7 @@ pub const Fingerprint = extern struct { ) Fingerprint { const f: Fingerprint = .{ .kv8 = @intFromBool(s.kv8), + .pages = @intFromBool(s.pages), .world = @intCast(world), .rank = @intCast(rank), .prompt_fixed_k = @intFromBool(s.prompt_fixed_k), @@ -191,6 +195,7 @@ pub fn mismatch(all: []const Fingerprint) ?Mismatch { if (f.rank != r) return .{ .field = .rank, .rank = r }; const fields = .{ "kv8", + "pages", "prompt_fixed_k", "world", "tiled", @@ -299,6 +304,7 @@ fn explain(f: Fingerprint.Field) []const u8 { .rank => "a rank number given twice or out of order", .world => "the ranks were started with different world sizes", .kv8 => "--kv-cache fp8", + .pages => "TF_KV_PAGES (unset or auto pages from three ranks)", .prompt_fixed_k => "TF_GLM_PROMPT_K", .tiled => "TF_GLM_MM", .split => "TF_GLM_PREFILL_REDUCE", @@ -378,6 +384,8 @@ test "fingerprints: agreeing ranks pass; the first differing field and rank are const heat_on: Settings = .{ .heat = .{ .high_c = 92, .low_c = 88 } }; all[1] = Fingerprint.of(2, 1, heat_on, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); try std.testing.expectEqual(Fingerprint.Field.heat_high, mismatch(&all).?.field); + all[1] = Fingerprint.of(2, 1, .{ .pages = true }, 8192, true, h0, h0, hashBytes("/hub/snapshots/423acf37")); + try std.testing.expectEqual(Mismatch{ .field = .pages, .rank = 1 }, mismatch(&all).?); // TF_KV_PAGES on one rank all[1] = Fingerprint.of(2, 1, .{}, 8192, true, h0, h0, hashBytes("/hub/snapshots/other")); try std.testing.expectEqual(Fingerprint.Field.checkpoint, mismatch(&all).?.field); all[1] = Fingerprint.of(2, 1, .{}, 8192, false, h0, h0, hashBytes("/hub/snapshots/423acf37")); // --no-drafts on rank 1 only @@ -501,7 +509,7 @@ test "every media limit is part of the ranks' agreement" { test "every scheduling and identity field that differs stops the ranks" { const zero = Fingerprint.of(2, 0, .{}, 8192, true, @splat(0), @splat(0), hashBytes("/same")); const one = Fingerprint.of(2, 1, .{}, 8192, true, @splat(0), @splat(0), hashBytes("/same")); - inline for (.{ "layout", "world", "rank", "split", "kda_wide", "pieces", "layers", "mtp", "vision", "video_tokens", "max_images", "max_videos", "max_frames", "capacity", "heat_high", "heat_low" }) |name| { + inline for (.{ "layout", "world", "rank", "pages", "split", "kda_wide", "pieces", "layers", "mtp", "vision", "video_tokens", "max_images", "max_videos", "max_frames", "capacity", "heat_high", "heat_low" }) |name| { var all = [_]Fingerprint{ zero, one }; @field(all[1], name) += 1; try std.testing.expectEqual(@field(Fingerprint.Field, name), mismatch(&all).?.field); diff --git a/zig/src/families/glm/cuda_snapshot_layout.zig b/zig/src/families/glm/cuda_snapshot_layout.zig index f267c0cb88..b01d943e1f 100644 --- a/zig/src/families/glm/cuda_snapshot_layout.zig +++ b/zig/src/families/glm/cuda_snapshot_layout.zig @@ -1,21 +1,37 @@ //! A compact prefix of a rank's committed GLM state; live sequence addresses and kernel layouts stay fixed. const std = @import("std"); const State = @import("cuda_state.zig").State; +const page_rows = @import("cuda_state.zig").page_rows; const Config = @import("config.zig").Config; const cache = @import("cuda_dsa_block.zig"); const sparse = @import("cuda_sparse.zig"); -pub const Part = struct { live: u64, len: usize }; +/// `len` bytes from `live`, or on a paged sequence's pages `span` bytes a page, the pages `page` bytes apart. +pub const Part = struct { live: u64, len: usize, span: usize = 0, page: u64 = 0 }; pub const Plan = struct { parts: [5 * (@import("config.zig").max_layers + 1) + 2]Part = undefined, count: usize = 0, len: usize = 0, + ids: []const u32 = &.{}, // a paged sequence's pages in row order + sizing: bool = false, // `size`: the rows counted, not read, so no page need be mapped pub fn init(st: *const State, c: *const Config, at: usize, head: ?usize) !Plan { + return make(st, c, at, head, false); + } + + /// A prefix's bytes at `at`, rows to `capacity` counted held or not (a paged sequence holds its request's alone). + pub fn size(st: *const State, c: *const Config, capacity: usize, at: usize, head: ?usize) !usize { + var whole = st.*; + whole.capacity = capacity; + return (try make(&whole, c, at, head, true)).len; + } + + fn make(st: *const State, c: *const Config, at: usize, head: ?usize, sizing: bool) !Plan { if (at > st.capacity or st.caches.len > @import("config.zig").max_layers) return error.SnapshotShape; if (st.rings.len != 2 * (st.caches.len + 1)) return error.SnapshotShape; - var p: Plan = .{}; - const first = if (st.caches.len > 0) st.caches[0].lc else st.mtp_caches.lc; + var p: Plan = .{ .ids = st.pages, .sizing = sizing }; + // a paged sequence's caches are its pages: its fields up to them end at its first ring + const first = if (st.table != 0) st.rings[0] else if (st.caches.len > 0) st.caches[0].lc else st.mtp_caches.lc; if (first < st.arena.buf.ptr) return error.SnapshotShape; try p.add(st, st.arena.buf.ptr, @intCast(first - st.arena.buf.ptr)); for (st.caches, 0..) |x, j| try p.rows(st, c, x, j, at); @@ -25,7 +41,9 @@ pub const Plan = struct { } if (st.mtp_pos_dev < st.arena.buf.ptr or st.mtp_pos_dev - st.arena.buf.ptr > st.arena.buf.len) return error.SnapshotShape; - try p.add(st, st.mtp_pos_dev, st.arena.buf.len - @as(usize, @intCast(st.mtp_pos_dev - st.arena.buf.ptr))); + const end = if (st.table != 0) st.table else st.arena.buf.ptr + st.arena.buf.len; // its table stays its own + if (end < st.mtp_pos_dev) return error.SnapshotShape; + try p.add(st, st.mtp_pos_dev, @intCast(end - st.mtp_pos_dev)); return p; } @@ -33,11 +51,28 @@ pub const Plan = struct { const latent = try std.math.mul(usize, at, @as(usize, c.kv_lora) * @as(usize, if (x.ls == 0) 2 else 1)); const ring = try std.math.mul(usize, @min(at, st.ring_rows), st.ring_row); const pooled = try std.math.mul(usize, @min(at / sparse.pool + 2, x.pk_rows), @as(usize, c.i_dim) * 2); - try p.add(st, x.lc, latent); + try p.addCache(st, x, x.lc, latent, page_rows * @as(usize, c.kv_lora) * 2); if (x.ls != 0) try p.add(st, x.ls, try std.math.mul(usize, at, @sizeOf(f32))); try p.add(st, st.rings[2 * j], ring); try p.add(st, st.rings[2 * j + 1], ring); - try p.add(st, x.pk, pooled); + try p.addCache(st, x, x.pk, pooled, page_rows / sparse.pool * @as(usize, c.i_dim) * 2); + } + + /// A cache's rows: in the arena, or on the sequence's pages, `span` bytes a page. + fn addCache(p: *Plan, st: *const State, x: cache.Caches, live: u64, len: usize, span: usize) !void { + if (x.pt == 0) return p.add(st, live, len); + if (len == 0) return; + if ((!p.sizing and (len - 1) / span >= p.ids.len) or p.count == p.parts.len) return error.SnapshotShape; + p.len = try std.math.add(usize, p.len, len); + p.parts[p.count] = .{ .live = live, .len = len, .span = span, .page = x.ps }; + p.count += 1; + } + + /// Where a part's bytes from `done` on sit, as many as lie in a row there. + pub fn piece(p: *const Plan, part: Part, done: usize) Part { + if (part.span == 0) return .{ .live = part.live + done, .len = part.len - done }; + const off = done % part.span; + return .{ .live = part.live + @as(u64, p.ids[done / part.span]) * part.page + off, .len = @min(part.span - off, part.len - done) }; } fn add(p: *Plan, st: *const State, live: u64, len: usize) !void { diff --git a/zig/src/families/glm/cuda_snapshot_pages.zig b/zig/src/families/glm/cuda_snapshot_pages.zig index 1de3951036..43753bb1a9 100644 --- a/zig/src/families/glm/cuda_snapshot_pages.zig +++ b/zig/src/families/glm/cuda_snapshot_pages.zig @@ -57,9 +57,9 @@ pub const Pool = struct { while (done < part.len) { const at = off + done; const target = arena.ptr + @as(u64, lease.ids.items[at / page]) * page + at % page; - const n = @min(part.len - done, page - at % page); - const live = part.live + done; - try arena.d.check(arena.d.api.cuMemcpyDtoDAsync_v2(if (into) target else live, if (into) live else target, n, stream.handle), "cuMemcpyDtoDAsync"); + const live = plan.piece(part, done); + const n = @min(live.len, page - at % page); + try arena.d.check(arena.d.api.cuMemcpyDtoDAsync_v2(if (into) target else live.live, if (into) live.live else target, n, stream.handle), "cuMemcpyDtoDAsync"); done += n; } off += part.len; diff --git a/zig/src/families/glm/cuda_snapshots.zig b/zig/src/families/glm/cuda_snapshots.zig index 56ea95f05b..396c1c412c 100644 --- a/zig/src/families/glm/cuda_snapshots.zig +++ b/zig/src/families/glm/cuda_snapshots.zig @@ -26,8 +26,9 @@ pub fn agree(e: anytype, budget: u64) !void { pub fn bytes(e: anytype, at: usize) u64 { const head: ?usize = if (e.mtp) @min(e.capacity, std.math.add(usize, at, @import("cuda_state.zig").window_rows) catch return std.math.maxInt(u64)) else null; - const p = layout.Plan.init(&e.cur.st, &e.c, at, head) catch return std.math.maxInt(u64); - return pages.rounded(p.len) catch std.math.maxInt(u64); + // a paged sequence holds its request's rows: its prefix is counted to the capacity, in a window's bytes + const n = layout.Plan.size(&e.cur.st, &e.c, e.capacity, at, head) catch return std.math.maxInt(u64); + return pages.rounded(n) catch std.math.maxInt(u64); } pub fn keep(e: anytype) !u64 { diff --git a/zig/src/families/glm/cuda_sparse.zig b/zig/src/families/glm/cuda_sparse.zig index 28af2fe24d..c09bf39eab 100644 --- a/zig/src/families/glm/cuda_sparse.zig +++ b/zig/src/families/glm/cuda_sparse.zig @@ -3,6 +3,7 @@ const std = @import("std"); const cuda = @import("cuda"); const glue = @import("cuda_glue.zig"); +const latent = @import("cuda_latent.zig"); /// sparse.POOL, TOPK_POOLS, SELECT_LIST, SCORE_LOOP_FROM, SCORE_LOOP; geometry.MLA_SELECT_ROWS. pub const pool = 4; @@ -19,12 +20,12 @@ fn cdiv(a: usize, b: usize) usize { } /// sparse.index_update: the window's LayerNormed index keys and gates into ik and ig at pos, completed pools into pk. -pub fn indexUpdate(set: glue.Set, stream: cuda.Stream, k_raw: u64, k_stride: usize, gate: u64, ln_w: u64, ln_b: u64, ape: u64, ik: u64, ig: u64, pk: u64, pos: u64, rows: usize) !void { +pub fn indexUpdate(set: glue.Set, stream: cuda.Stream, k_raw: u64, k_stride: usize, gate: u64, ln_w: u64, ln_b: u64, ape: u64, ik: u64, ig: u64, pk: u64, t: latent.Table, pos: u64, rows: usize) !void { var a: cuda.Args = .{}; inline for (.{ k_raw, @as(i64, @intCast(k_stride)), gate, ln_w, ln_b, ik, ig, pos, @as(f32, 1e-6) }) |v| a.add(v); try cuda.launch.launch(set.glue.index_write, .{ .grid = .{ .x = @intCast(rows) }, .block = .{ .x = 32 } }, stream, &a); var b: cuda.Args = .{}; - inline for (.{ ik, ig, ape, pk, pos, @as(i64, @intCast(rows)) }) |v| b.add(v); + inline for (.{ ik, ig, ape, pk, pos, @as(i64, @intCast(rows)), t.pt, @as(i64, @intCast(t.ps)) }) |v| b.add(v); try cuda.launch.launch(set.glue.pool_keys, .{ .grid = .{ .x = @intCast(rows / 4 + 2) }, .block = .{ .x = 32 } }, stream, &b); } @@ -48,12 +49,12 @@ pub fn headScale(heads: usize) f32 { /// The scratch of one selectTokens: scores fp32 [min(rows, 512)][np], and with np past select_list the list rows. pub const Scratch = struct { scores: u64, ls: u64, li: u64, pools: u64, at: u64 }; -/// sparse.select_tokens: each row's tokens (i32 [rows][width], ascending, -1 padded), count 0 under the dense limit. -pub fn selectTokens(set: glue.Set, ops: cuda.Module, stream: cuda.Stream, qi: u64, wts: u64, w_stride: usize, pk: u64, pos: u64, host_pos: usize, rows: usize, heads: usize, d: usize, np_max_cap: usize, s: Scratch, tokens: u64, counts: u64) !void { - return selectTokensProfile(set, ops, stream, qi, wts, w_stride, pk, pos, host_pos, rows, heads, d, np_max_cap, s, tokens, counts, null); +/// sparse.select_tokens: rows' tokens (i32 [rows][width], ascending, -1 padded; on pages their rows); count 0 if dense. +pub fn selectTokens(set: glue.Set, ops: cuda.Module, stream: cuda.Stream, qi: u64, wts: u64, w_stride: usize, pk: u64, t: latent.Table, pos: u64, host_pos: usize, rows: usize, heads: usize, d: usize, np_max_cap: usize, s: Scratch, tokens: u64, counts: u64) !void { + return selectTokensProfile(set, ops, stream, qi, wts, w_stride, pk, t, pos, host_pos, rows, heads, d, np_max_cap, s, tokens, counts, null); } -pub fn selectTokensProfile(set: glue.Set, ops: cuda.Module, stream: cuda.Stream, qi: u64, wts: u64, w_stride: usize, pk: u64, pos: u64, host_pos: usize, rows: usize, heads: usize, d: usize, np_max_cap: usize, s: Scratch, tokens: u64, counts: u64, profile: ?*@import("cuda_prof.zig").Prof) !void { +pub fn selectTokensProfile(set: glue.Set, ops: cuda.Module, stream: cuda.Stream, qi: u64, wts: u64, w_stride: usize, pk: u64, t: latent.Table, pos: u64, host_pos: usize, rows: usize, heads: usize, d: usize, np_max_cap: usize, s: Scratch, tokens: u64, counts: u64, profile: ?*@import("cuda_prof.zig").Prof) !void { const prof = @import("cuda_prof.zig"); const np_max = poolBucket(host_pos, rows, np_max_cap); const b = @min(rows, select_rows); @@ -73,7 +74,7 @@ pub fn selectTokensProfile(set: glue.Set, ops: cuda.Module, stream: cuda.Stream, } const nb = cdiv(scoreColumns(host_pos + a, n, np_max), 64); var sa: cuda.Args = .{}; - inline for (.{ qi + a * heads * d * 2, wts + a * w_stride * 2, @as(i64, @intCast(w_stride)), pk, s.scores, at, @as(i64, @intCast(np_max)), @as(i64, @intCast(nb)), @as(f32, @floatCast(std.math.pow(f64, @floatFromInt(d), -0.5))), headScale(heads), @as(i64, @intCast(heads)), @as(c_int, @intCast(loop)) }) |v| sa.add(v); + inline for (.{ qi + a * heads * d * 2, wts + a * w_stride * 2, @as(i64, @intCast(w_stride)), pk, s.scores, at, @as(i64, @intCast(np_max)), @as(i64, @intCast(nb)), @as(f32, @floatCast(std.math.pow(f64, @floatFromInt(d), -0.5))), headScale(heads), @as(i64, @intCast(heads)), @as(c_int, @intCast(loop)), t.pt, @as(i64, @intCast(t.ps)) }) |v| sa.add(v); if (d != 128 or heads > 32) return error.IndexShape; var stamp = prof.span(profile, stream, .index_score); try cuda.launch.launch(set.glue.scores, .{ .grid = .{ .x = @intCast(n), .y = @intCast(cdiv(nb, loop)) }, .block = .{ .x = 128 } }, stream, &sa); @@ -93,6 +94,8 @@ pub fn selectTokensProfile(set: glue.Set, ops: cuda.Module, stream: cuda.Stream, args.add(@as(c_int, topk_pools)); args.add(tokens); args.add(counts); + args.add(t.pt); + args.add(@as(i64, @intCast(t.page_rows))); try cuda.launch.launch(assemble, .{ .grid = .{ .x = @intCast(rows) }, .block = .{ .x = 256 } }, stream, &args); } diff --git a/zig/src/families/glm/cuda_state.zig b/zig/src/families/glm/cuda_state.zig index f039b6bb45..2abb4aa094 100644 --- a/zig/src/families/glm/cuda_state.zig +++ b/zig/src/families/glm/cuda_state.zig @@ -50,6 +50,59 @@ fn arena(d: *const cuda.Driver, sizes: []const usize, out: []u64) !Arena { return a; } +/// Rows a KV page holds: one dense latent chunk (latent.chunk), and the keys of its 128 pools. +pub const page_rows = latent.chunk; + +/// Where a paged sequence's caches are: the pool's pages at `kv`, `page_bytes` apart, through a table of `pages`. +pub const Paged = struct { kv: u64, pages: usize, page_bytes: u64 }; + +/// The DSA (MLA) layers among the loaded ones; the MTP layer's caches come after theirs. +pub fn mlaLayers(w: *const weights.Weights) usize { + var nd: usize = 0; + for (w.layers) |l| nd += @intFromBool(l.mixer == .mla); + return nd; +} + +/// One cache's part of a page: its latent rows, then their pools' keys. +fn pageCacheBytes(c: anytype) usize { + return page_rows * c.kv_lora * 2 + page_rows / sparse.pool * c.i_dim * 2; +} + +/// A page's bytes: every DSA layer's part, then the MTP layer's (13,056 a row on GLM-5.3-Flash). +pub fn pageBytes(w: *const weights.Weights) u64 { + return (mlaLayers(w) + 1) * pageCacheBytes(&w.config); +} + +/// Cache `j`'s rows on pages: its part of page 0 at `p.kv`, the others through the table. +fn onPages(x: *dsa_block.Caches, c: anytype, p: Paged, table: u64, j: usize) void { + x.lc = p.kv + j * pageCacheBytes(c); + x.pk = x.lc + page_rows * c.kv_lora * 2; + x.pt = table; + x.ps = p.page_bytes; +} + +/// A State's fields in order; on pages (a table of `table` entries) no latent or pooled rows, and the table last. +fn stateSizes(gpa: std.mem.Allocator, w: *const weights.Weights, capacity: usize, kv8: bool, table: ?usize) ![]usize { + const c = &w.config; + const s = &w.share; + const nd = mlaLayers(w); + const nk = w.layers.len - nd; + const n = s.kda.len() * c.kda_dim; + const kw = 3 * n + 2 * c.kda_dim + s.kda.len(); + const pk_rows = capacity / sparse.pool + 2; + const ring_rows = @min(capacity, index_ring); + const rows: usize = @intFromBool(table == null); // on pages a cache's rows are the pool's + var sizes: std.ArrayList(usize) = .empty; + errdefer sizes.deinit(gpa); + try sizes.appendSlice(gpa, &.{ 4, nk * 3 * 3 * n * 2, 2 * nk * s.kda.len() * kda.dk * kda.dv * 4, nk * window_rows * kw * 2 }); + try sizes.appendSlice(gpa, &.{ nk * window_rows * n * 4, nk * window_rows * n * 2, nk * window_rows * n * 4, nk * window_rows * s.kda.len() * 4 }); + const lc_bytes = try latent.cacheBytes(capacity, c.kv_lora, kv8); + for (0..nd + 1) |_| try sizes.appendSlice(gpa, &.{ rows * lc_bytes, ring_rows * c.i_dim * 2, ring_rows * c.i_dim * 2, rows * pk_rows * c.i_dim * 2 }); + try sizes.appendSlice(gpa, &.{ 4, c.hidden * 2 }); // MTP position and sequence-owned first-draft row + if (table) |t| try sizes.append(gpa, t * 4); + return sizes.toOwnedSlice(gpa); +} + /// A rank's B16 matmul shapes ([n, k]) whose split-K partials the buffers hold. fn b16Shapes(w: *const weights.Weights) [12][2]usize { const c = &w.config; @@ -279,34 +332,44 @@ pub const State = struct { base: usize = 0, // the first position the main rings hold (their pointers are ring - base rows) mtp_base: usize = 0, prompt_hidden: u64, // first-draft row owned and charged by this sequence arena + table: u64 = 0, // on pages: its page table (i32, `pages` entries), else 0 + pages: []const u32 = &.{}, // on pages: the page ids its table holds, in row order (none while no stream holds it) pub fn init(gpa: std.mem.Allocator, d: *const cuda.Driver, w: *const weights.Weights, capacity: usize) !State { return initFormat(gpa, d, w, capacity, false); } pub fn initFormat(gpa: std.mem.Allocator, d: *const cuda.Driver, w: *const weights.Weights, capacity: usize, kv8: bool) !State { - const lc_bytes = try latent.cacheBytes(capacity, w.config.kv_lora, kv8); + return make(gpa, d, w, capacity, kv8, null); + } + + /// A sequence whose latent rows and pooled keys are pages of `p`; it holds no rows until a stream maps some. + pub fn initPaged(gpa: std.mem.Allocator, d: *const cuda.Driver, w: *const weights.Weights, capacity: usize, p: Paged) !State { + return make(gpa, d, w, capacity, false, p); + } + + /// Bytes a sequence on pages allocates at any length (its fixed part, as `arena` rounds it), a table of `pages`. + pub fn pagedBytes(gpa: std.mem.Allocator, w: *const weights.Weights, capacity: usize, pages: usize) !usize { + const sizes = try stateSizes(gpa, w, capacity, false, pages); + defer gpa.free(sizes); + var total: usize = 0; + for (sizes) |n| total += std.mem.alignForward(usize, n, 256); + return total; + } + + fn make(gpa: std.mem.Allocator, d: *const cuda.Driver, w: *const weights.Weights, capacity: usize, kv8: bool, paged: ?Paged) !State { const c = &w.config; const s = &w.share; - var nk: usize = 0; - var nd: usize = 0; - for (w.layers) |l| switch (l.mixer) { - .kda => nk += 1, - .mla => nd += 1, - }; + const nk = w.layers.len - mlaLayers(w); + const nd = mlaLayers(w); const n = s.kda.len() * c.kda_dim; const kw = 3 * n + 2 * c.kda_dim + s.kda.len(); const pk_rows = capacity / sparse.pool + 2; - var sizes: std.ArrayList(usize) = .empty; - defer sizes.deinit(gpa); - try sizes.appendSlice(gpa, &.{ 4, nk * 3 * 3 * n * 2, 2 * nk * s.kda.len() * kda.dk * kda.dv * 4, nk * window_rows * kw * 2 }); - try sizes.appendSlice(gpa, &.{ nk * window_rows * n * 4, nk * window_rows * n * 2, nk * window_rows * n * 4, nk * window_rows * s.kda.len() * 4 }); - const ring_rows = @min(capacity, index_ring); - for (0..nd + 1) |_| try sizes.appendSlice(gpa, &.{ lc_bytes, ring_rows * c.i_dim * 2, ring_rows * c.i_dim * 2, pk_rows * c.i_dim * 2 }); - try sizes.appendSlice(gpa, &.{ 4, c.hidden * 2 }); // MTP position and sequence-owned first-draft row - const p = try gpa.alloc(u64, sizes.items.len); + const sizes = try stateSizes(gpa, w, capacity, kv8, if (paged) |q| q.pages else null); + defer gpa.free(sizes); + const p = try gpa.alloc(u64, sizes.len); defer gpa.free(p); - var a = try arena(d, sizes.items, p); + var a = try arena(d, sizes, p); errdefer a.buf.free(); const cur = try gpa.alloc(u1, nk); errdefer gpa.free(cur); @@ -316,7 +379,20 @@ pub const State = struct { for (caches, 0..) |*x, j| x.* = .{ .lc = p[8 + 4 * j], .ls = if (kv8) p[8 + 4 * j] + capacity * c.kv_lora else 0, .ik = p[9 + 4 * j], .ig = p[10 + 4 * j], .pk = p[11 + 4 * j], .pk_rows = pk_rows }; const rings = try gpa.alloc(u64, 2 * (nd + 1)); for (0..nd + 1) |j| rings[2 * j ..][0..2].* = .{ p[9 + 4 * j], p[10 + 4 * j] }; - return .{ .ring_rows = ring_rows, .ring_row = c.i_dim * 2, .rings = rings, .arena = a, .gpa = gpa, .capacity = capacity, .pos_dev = p[0], .conv = p[1], .rec = p[2], .proj = p[3], .save_k = p[4], .save_v = p[5], .save_g = p[6], .save_b = p[7], .cur = cur, .heads = s.kda.len(), .mtp_caches = .{ .lc = p[8 + 4 * nd], .ls = if (kv8) p[8 + 4 * nd] + capacity * c.kv_lora else 0, .ik = p[9 + 4 * nd], .ig = p[10 + 4 * nd], .pk = p[11 + 4 * nd], .pk_rows = pk_rows }, .mtp_pos_dev = p[12 + 4 * nd], .prompt_hidden = p[13 + 4 * nd], .kda_layers = nk, .caches = caches, .conv_c = 3 * n, .rec_layer = s.kda.len() * kda.dk * kda.dv * 4, .proj_layer = window_rows * kw, .proj_row = kw }; + const ring_rows = @min(capacity, index_ring); + var st: State = .{ .ring_rows = ring_rows, .ring_row = c.i_dim * 2, .rings = rings, .arena = a, .gpa = gpa, .capacity = capacity, .pos_dev = p[0], .conv = p[1], .rec = p[2], .proj = p[3], .save_k = p[4], .save_v = p[5], .save_g = p[6], .save_b = p[7], .cur = cur, .heads = s.kda.len(), .mtp_caches = .{ .lc = p[8 + 4 * nd], .ls = if (kv8) p[8 + 4 * nd] + capacity * c.kv_lora else 0, .ik = p[9 + 4 * nd], .ig = p[10 + 4 * nd], .pk = p[11 + 4 * nd], .pk_rows = pk_rows }, .mtp_pos_dev = p[12 + 4 * nd], .prompt_hidden = p[13 + 4 * nd], .kda_layers = nk, .caches = caches, .conv_c = 3 * n, .rec_layer = s.kda.len() * kda.dk * kda.dv * 4, .proj_layer = window_rows * kw, .proj_row = kw }; + if (paged) |q| { + st.table = p[14 + 4 * nd]; + st.capacity = 0; // no rows until a stream's pages are mapped + st.place(w, q); + } + return st; + } + + /// A paged sequence's caches on the pages of `p` (the engine's own, made before its pool, placed when it is made). + pub fn place(st: *State, w: *const weights.Weights, p: Paged) void { + for (st.caches, 0..) |*x, j| onPages(x, &w.config, p, st.table, j); + onPages(&st.mtp_caches, &w.config, p, st.table, st.caches.len); } pub fn deinit(st: *State) void { diff --git a/zig/src/native/cuda.zig b/zig/src/native/cuda.zig index 52aff4316d..6dd0ffc7d8 100644 --- a/zig/src/native/cuda.zig +++ b/zig/src/native/cuda.zig @@ -349,7 +349,7 @@ fn openWith(comptime F: type, a: Allocator, gpa: Allocator, io: std.Io, o: api.O const layout = kv_pages.layout(loaded); var paged: ?kv_pages.Plan = null; const streams = if (layout) |l| blk: { - paged = try kv_pages.plan(F, a, l, room, o, @intCast(window), problem, host_cache) orelse return null; + paged = try kv_pages.plan(F, a, l, room, o, @intCast(window), problem, host_cache, kv_pages.enginePaged(loaded)) orelse return null; break :blk paged.?.lanes; } else budget.admit(grow, served.stream_bytes, free, o.lanes, o.lanes_fixed) catch |e| { problem.* = switch (e) { @@ -384,7 +384,7 @@ fn openWith(comptime F: type, a: Allocator, gpa: Allocator, io: std.Io, o: api.O const held = served.stream_bytes * (streams -| free); const kept = kv_pages.kept(F, loaded, paged, o, room, held, host_budget, host_cache); const cache = kept.copied; - const shown = try kv_pages.text(F, a, served.stream_bytes, layout, paged, @intCast(window), kept, o.kv8); + const shown = try kv_pages.text(F, a, served.stream_bytes, layout, paged, kv_pages.enginePaged(loaded), @intCast(window), kept, o.kv8); h.startup = try std.fmt.allocPrint(gpa, @import("cuda_startup.zig").format, .{ capability, device, name, if (after.unified) ", memory shared with the host" else "", toGib(model), if (learn_bytes > 0) try std.fmt.allocPrint(a, " and {d:.2} GiB kept for --slide's trainer", .{toGib(learn_bytes)}) else "", streams, if (streams == 1) "" else "s", shown[0], toGib(room), toGib(after.reserve), @import("cuda_startup.zig").rows(F, loaded), if (segments > 1) try std.fmt.allocPrint(a, ", {d} staggered segments a call", .{segments}) else "", shown[1], if (kernels) |dir| try std.fmt.allocPrint(a, "glue kernels captured at {s}", .{dir}) else "own glue kernels", diff --git a/zig/src/native/cuda_kv_pool.zig b/zig/src/native/cuda_kv_pool.zig index 754602de1d..9341fc903d 100644 --- a/zig/src/native/cuda_kv_pool.zig +++ b/zig/src/native/cuda_kv_pool.zig @@ -12,6 +12,11 @@ pub fn layout(loaded: anytype) ?kv_pool.Layout { return if (comptime @hasField(@TypeOf(loaded), "layout")) loaded.layout else null; } +/// Whether the engine's own sequence takes pages too (GLM's when it pages): the plan sets a window of them aside first. +pub fn enginePaged(loaded: anytype) bool { + return if (comptime @hasField(@TypeOf(loaded), "engine_paged")) loaded.engine_paged else false; +} + /// The real native startup room before page planning and cache allocation. pub fn startupRoom(comptime F: type, a: Allocator, loaded: anytype, o: api.Open, available: u64, host_budget: u64, problem: *[]const u8) !?u64 { const memory = @import("cuda_family_memory.zig"); @@ -37,18 +42,18 @@ pub fn placeHost(comptime F: type, a: Allocator, loaded: anytype, o: api.Open, a fn fits(comptime F: type, loaded: anytype, o: api.Open, room: u64, window: u64, stream_bytes: u64) bool { const want = budget.cacheBytes(o.prompt_cache_gib, std.math.maxInt(u64), 0); const rows: u64 = if (comptime @hasDecl(F, "window_rows")) F.window_rows else 0; - if (layout(loaded)) |l| return kv_pool.holds(l, .{ .room = room, .engine = 0, .lanes = o.lanes, .fixed = o.lanes_fixed, .cache = want, .snaps = named(o), .window = window + rows }); + if (layout(loaded)) |l| return kv_pool.holds(l, .{ .room = room, .engine = 0, .lanes = o.lanes, .fixed = o.lanes_fixed, .cache = want, .snaps = named(o), .window = window + rows, .engine_paged = enginePaged(loaded) }); const n = budget.admit(room, stream_bytes, loaded.free_streams, o.lanes, o.lanes_fixed) catch return false; return room -| stream_bytes * (n -| loaded.free_streams) >= want; } -/// The pool beside the engine's own full-window sequence, which `model` counts once; null: `problem` says why not. -pub fn plan(comptime F: type, a: Allocator, l: kv_pool.Layout, room: u64, o: api.Open, window: u64, problem: *[]const u8, host_mode: bool) !?Plan { +/// The pool beside the engine's own sequence (in `model`: a window, or on pages its fixed part); null: see `problem`. +pub fn plan(comptime F: type, a: Allocator, l: kv_pool.Layout, room: u64, o: api.Open, window: u64, problem: *[]const u8, host_mode: bool, engine_paged: bool) !?Plan { const rows: u64 = if (comptime @hasDecl(F, "window_rows")) F.window_rows else 0; const want = if (host_mode) 0 else budget.cacheBytes(o.prompt_cache_gib, std.math.maxInt(u64), 0); - return kv_pool.plan(l, .{ .room = room, .engine = 0, .lanes = o.lanes, .fixed = o.lanes_fixed, .cache = want, .snaps = named(o), .window = window + rows }) catch |e| { + return kv_pool.plan(l, .{ .room = room, .engine = 0, .lanes = o.lanes, .fixed = o.lanes_fixed, .cache = want, .snaps = named(o), .window = window + rows, .engine_paged = engine_paged }) catch |e| { problem.* = switch (e) { - error.NoStream => "the CUDA memory budget fits no stream beside the model", + error.NoStream => if (engine_paged) try std.fmt.allocPrint(a, "the CUDA memory budget fits no stream beside the model: one at a {d}-token window takes {d:.2} GiB of KV pages, and the budget leaves {d:.2} GiB", .{ window, toGib(l.pages(window + rows) * l.pageBytes()), toGib(room) }) else "the CUDA memory budget fits no stream beside the model", error.TooMany => try std.fmt.allocPrint(a, "--parallel {d} needs {d:.2} GiB for its other streams' fixed parts and a page each, and the CUDA memory budget leaves {d:.2} GiB: serve fewer, or free device memory", .{ o.lanes, toGib((o.lanes - 1) * (l.fixed_bytes + l.pageBytes())), toGib(room) }), }; return null; @@ -71,7 +76,8 @@ pub const Kept = struct { snaps: usize, copied: u64, snap_bytes: u64, host: bool pub fn kept(comptime F: type, loaded: anytype, paged: ?Plan, o: api.Open, room: u64, held: u64, host_budget: u64, host_mode: bool) Kept { if (host_mode) return .{ .snaps = 0, .copied = hostBudget(o, host_budget), .snap_bytes = stagingBytes(F, loaded, hostBudget(o, host_budget)), .host = true, .most = named(o) orelse std.math.maxInt(usize) }; - if (comptime !@hasDecl(F, "snap_slots")) return .{ .snaps = 0, .copied = if (@hasDecl(F, "snap_save")) budget.cacheBytes(o.prompt_cache_gib, room, held) else 0, .snap_bytes = 0 }; + // on pages the plan set the cache aside already; `held` counts every stream at a whole window + if (comptime !@hasDecl(F, "snap_slots")) return .{ .snaps = 0, .copied = if (!@hasDecl(F, "snap_save")) 0 else if (paged) |p| p.copied else budget.cacheBytes(o.prompt_cache_gib, room, held), .snap_bytes = 0 }; const b: u64 = loaded.snap_bytes; const s = if (paged) |p| kv_pool.split(p.snaps * b + p.copied, b, p.snaps) else kv_pool.split(budget.cacheBytes(o.prompt_cache_gib, room, held), b, named(o)); return .{ .snaps = s.snaps, .copied = s.copied, .snap_bytes = b }; @@ -86,8 +92,8 @@ pub fn store(comptime F: type, s: *api.prompt_cache.Store, target: anytype, k: K } /// The startup line's streams and cache. -pub fn text(comptime F: type, a: Allocator, stream_bytes: u64, l: ?kv_pool.Layout, paged: ?Plan, window: u64, k: Kept, kv8: bool) ![2][]const u8 { - const shape = if (paged) |p| try std.fmt.allocPrint(a, "the engine's own at a {d}-token window ({d:.2} GiB, in the model's), {d} more taking {d} pages of {d} rows ({d:.2} GiB) as they need them, {d:.0} MiB fixed each", .{ window, toGib(stream_bytes), p.lanes - 1, p.pages, l.?.page_tokens, toGib(p.pages * l.?.pageBytes()), @as(f64, @floatFromInt(l.?.fixed_bytes)) / (1 << 20) }) else try std.fmt.allocPrint(a, "{d:.2} GiB each at a {d}-token window{s}", .{ toGib(stream_bytes), window, if (kv8) " (fp8 caches)" else "" }); +pub fn text(comptime F: type, a: Allocator, stream_bytes: u64, l: ?kv_pool.Layout, paged: ?Plan, engine_paged: bool, window: u64, k: Kept, kv8: bool) ![2][]const u8 { + const shape = if (paged) |p| (if (engine_paged) try std.fmt.allocPrint(a, "the engine's own and {d} more taking {d} pages of {d} rows ({d:.2} GiB) as they need them (enough for one at a {d}-token window), {d:.0} MiB fixed each", .{ p.lanes - 1, p.pages, l.?.page_tokens, toGib(p.pages * l.?.pageBytes()), window, @as(f64, @floatFromInt(l.?.fixed_bytes)) / (1 << 20) }) else try std.fmt.allocPrint(a, "the engine's own at a {d}-token window ({d:.2} GiB, in the model's), {d} more taking {d} pages of {d} rows ({d:.2} GiB) as they need them, {d:.0} MiB fixed each", .{ window, toGib(stream_bytes), p.lanes - 1, p.pages, l.?.page_tokens, toGib(p.pages * l.?.pageBytes()), @as(f64, @floatFromInt(l.?.fixed_bytes)) / (1 << 20) })) else try std.fmt.allocPrint(a, "{d:.2} GiB each at a {d}-token window{s}", .{ toGib(stream_bytes), window, if (kv8) " (fp8 caches)" else "" }); const placed = comptime @hasDecl(F, "snap_place"); // the startup line says where its prompt states went, and why const cache = if (k.copied == 0) ", no prompt cache" else if (k.host) try std.fmt.allocPrint(a, ", {s}a {d:.1} GiB prompt cache in host memory with {d:.0} MiB of device staging", .{ if (placed) "prompt states on host, the cache doesn't fit on the card: " else "", toGib(k.copied), @as(f64, @floatFromInt(k.snap_bytes)) / (1 << 20) }) else if (comptime @hasDecl(F, "snap_slots")) try std.fmt.allocPrint(a, ", {s}a prompt cache of {d} snapshot slots ({d:.1} GiB) and {d:.1} GiB of copied rows", .{ if (placed) "prompt states on device: " else "", k.snaps, toGib(k.snaps * k.snap_bytes), toGib(k.copied) }) else try std.fmt.allocPrint(a, ", a {d:.1} GiB prompt cache", .{toGib(k.copied)}); return .{ shape, cache }; @@ -147,11 +153,49 @@ test "kv placement: a discrete card keeps prompt states when the asked cache fit const shared = .{ .ctx = @as(*anyopaque, &ctx), .host_snaps = false, .snap_bytes = @as(u64, 48 << 20), .layout = @as(?kv_pool.Layout, l), .free_streams = @as(u32, 1) }; try std.testing.expectEqual(@as(?bool, false), try placeHost(Placing, a, shared, o, 12 << 30, 70000, 0, &problem)); try std.testing.expectEqual(@as(?bool, true), try placeHost(Fixture, a, card, o, 70 << 30, 70000, 0, &problem)); - const on_host = (try text(Placing, a, 1 << 30, l, null, 70000, .{ .snaps = 0, .copied = 8 << 30, .snap_bytes = 48 << 20, .host = true }, false))[1]; + const on_host = (try text(Placing, a, 1 << 30, l, null, false, 70000, .{ .snaps = 0, .copied = 8 << 30, .snap_bytes = 48 << 20, .host = true }, false))[1]; try std.testing.expect(std.mem.startsWith(u8, on_host, ", prompt states on host, the cache doesn't fit on the card: a 8.0 GiB")); - const on_device = (try text(Placing, a, 1 << 30, l, null, 70000, .{ .snaps = 64, .copied = 13 << 30, .snap_bytes = 48 << 20 }, false))[1]; + const on_device = (try text(Placing, a, 1 << 30, l, null, false, 70000, .{ .snaps = 64, .copied = 13 << 30, .snap_bytes = 48 << 20 }, false))[1]; try std.testing.expect(std.mem.startsWith(u8, on_device, ", prompt states on device: a prompt cache of 64 snapshot slots")); } +const Copies = struct { + pub const snap_save = 1; +}; +test "kv pages: a family without snapshot slots keeps the paged plan's cache, not what window-sized streams leave" { + const l: kv_pool.Layout = .{ .fixed_bytes = 180 << 20, .token_bytes = 13_056, .page_tokens = 512, .snap_bytes = 0 }; + const p = try kv_pool.plan(l, .{ .room = 40 << 30, .engine = 0, .lanes = 4, .cache = 16 << 30 }); + const o: api.Open = .{ .dir = "", .model_type = "g", .lanes = 4 }; + const k = kept(Copies, .{}, p, o, 40 << 30, 3 * (12 << 30), 0, false); // held: three streams at a 1M window + try std.testing.expectEqual(@as(u64, 16 << 30), k.copied); + try std.testing.expectEqual(@as(usize, 0), k.snaps); + try std.testing.expectEqual(@as(u64, 4 << 30), kept(Copies, .{}, null, o, 40 << 30, 3 * (12 << 30), 0, false).copied); +} +const Owned = struct { + pub const snap_save = 1; + pub const window_rows = 8; +}; +test "kv pages: an engine whose own rows are pages plans a window of them first, refuses a room under it, and says so" { + var arena = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena.deinit(); + const a = arena.allocator(); + var problem: []const u8 = ""; + const l: kv_pool.Layout = .{ .fixed_bytes = 180 << 20, .token_bytes = 13_056, .page_tokens = 512, .snap_bytes = 0 }; + const o: api.Open = .{ .dir = "", .model_type = "g", .lanes = 4, .prompt_cache_gib = 4 }; + try std.testing.expect(enginePaged(.{ .layout = @as(?kv_pool.Layout, l), .engine_paged = true }) and !enginePaged(.{ .layout = @as(?kv_pool.Layout, l) })); + const p = (try plan(Owned, a, l, 40 << 30, o, 8192, &problem, false, true)).?; + try std.testing.expectEqual(@as(u32, 4), p.lanes); + try std.testing.expectEqual(@as(usize, 4 * 17), p.pages); // every stream at most 8,192 rows and a verify window: 17 pages + const floor = 17 * l.pageBytes(); + try std.testing.expectEqual(@as(?Plan, null), try plan(Owned, a, l, floor - 1, o, 8192, &problem, false, true)); + try std.testing.expect(std.mem.indexOf(u8, problem, "8192-token window") != null); + const shown = (try text(Owned, a, 180 << 20, l, p, true, 8192, .{ .snaps = 0, .copied = 4 << 30, .snap_bytes = 0 }, false))[0]; + try std.testing.expectEqualStrings("the engine's own and 3 more taking 68 pages of 512 rows (0.42 GiB) as they need them (enough for one at a 8192-token window), 180 MiB fixed each", shown); + // the asked cache holds past the window's pages and a fixed part and page for each other stream, not a byte less + const each = (180 << 20) + l.pageBytes(); + const loaded = .{ .layout = @as(?kv_pool.Layout, l), .engine_paged = true, .free_streams = @as(u32, 1) }; + try std.testing.expect(fits(Owned, loaded, o, floor + 3 * each + (4 << 30), 8192, 0)); + try std.testing.expect(!fits(Owned, loaded, o, floor + 3 * each + (4 << 30) - 1, 8192, 0)); +} test "kv merge: host armed-state staging is reserved once before planning the page pool" { const loaded = .{ .host_snaps = true, .snap_bytes = @as(u64, 8 << 20) }; try std.testing.expectEqual(@as(u64, 8 << 20), stagingBytes(Fixture, loaded, 1 << 30)); diff --git a/zig/src/native/cuda_kv_safety_test.zig b/zig/src/native/cuda_kv_safety_test.zig index 1a53a7b590..7f2b634aa9 100644 --- a/zig/src/native/cuda_kv_safety_test.zig +++ b/zig/src/native/cuda_kv_safety_test.zig @@ -20,7 +20,7 @@ pub fn run(comptime planner: type, disabled: bool) !void { var allocations: usize = 0; const left = try planner.startupRoom(F, arena.allocator(), loaded, o, available, 1 << 30, &problem); if (left) |room| { - const plan = (try planner.plan(F, arena.allocator(), layout, room, o, 1024, &problem, true)).?; + const plan = (try planner.plan(F, arena.allocator(), layout, room, o, 1024, &problem, true, planner.enginePaged(loaded))).?; const kept = planner.kept(F, loaded, plan, o, room, 0, 1 << 30, planner.host(F, loaded)); var store: api.prompt_cache.Store = undefined; if (kept.copied > 0) try planner.store(F, &store, &allocations, kept); diff --git a/zig/tests/cuda/contract_glm.zig b/zig/tests/cuda/contract_glm.zig new file mode 100644 index 0000000000..f66d37edbd --- /dev/null +++ b/zig/tests/cuda/contract_glm.zig @@ -0,0 +1,62 @@ +//! GLM-5.3-Flash's KV page kernels under the kernel contract: their two modules and one case for each launch. +const std = @import("std"); +const cuda = @import("cuda"); +const core = @import("core"); +const glm = @import("glm_cuda"); +const check = @import("check.zig"); +const pages_k = @import("contract_glm_pages.zig"); +const index_k = @import("contract_glm_index.zig"); +const attn_k = @import("contract_glm_attn.zig"); + +const g = core.contract.gates; +const kk = cuda.kernels; + +/// The latent write, the pooled keys, the index scores, the selected tokens' rows and the MLA chunks. +pub const Cases = struct { + glue: cuda.Module, + ops: cuda.Module, + lat: pages_k.LatWrite, + pool: pages_k.PoolKeys, + scores: index_k.Scores, + tokens: index_k.Tokens, + chunks: [4]attn_k.Chunks, + + pub fn init(gpu: check.Gpu, s: cuda.Stream) !Cases { + var glue = try cuda.Module.load(gpu.d, kk.glm_glue); + errdefer glue.unload(); + const fns = try glm.glue.Kernels.load(glue); + var ops = try cuda.Module.load(gpu.d, kk.glm_ops); + errdefer ops.unload(); + var cases: Cases = .{ + .glue = glue, + .ops = ops, + .lat = .{ .rig = .{ .gpu = gpu, .s = s }, .f = fns.lat_write }, + .pool = .{ .rig = .{ .gpu = gpu, .s = s }, .f = fns.pool_keys }, + .scores = .{ .rig = .{ .gpu = gpu, .s = s }, .f = fns.scores }, + .tokens = .{ .rig = .{ .gpu = gpu, .s = s }, .f = try ops.function("tf_glm_dsa_tokens") }, + .chunks = undefined, + }; + for (&cases.chunks, [_]u32{ 16, 16, 32, 32 }, [_]bool{ false, true, false, true }) |*x, hb, sparse| + x.* = .{ .rig = .{ .gpu = gpu, .s = s }, .f = if (hb == 16) fns.chunks_16 else fns.chunks_32, .merge = fns.merge, .form = .{ .hb = hb, .sparse = sparse } }; + return cases; + } + + /// Each case's kernel in the contract's order; the cases stay where they are until deinit. + pub fn append(cases: *Cases, a: std.mem.Allocator, kernels: *std.ArrayList(g.Kernel)) !void { + try kernels.append(a, cases.lat.kernel()); + try kernels.append(a, cases.pool.kernel()); + try kernels.append(a, cases.scores.kernel()); + try kernels.append(a, cases.tokens.kernel()); + for (&cases.chunks) |*x| try kernels.append(a, x.kernel()); + } + + pub fn deinit(cases: *Cases) void { + for (&cases.chunks) |*x| x.deinit(); + cases.tokens.deinit(); + cases.ops.unload(); + cases.scores.deinit(); + cases.pool.deinit(); + cases.lat.deinit(); + cases.glue.unload(); + } +}; diff --git a/zig/tests/cuda/contract_glm_attn.zig b/zig/tests/cuda/contract_glm_attn.zig new file mode 100644 index 0000000000..55cc454382 --- /dev/null +++ b/zig/tests/cuda/contract_glm_attn.zig @@ -0,0 +1,227 @@ +//! GLM's MLA chunk kernels on KV pages under the contract on CUDA: dense and sparse, each run through the chunk merge. +const std = @import("std"); +const cuda = @import("cuda"); +const core = @import("core"); +const glm = @import("glm_cuda"); +const rig_ = @import("contract_rig.zig"); +const pages = @import("contract_glm_pages.zig"); + +const contract = core.contract; +const c = contract.spec; +const g = contract.gates; +const lw = pages.lw; +const heads = 32; +const chunk = glm.latent.chunk; +const scale: f64 = 1.0 / 16.0; // qk 256's 1 / sqrt: exact in fp32 +const soft_why = "online softmax over 512-key chunks, then the chunk merge: no input makes its sums exact in fp32"; + +/// A key's 512 products, then a chunk's 512 keys and the merge's chunks. +fn attnDepth(s: c.Shape) usize { + return lw + chunk + s.k / chunk + 1; +} + +pub const Form = struct { hb: u32, sparse: bool }; + +/// rows x a row's outputs (heads x 512) x, dense, the page edge its rows straddle; sparse, a row's tokens. +pub fn spec(comptime f: Form) c.Spec { + const name = std.fmt.comptimePrint("cuda glm tf_glm_chunks_{d} {s} on pages out of order, with tf_glm_merge", .{ f.hb, if (f.sparse) "sparse" else "dense" }); + const served: []const c.Shape = switch (f.hb) { + 16 => if (f.sparse) &.{.{ .rows = 8, .n = heads * lw, .k = glm.sparse.width }} else &.{.{ .rows = 8, .n = heads * lw, .k = 2 * chunk }}, + else => if (f.sparse) &.{.{ .rows = 64, .n = heads * lw, .k = 300 }} else &.{.{ .rows = 64, .n = heads * lw, .k = chunk }}, + }; + return .{ .name = name, .accum = .f32, .depth = attnDepth, .ops = 8, .out = .bf16, .served = served, .sums = .{ .f16 = 0, .bf16 = 0, .exact = false, .why = soft_why } }; +} + +/// One chunk kernel's calls: queries, the cache's rows on pages out of order, and for sparse each row's token list. +pub const Chunks = struct { + rig: rig_.Rig, + f: cuda.Function, + merge: cuda.Function, + form: Form, + shape: c.Shape = undefined, + q: []u16 = &.{}, + cache: []u16 = &.{}, // the latent rows in row order, as the pages hold them + keys: []u32 = &.{}, // sparse: each row's tokens [rows][k] + tol: []f64 = &.{}, + ids: []i32 = &.{}, + at: usize = 0, + nch: usize = 0, + in: [4]u64 = @splat(0), + pos: u64 = 0, + pt: u64 = 0, + parts: [3]u64 = @splat(0), + out: u64 = 0, + + pub fn deinit(self: *Chunks) void { + self.free(); + self.rig.deinit(); + } + + fn free(self: *Chunks) void { + const a = self.rig.gpu.gpa; + a.free(self.q); + a.free(self.cache); + a.free(self.keys); + a.free(self.tol); + a.free(self.ids); + self.q = &.{}; + self.cache = &.{}; + self.keys = &.{}; + self.tol = &.{}; + self.ids = &.{}; + } + + pub fn kernel(self: *Chunks) g.Kernel { + return .{ .spec = switch (self.form.hb) { + 16 => if (self.form.sparse) spec(.{ .hb = 16, .sparse = true }) else spec(.{ .hb = 16, .sparse = false }), + else => if (self.form.sparse) spec(.{ .hb = 32, .sparse = true }) else spec(.{ .hb = 32, .sparse = false }), + }, .ctx = self, .vt = &vt }; + } + + const vt = g.hooks(Chunks); + + /// Keys row r sees: dense the rows through at + r, sparse its tokens. + fn seen(self: *const Chunks, r: usize) usize { + return if (self.form.sparse) self.shape.k else self.at + r + 1; + } + + fn key(self: *const Chunks, r: usize, i: usize) usize { + return if (self.form.sparse) self.keys[r * self.shape.k + i] else i; + } + + /// Queries and rows in [-1, 1); dense from row k - rows / 2, across a page edge; sparse tokens under 4,096. + pub fn prepare(self: *Chunks, shape: c.Shape, fill: g.Fill, seed: u64) !bool { + if (shape.n != heads * lw or shape.rows == 0 or shape.k == 0) return false; + self.free(); + self.rig.reset(); + self.shape = shape; + const a = self.rig.gpu.gpa; + var prng = std.Random.DefaultPrng.init(seed); + const rng = prng.random(); + const span: usize = if (self.form.sparse) 4096 else shape.k + shape.rows; + self.at = if (self.form.sparse) 0 else shape.k - @min(shape.k, shape.rows / 2); + self.nch = ((if (self.form.sparse) shape.k else self.at + shape.rows) + chunk - 1) / chunk; + self.q = try a.alloc(u16, shape.rows * heads * lw); + self.cache = try a.alloc(u16, span * lw); + for (self.q) |*v| v.* = c.toBf16(rng.float(f32) * 2 - 1); + for (self.cache) |*v| v.* = c.toBf16(rng.float(f32) * 2 - 1); + if (fill == .nonfinite) { + self.q[3] = c.toBf16(std.math.nan(f32)); + if (!self.form.sparse) self.cache[9] = c.toBf16(std.math.inf(f32)); + } + if (self.form.sparse) { + self.keys = try a.alloc(u32, shape.rows * shape.k); + for (0..shape.rows) |r| for (0..shape.k) |i| { + self.keys[r * shape.k + i] = @intCast((i * 4096 / shape.k + rng.uintLessThan(usize, 4096 / shape.k)) % 4096); + }; + if (fill == .nonfinite and shape.rows > 1) self.cache[self.keys[shape.k] * lw + 9] = c.toBf16(std.math.inf(f32)); + } + const n_pages = (span + pages.rows_a_page - 1) / pages.rows_a_page; + self.ids = try pages.shuffled(a, rng, n_pages); + const arena = try a.alloc(u8, (n_pages + 3) * pages.page_bytes); + defer a.free(arena); + @memset(arena, 0); + for (0..span) |t| { + const at = pages.part_bytes + @as(usize, @intCast(self.ids[t / pages.rows_a_page])) * pages.page_bytes + (t % pages.rows_a_page) * lw * 2; + @memcpy(arena[at..][0 .. lw * 2], std.mem.sliceAsBytes(self.cache[t * lw ..][0..lw])); + } + const page_rows = pages.page_bytes / (lw * 2); + const tok = try a.alloc(i32, if (self.form.sparse) shape.rows * shape.k else 1); + defer a.free(tok); + const cnt = try a.alloc(i32, shape.rows); + defer a.free(cnt); + @memset(tok, 0); + @memset(cnt, @intCast(shape.k)); + if (self.form.sparse) for (tok, self.keys) |*v, t| { // as the token lists hold them: a token's row in the pages + v.* = @intCast(@as(usize, @intCast(self.ids[t / pages.rows_a_page])) * page_rows + t % pages.rows_a_page); + }; + const p: i32 = @intCast(self.at); + self.in[0] = try self.rig.input(std.mem.sliceAsBytes(self.q)); + self.in[1] = try self.rig.input(arena) + pages.part_bytes; + self.in[2] = try self.rig.input(std.mem.sliceAsBytes(tok)); + self.in[3] = try self.rig.input(std.mem.sliceAsBytes(cnt)); + self.pos = try self.rig.input(std.mem.asBytes(&p)); + self.pt = try self.rig.input(std.mem.sliceAsBytes(self.ids)); + const n = self.nch * shape.rows * heads; + self.parts = .{ try self.rig.scratch(n * lw * 4), try self.rig.scratch(n * 4), try self.rig.scratch(n * 4) }; + self.out = try self.rig.output(shape.rows * heads * lw * 2); + return true; + } + + pub fn run(self: *Chunks) !void { + const rows: i64 = @intCast(self.shape.rows); + const w: i64 = if (self.form.sparse) @intCast(self.shape.k) else 0; + var a: cuda.Args = .{}; + inline for (.{ self.in[0], self.in[1], self.pos, self.in[2], self.in[3], self.parts[0], self.parts[1], self.parts[2], rows, @as(i64, heads), w, @as(f32, @floatCast(scale)), @as(c_int, @intFromBool(self.form.sparse)), self.pt, @as(i64, pages.page_bytes) }) |v| a.add(v); + const shared = if (self.form.hb == 32) glm.glue.chunksSmem(32) else glm.glue.chunksSmem(16); + try cuda.launch.launch(self.f, .{ .grid = .{ .x = @intCast(self.shape.rows), .y = (heads + self.form.hb - 1) / self.form.hb, .z = @intCast(self.nch) }, .block = .{ .x = 256 }, .shared = shared }, self.rig.s, &a); + var m: cuda.Args = .{}; + inline for (.{ self.parts[0], self.parts[1], self.parts[2], self.out, self.in[3], rows, @as(i64, heads), @as(c_int, @intCast(self.nch)), @as(c_int, @intFromBool(self.form.sparse)) }) |v| m.add(v); + try cuda.launch.launch(self.merge, .{ .grid = .{ .x = @intCast(self.shape.rows), .y = heads }, .block = .{ .x = 128 } }, self.rig.s, &m); + try self.rig.s.synchronize(); + } + + pub fn count(self: *Chunks) usize { + return self.shape.rows * heads * lw; + } + + /// Row r, head h: softmax(q . c scale) . c over its keys, dots and sums in `acc` (null: float64), into `o`. + fn attend(self: *Chunks, r: usize, h: usize, acc: ?c.Accum, order: g.Order, o: []f64, tol: ?[]f64) !void { + const a = self.rig.gpu.gpa; + const n = self.seen(r); + const s = try a.alloc(f64, 3 * n); + defer a.free(s); + const ds = s[n .. 2 * n]; + const p = s[2 * n ..]; + var terms: [lw]f64 = undefined; + var most = -std.math.inf(f64); + for (0..n) |i| { + var abs: f64 = 0; + for (&terms, 0..) |*x, d| { + x.* = c.fromBf16(self.q[(r * heads + h) * lw + d]) * c.fromBf16(self.cache[self.key(r, i) * lw + d]); + abs += @abs(x.*); + } + s[i] = (if (acc) |k| pages.sumIn(k, order, &terms) else pages.sumIn(.none, .sequential, &terms)) * scale; + ds[i] = abs * scale * 520 * 0x1p-24; + most = pages.maxNan(most, s[i]); + } + for (p, s) |*x, v| x.* = @exp(v - most); + const l = if (acc) |k| pages.sumIn(k, order, p) else pages.sumIn(.none, .sequential, p); + const col = try a.alloc(f64, n); + defer a.free(col); + for (o, 0..) |*od, d| { + var spread: f64 = 0; + for (col, p, 0..) |*x, w, i| { + const v = c.fromBf16(self.cache[self.key(r, i) * lw + d]); + x.* = w * v; + spread += w * @abs(v) * (0x1p-8 + ds[i]); + } + od.* = (if (acc) |k| pages.sumIn(k, order, col) else pages.sumIn(.none, .sequential, col)) / l; + if (tol) |t| t[d] = @abs(od.*) * 0x1p-8 + spread / l + 1e-30; + } + } + + /// float64 o = sum p c / sum p, p = exp(s - max); bound: bf16 out and p (2^-8 each), 520 2^-24 scale sum |q c|. + pub fn reference(self: *Chunks, want: []f64, mag: []f64) !void { + const a = self.rig.gpu.gpa; + a.free(self.tol); + self.tol = try a.alloc(f64, want.len); + for (0..self.shape.rows) |r| for (0..heads) |h| { + const at = (r * heads + h) * lw; + try self.attend(r, h, null, .sequential, want[at..][0..lw], self.tol[at..][0..lw]); + }; + for (mag, want) |*m, w| m.* = @abs(w); + } + + pub fn bounds(self: *Chunks, _: []const f64, _: []const f64, allow: []f64) !void { + for (allow, self.tol) |*l, t| l.* = if (std.math.isFinite(t)) t else std.math.floatMax(f64); + } + + pub fn simulate(self: *Chunks, acc: c.Accum, order: g.Order, out: []f64) !void { + for (0..self.shape.rows) |r| for (0..heads) |h| { + const at = (r * heads + h) * lw; + try self.attend(r, h, acc, order, out[at..][0..lw], null); + for (out[at..][0..lw]) |*v| v.* = c.fromBf16(c.toBf16(@floatCast(v.*))); + }; + } +}; diff --git a/zig/tests/cuda/contract_glm_index.zig b/zig/tests/cuda/contract_glm_index.zig new file mode 100644 index 0000000000..e3936feb21 --- /dev/null +++ b/zig/tests/cuda/contract_glm_index.zig @@ -0,0 +1,298 @@ +//! GLM's DSA index kernels on KV pages under the contract on CUDA: the pool scores and the token lists' page rows. +const std = @import("std"); +const cuda = @import("cuda"); +const core = @import("core"); +const glm = @import("glm_cuda"); +const rig_ = @import("contract_rig.zig"); +const pages = @import("contract_glm_pages.zig"); + +const contract = core.contract; +const c = contract.spec; +const g = contract.gates; +const sparse = glm.sparse; +const dk = pages.dk; +const heads = 32; +const w_stride = dk + heads; // the head weights sit after the index keys in a row, as the block passes them + +fn noSums(_: c.Shape) usize { + return 0; +} + +/// A key's 128 products, then the 32 heads' weighted sum. +fn scoreDepth(_: c.Shape) usize { + return dk + heads; +} + +const nan_relu = "relu is fmaxf(x, 0), as Triton's tl.maximum: a NaN dot scores 0 where float64 keeps the NaN"; + +/// scores: rows x pool columns (k: an index key's width); tokens: rows x K pools a row. +pub fn spec(comptime which: enum { scores, tokens }) c.Spec { + return switch (which) { + .scores => .{ .name = "cuda glm tf_glm_scores on pages out of order", .accum = .f32, .depth = scoreDepth, .ops = 2, .out = .f32, .served = &.{ .{ .rows = 8, .n = 2048, .k = dk }, .{ .rows = 64, .n = 1024, .k = dk } }, .known = .{ .gate = .nonfinite, .finding = nan_relu } }, + .tokens => .{ .name = "cuda glm tf_glm_dsa_tokens on pages out of order", .accum = .none, .depth = noSums, .out = .raw, .nan = .{ .stated = "integers only: pool ids, positions and page ids" }, .served = &.{ .{ .rows = 8, .n = sparse.topk_pools, .k = 1 }, .{ .rows = 64, .n = sparse.topk_pools, .k = 1 } } }, + }; +} + +const maxNan = pages.maxNan; +const held = pages.held; +const sumIn = pages.sumIn; + +/// sparse._scores: each row's visible pools scored by 32 heads (a weighted sum of relu(q . key)), the rest -inf. +pub const Scores = struct { + rig: rig_.Rig, + f: cuda.Function, + shape: c.Shape = undefined, + q: []u16 = &.{}, + w: []u16 = &.{}, + keys: []u16 = &.{}, // the pooled keys in pool order, as the pages hold them + ids: []i32 = &.{}, + at: usize = 0, + seen: usize = 0, // the last row's visible pools + scale: f32 = 0, // the dots' scale and the head weights' (served: 1/sqrt(128), 1/sqrt(32); stress: powers of two) + wscale: f32 = 0, + in: [3]u64 = @splat(0), + pos: u64 = 0, + pt: u64 = 0, + out: u64 = 0, + + pub fn deinit(self: *Scores) void { + self.free(); + self.rig.deinit(); + } + + fn free(self: *Scores) void { + const a = self.rig.gpu.gpa; + inline for (.{ "q", "w", "keys" }) |name| { + a.free(@field(self, name)); + @field(self, name) = &.{}; + } + a.free(self.ids); + self.ids = &.{}; + } + + pub fn kernel(self: *Scores) g.Kernel { + return .{ .spec = spec(.scores), .ctx = self, .vt = &vt }; + } + + const vt = g.hooks(Scores); + + fn visible(self: *const Scores, r: usize) usize { + return (self.at + r + 1) / sparse.pool; + } + + /// Queries and keys in [-2, 2); stress: integers in [-8, 8] and power-of-two scales; non-finite: NaN and inf. + pub fn prepare(self: *Scores, shape: c.Shape, fill: g.Fill, seed: u64) !bool { + if (shape.k != dk or shape.n % 64 != 0 or shape.rows == 0) return false; + self.free(); + self.rig.reset(); + self.shape = shape; + const a = self.rig.gpu.gpa; + var prng = std.Random.DefaultPrng.init(seed); + const rng = prng.random(); + self.at = if (shape.rows <= 8) 4100 else 2000; + self.scale = if (fill == .stress) 0.125 else @floatCast(std.math.pow(f64, dk, -0.5)); + self.wscale = if (fill == .stress) 0.25 else sparse.headScale(heads); + self.seen = self.visible(shape.rows - 1); + if (self.seen > shape.n) return false; + self.q = try a.alloc(u16, shape.rows * heads * dk); + self.w = try a.alloc(u16, shape.rows * w_stride); + self.keys = try a.alloc(u16, self.seen * dk); + for ([_][]u16{ self.q, self.keys }) |x| for (x) |*v| { + v.* = c.toBf16(if (fill == .stress) @floatFromInt(@as(i32, @intCast(rng.uintLessThan(u32, 17))) - 8) else rng.float(f32) * 4 - 2); + }; + for (self.w) |*v| v.* = c.toBf16(if (fill == .stress) 1 else rng.float(f32) * 2 - 1); + if (fill == .nonfinite) { + self.q[3] = c.toBf16(std.math.nan(f32)); + self.keys[2 * dk + 5] = c.toBf16(std.math.inf(f32)); + } + const n_pages = (self.seen + 127) / 128; + self.ids = try pages.shuffled(a, rng, n_pages); + const arena = try a.alloc(u8, (n_pages + 3) * pages.page_bytes); + defer a.free(arena); + @memset(arena, 0); + const pk = pages.part_bytes + pages.rows_a_page * pages.lw * 2; // the cache's pooled keys in page 0 + for (0..self.seen) |p| { + const at = pk + @as(usize, @intCast(self.ids[p / 128])) * pages.page_bytes + (p % 128) * dk * 2; + @memcpy(arena[at..][0 .. dk * 2], std.mem.sliceAsBytes(self.keys[p * dk ..][0..dk])); + } + const p: i32 = @intCast(self.at); + self.in[0] = try self.rig.input(std.mem.sliceAsBytes(self.q)); + self.in[1] = try self.rig.input(std.mem.sliceAsBytes(self.w)); + self.in[2] = try self.rig.input(arena) + pk; + self.pos = try self.rig.input(std.mem.asBytes(&p)); + self.pt = try self.rig.input(std.mem.sliceAsBytes(self.ids)); + self.out = try self.rig.output(shape.rows * shape.n * 4); + return true; + } + + pub fn run(self: *Scores) !void { + const rows = self.shape.rows; + const nb = self.shape.n / 64; + const loop: usize = if (rows >= sparse.score_loop_from) sparse.score_loop else 1; + var a: cuda.Args = .{}; + inline for (.{ self.in[0], self.in[1], @as(i64, w_stride), self.in[2], self.out, self.pos, @as(i64, @intCast(self.shape.n)), @as(i64, @intCast(nb)), self.scale, self.wscale, @as(i64, heads), @as(c_int, @intCast(loop)), self.pt, @as(i64, pages.page_bytes) }) |v| a.add(v); + try cuda.launch.launch(self.f, .{ .grid = .{ .x = @intCast(rows), .y = @intCast((nb + loop - 1) / loop) }, .block = .{ .x = 128 } }, self.rig.s, &a); + try self.rig.s.synchronize(); + } + + pub fn count(self: *Scores) usize { + return self.shape.rows * self.shape.n; + } + + /// Row r's score of pool p with every sum in `acc` (null: float64), its terms' magnitude into `mag`. + fn score(self: *const Scores, r: usize, p: usize, acc: ?c.Accum, order: g.Order, mag: *f64) f64 { + mag.* = 0; + if (p >= self.visible(r)) return -std.math.inf(f64); + const scale: f64 = self.scale; + const wscale: f64 = self.wscale; + var dots: [dk]f64 = undefined; + var terms: [heads]f64 = undefined; + for (&terms, 0..) |*t, h| { + var abs: f64 = 0; + for (&dots, 0..) |*x, d| { + x.* = c.fromBf16(self.q[(r * heads + h) * dk + d]) * c.fromBf16(self.keys[p * dk + d]); + abs += @abs(x.*); + } + const wv = c.fromBf16(self.w[r * w_stride + h]) * wscale; + const dot = if (acc) |k| sumIn(k, order, &dots) else sumIn(.none, .sequential, &dots); + t.* = wv * maxNan(dot * scale, 0); + mag.* += if (std.math.isFinite(abs)) @abs(wv) * scale * abs else 0; + } + return if (acc) |k| sumIn(k, order, &terms) else sumIn(.none, .sequential, &terms); + } + + pub fn reference(self: *Scores, want: []f64, mag: []f64) !void { + for (0..self.shape.rows) |r| for (0..self.shape.n) |p| { + want[r * self.shape.n + p] = self.score(r, p, null, .sequential, &mag[r * self.shape.n + p]); + }; + } + + pub fn simulate(self: *Scores, acc: c.Accum, order: g.Order, out: []f64) !void { + var mag: f64 = 0; + for (0..self.shape.rows) |r| for (0..self.shape.n) |p| { + out[r * self.shape.n + p] = held(.f32, self.score(r, p, acc, order, &mag)); + }; + } +}; + +/// sparse.select_tokens' assembly on pages: each row's K pools' 4 tokens and the visible tail as rows in the pages. +pub const Tokens = struct { + rig: rig_.Rig, + f: cuda.Function, + shape: c.Shape = undefined, + pools: []i64 = &.{}, + ids: []i32 = &.{}, + at: usize = 0, + in: u64 = 0, + pos: u64 = 0, + pt: u64 = 0, + tokens: u64 = 0, + counts: u64 = 0, + msg: [200]u8 = undefined, + + const page_rows = pages.page_bytes / (pages.lw * 2); // a page's step in latent rows + + pub fn deinit(self: *Tokens) void { + self.free(); + self.rig.deinit(); + } + + fn free(self: *Tokens) void { + const a = self.rig.gpu.gpa; + a.free(self.pools); + a.free(self.ids); + self.pools = &.{}; + self.ids = &.{}; + } + + pub fn kernel(self: *Tokens) g.Kernel { + return .{ .spec = spec(.tokens), .ctx = self, .vt = &vt }; + } + + const vt = g.hooks(Tokens); + + fn width(self: *const Tokens) usize { + return 4 * self.shape.n + 3; + } + + /// From row 2046 on, so the first rows still see K pools or fewer (count 0) and the rest select K of theirs. + pub fn prepare(self: *Tokens, shape: c.Shape, _: g.Fill, seed: u64) !bool { + if (shape.n == 0 or shape.rows == 0) return false; + self.free(); + self.rig.reset(); + self.shape = shape; + const a = self.rig.gpu.gpa; + var prng = std.Random.DefaultPrng.init(seed); + const rng = prng.random(); + self.at = 2046; + self.pools = try a.alloc(i64, shape.rows * shape.n); + for (0..shape.rows) |r| { + const seen = (self.at + r + 1) / sparse.pool; + for (self.pools[r * shape.n ..][0..shape.n]) |*v| v.* = rng.uintLessThan(u32, @intCast(seen)); + } + self.ids = try pages.shuffled(a, rng, (self.at + shape.rows + pages.rows_a_page - 1) / pages.rows_a_page); + const p: i32 = @intCast(self.at); + self.in = try self.rig.input(std.mem.sliceAsBytes(self.pools)); + self.pos = try self.rig.input(std.mem.asBytes(&p)); + self.pt = try self.rig.input(std.mem.sliceAsBytes(self.ids)); + self.tokens = try self.rig.output(shape.rows * self.width() * 4); + self.counts = try self.rig.output(shape.rows * 4); + return true; + } + + pub fn run(self: *Tokens) !void { + var a: cuda.Args = .{}; + a.add(self.in); + a.add(self.pos); + a.add(@as(c_int, @intCast(self.shape.n))); + a.add(self.tokens); + a.add(self.counts); + a.add(self.pt); + a.add(@as(i64, page_rows)); + try cuda.launch.launch(self.f, .{ .grid = .{ .x = @intCast(self.shape.rows) }, .block = .{ .x = 256 } }, self.rig.s, &a); + try self.rig.s.synchronize(); + } + + pub fn count(self: *Tokens) usize { + return self.shape.rows * (self.width() + 1) * 4; + } + + pub fn reference(_: *Tokens, want: []f64, mag: []f64) !void { + @memset(want, 0); + @memset(mag, 0); + } + + /// Token t's row in the pages: row t % 512 of page ids[t / 512], pages page_rows rows apart. + fn row(self: *const Tokens, t: i64) i32 { + const at: usize = @intCast(t); + return @intCast(@as(i64, self.ids[at / pages.rows_a_page]) * page_rows + @as(i64, @intCast(at % pages.rows_a_page))); + } + + /// Every row's list against the pools and the table on the host, and its count (0 while K pools cover it). + pub fn check(self: *Tokens, _: g.Gate) !?[]const u8 { + const a = self.rig.gpu.gpa; + const w = self.width(); + const k = self.shape.n; + const got = try a.dupe(u8, try self.rig.bytes(self.tokens, self.shape.rows * w * 4)); + defer a.free(got); + const counts = try self.rig.bytes(self.counts, self.shape.rows * 4); + for (0..self.shape.rows) |r| { + const q: i64 = @intCast(self.at + r); + const seen = @divFloor(q + 1, sparse.pool); + var tail: i32 = 0; + for (0..w) |i| { + const want: i32 = if (i < 4 * k) self.row(self.pools[r * k + i / 4] * 4 + @as(i64, @intCast(i % 4))) else blk: { + const t = seen * 4 + @as(i64, @intCast(i - 4 * k)); + tail += @intFromBool(t <= q); + break :blk if (t <= q) self.row(t) else -1; + }; + const v = std.mem.readInt(i32, got[(r * w + i) * 4 ..][0..4], .little); + if (v != want) return try std.fmt.bufPrint(&self.msg, "row {d} token {d}: {d} where the table gives {d}", .{ r, i, v, want }); + } + const n: i32 = if (seen > k) @as(i32, @intCast(4 * k)) + tail else 0; + const v = std.mem.readInt(i32, counts[r * 4 ..][0..4], .little); + if (v != n) return try std.fmt.bufPrint(&self.msg, "row {d}: count {d} where {d} is due", .{ r, v, n }); + } + return null; + } +}; diff --git a/zig/tests/cuda/contract_glm_pages.zig b/zig/tests/cuda/contract_glm_pages.zig new file mode 100644 index 0000000000..f9b07e0ebe --- /dev/null +++ b/zig/tests/cuda/contract_glm_pages.zig @@ -0,0 +1,324 @@ +//! GLM's KV page kernels under the contract on CUDA: the latent write and the pooled keys onto pages out of order. +const std = @import("std"); +const cuda = @import("cuda"); +const core = @import("core"); +const rig_ = @import("contract_rig.zig"); + +const contract = core.contract; +const c = contract.spec; +const g = contract.gates; + +/// Rows a page holds, a latent row's elements, an index key's, and the pools a page holds. +pub const rows_a_page = 512; +pub const lw = 512; +pub const dk = 128; +const pools_a_page = rows_a_page / 4; + +/// A cache's part of a page in bytes: its 512 latent rows, then their 128 pooled keys. +pub const part_bytes = rows_a_page * lw * 2 + pools_a_page * dk * 2; +/// The tests' pages hold two caches' parts and test the second, so a cache's rows sit past its page's start. +pub const page_bytes = 2 * part_bytes; + +/// Page ids for `n` pages: `n` of 0 .. n + 2 in shuffled order, so the table is out of order and skips some. +pub fn shuffled(a: std.mem.Allocator, rng: std.Random, n: usize) ![]i32 { + const all = try a.alloc(i32, n + 3); + defer a.free(all); + for (all, 0..) |*v, i| v.* = @intCast(i); + rng.shuffle(i32, all); + return a.dupe(i32, all[0..n]); +} + +/// The first row a call writes: a decode window across a page edge, a prompt chunk across four. +pub fn firstRow(rows: usize) usize { + return if (rows <= 64) 2 * rows_a_page - rows / 2 else 300; +} + +/// `len` bytes from device `src` to device `dst`, queued on the rig's stream. +fn copy(r: *rig_.Rig, dst: u64, src: u64, len: usize) !void { + try (cuda.DeviceBuffer{ .d = r.gpu.d, .ptr = dst, .len = len }).copyFrom(0, src, len, r.s.handle); +} + +fn noSums(_: c.Shape) usize { + return 0; +} + +/// A pool's four products summed in order. +fn fourTerms(_: c.Shape) usize { + return 4; +} + +/// A decode window's rows and a prompt chunk's: shape rows x the row's elements (k: the pool's rows for the keys). +pub fn spec(comptime which: enum { lat_write, pool_keys }) c.Spec { + return switch (which) { + .lat_write => .{ .name = "cuda glm tf_glm_lat_write on pages out of order", .accum = .none, .depth = noSums, .out = .bf16, .served = &.{ .{ .rows = 8, .n = lw, .k = 1 }, .{ .rows = 2048, .n = lw, .k = 1 } } }, + .pool_keys => .{ .name = "cuda glm tf_glm_pool_keys on pages out of order", .accum = .f32, .depth = fourTerms, .out = .bf16, .served = &.{ .{ .rows = 8, .n = dk, .k = 4 }, .{ .rows = 2048, .n = dk, .k = 4 } }, .sums = .{ .f16 = 0, .bf16 = 0, .exact = false, .why = "a pool's four bf16 products of softmax weights: no input makes the sum exact in fp32" } }, + }; +} + +/// Gate 1's values in `x` [rows][w]: NaN, +inf and -inf in the first rows, both infinities in row 3. +fn plant(x: []u16, rows: usize, w: usize) void { + const at = [_][2]usize{ .{ 0, w / 3 }, .{ 1, w / 2 }, .{ 2, w - 1 }, .{ 3, 0 }, .{ 3, w - 1 } }; + const put = [_]f32{ std.math.nan(f32), std.math.inf(f32), -std.math.inf(f32), std.math.inf(f32), -std.math.inf(f32) }; + for (at, put) |p, v| if (p[0] < rows) { + x[p[0] * w + p[1]] = c.toBf16(v); + }; +} + +/// latent._lat_write: `rows` latent rows from row `at` on through an out-of-order table, gathered back in order. +pub const LatWrite = struct { + rig: rig_.Rig, + f: cuda.Function, + shape: c.Shape = undefined, + lat: []u16 = &.{}, + ids: []i32 = &.{}, + at: usize = 0, + in: u64 = 0, + pos: u64 = 0, + pt: u64 = 0, + arena: u64 = 0, + out: u64 = 0, + + pub fn deinit(self: *LatWrite) void { + self.free(); + self.rig.deinit(); + } + + fn free(self: *LatWrite) void { + const a = self.rig.gpu.gpa; + a.free(self.lat); + a.free(self.ids); + self.lat = &.{}; + self.ids = &.{}; + } + + pub fn kernel(self: *LatWrite) g.Kernel { + return .{ .spec = spec(.lat_write), .ctx = self, .vt = &vt }; + } + + const vt = g.hooks(LatWrite); + + pub fn prepare(self: *LatWrite, shape: c.Shape, fill: g.Fill, seed: u64) !bool { + if (shape.n != lw or shape.rows == 0) return false; + self.free(); + self.rig.reset(); + self.shape = shape; + const a = self.rig.gpu.gpa; + var prng = std.Random.DefaultPrng.init(seed); + const rng = prng.random(); + self.at = firstRow(shape.rows); + self.lat = try a.alloc(u16, shape.rows * lw); + for (self.lat) |*v| v.* = c.toBf16(rng.float(f32) * 4 - 2); + if (fill == .nonfinite) plant(self.lat, shape.rows, lw); + const pages = (self.at + shape.rows + rows_a_page - 1) / rows_a_page; + self.ids = try shuffled(a, rng, pages); + const p: i32 = @intCast(self.at); + self.in = try self.rig.input(std.mem.sliceAsBytes(self.lat)); + self.pos = try self.rig.input(std.mem.asBytes(&p)); + self.pt = try self.rig.input(std.mem.sliceAsBytes(self.ids)); + self.arena = try self.rig.scratch((pages + 3) * page_bytes); + self.out = try self.rig.output(shape.rows * lw * 2); + return true; + } + + pub fn run(self: *LatWrite) !void { + var a: cuda.Args = .{}; + inline for (.{ self.in, @as(i64, lw), self.arena + part_bytes, self.pos, @as(i64, lw), self.pt, @as(i64, page_bytes) }) |v| a.add(v); + try cuda.launch.launch(self.f, .{ .grid = .{ .x = @intCast(self.shape.rows) }, .block = .{ .x = 128 } }, self.rig.s, &a); + var r: usize = 0; + while (r < self.shape.rows) { // the rows back from their pages, a page's run at a time + const row = self.at + r; + const n = @min(self.shape.rows - r, rows_a_page - row % rows_a_page); + const page: u64 = @intCast(self.ids[row / rows_a_page]); + try copy(&self.rig, self.out + r * lw * 2, self.arena + part_bytes + page * page_bytes + (row % rows_a_page) * lw * 2, n * lw * 2); + r += n; + } + try self.rig.s.synchronize(); + } + + pub fn count(self: *LatWrite) usize { + return self.shape.rows * lw; + } + + /// The rows as given: a write rounds nothing. + pub fn reference(self: *LatWrite, want: []f64, mag: []f64) !void { + for (want, mag, self.lat) |*w, *m, v| { + w.* = c.fromBf16(v); + m.* = if (std.math.isFinite(w.*)) @abs(w.*) else 0; + } + } +}; + +/// The float64 maximum that keeps a NaN, as the float64 formula's own arithmetic does. +pub fn maxNan(x: f64, y: f64) f64 { + return if (std.math.isNan(x) or std.math.isNan(y)) std.math.nan(f64) else @max(x, y); +} + +/// `v` as the accumulator `acc` holds it. +pub fn held(acc: c.Accum, v: f64) f64 { + return switch (acc) { + .f32 => @as(f32, @floatCast(v)), + .f16 => @as(f16, @floatCast(v)), + .bf16 => c.fromBf16(c.toBf16(@floatCast(v))), + .none => v, + }; +} + +/// Terms summed in `acc` in `order`: in turn, in halves, or every other term and then the rest. +pub fn sumIn(acc: c.Accum, order: g.Order, t: []const f64) f64 { + switch (order) { + .sequential => { + var s = t[0]; + for (t[1..]) |v| s = held(acc, s + v); + return s; + }, + .pairwise => return if (t.len == 1) t[0] else held(acc, sumIn(acc, .pairwise, t[0 .. t.len / 2]) + sumIn(acc, .pairwise, t[t.len / 2 ..])), + .strided => { + if (t.len == 1) return t[0]; + var s = [2]f64{ t[0], t[1] }; + for (t[2..], 2..) |v, i| s[i % 2] = held(acc, s[i % 2] + v); + return held(acc, s[0] + s[1]); + }, + } +} + +/// Four terms summed in `acc` in `order`: in turn, in pairs, or the odd terms then the even. +fn sum4(acc: c.Accum, order: g.Order, t: [4]f64) f64 { + return switch (order) { + .sequential => held(acc, held(acc, held(acc, t[0] + t[1]) + t[2]) + t[3]), + .pairwise => held(acc, held(acc, t[0] + t[1]) + held(acc, t[2] + t[3])), + .strided => held(acc, held(acc, t[0] + t[2]) + held(acc, t[1] + t[3])), + }; +} + +/// sparse._pool_keys: each complete pool's 4 rows softmaxed by their gates (+ ape), summed onto pages out of order. +pub const PoolKeys = struct { + rig: rig_.Rig, + f: cuda.Function, + shape: c.Shape = undefined, + ik: []u16 = &.{}, + ig: []u16 = &.{}, + ape: []u16 = &.{}, + ids: []i32 = &.{}, + at: usize = 0, + first: usize = 0, // the first complete pool + pools: usize = 0, + in: [3]u64 = @splat(0), + pos: u64 = 0, + pt: u64 = 0, + arena: u64 = 0, + out: u64 = 0, + + pub fn deinit(self: *PoolKeys) void { + self.free(); + self.rig.deinit(); + } + + fn free(self: *PoolKeys) void { + const a = self.rig.gpu.gpa; + inline for (.{ "ik", "ig", "ape" }) |name| { + a.free(@field(self, name)); + @field(self, name) = &.{}; + } + a.free(self.ids); + self.ids = &.{}; + } + + pub fn kernel(self: *PoolKeys) g.Kernel { + return .{ .spec = spec(.pool_keys), .ctx = self, .vt = &vt }; + } + + const vt = g.hooks(PoolKeys); + + /// Keys and gates in [-2, 2); non-finite: NaN, +inf and -inf gates in the first pools, an infinite key in a fourth. + pub fn prepare(self: *PoolKeys, shape: c.Shape, fill: g.Fill, seed: u64) !bool { + if (shape.n != dk or shape.k != 4 or shape.rows < 4) return false; + self.free(); + self.rig.reset(); + self.shape = shape; + const a = self.rig.gpu.gpa; + var prng = std.Random.DefaultPrng.init(seed); + const rng = prng.random(); + self.at = firstRow(shape.rows); + self.first = self.at / 4; + self.pools = (self.at + shape.rows) / 4 - self.first; + const n = self.at + shape.rows; + self.ik = try a.alloc(u16, n * dk); + self.ig = try a.alloc(u16, n * dk); + self.ape = try a.alloc(u16, 4 * dk); + for (self.ik) |*v| v.* = c.toBf16(rng.float(f32) * 4 - 2); + for (self.ig) |*v| v.* = c.toBf16(rng.float(f32) * 4 - 2); + for (self.ape) |*v| v.* = c.toBf16(rng.float(f32) - 0.5); + if (fill == .nonfinite) { + const put = [_]f32{ std.math.nan(f32), std.math.inf(f32), -std.math.inf(f32) }; + for (put, 0..) |v, i| if (i < self.pools) { + self.ig[(4 * (self.first + i) + 1) * dk + 7 * i] = c.toBf16(v); + }; + if (self.pools > 3) self.ik[(4 * (self.first + 3) + 2) * dk + 5] = c.toBf16(std.math.inf(f32)); + } + const pages = (n + rows_a_page - 1) / rows_a_page; + self.ids = try shuffled(a, rng, pages); + const p: i32 = @intCast(self.at); + inline for (.{ self.ik, self.ig, self.ape }, 0..) |x, i| self.in[i] = try self.rig.input(std.mem.sliceAsBytes(x)); + self.pos = try self.rig.input(std.mem.asBytes(&p)); + self.pt = try self.rig.input(std.mem.sliceAsBytes(self.ids)); + self.arena = try self.rig.scratch((pages + 3) * page_bytes); + self.out = try self.rig.output(self.pools * dk * 2); + return true; + } + + pub fn run(self: *PoolKeys) !void { + var a: cuda.Args = .{}; + const pk = self.arena + part_bytes + rows_a_page * lw * 2; // the cache's pooled keys in page 0 + inline for (.{ self.in[0], self.in[1], self.in[2], pk, self.pos, @as(i64, @intCast(self.shape.rows)), self.pt, @as(i64, page_bytes) }) |v| a.add(v); + try cuda.launch.launch(self.f, .{ .grid = .{ .x = @intCast(self.shape.rows / 4 + 2) }, .block = .{ .x = 32 } }, self.rig.s, &a); + var i: usize = 0; + while (i < self.pools) { // the pools back from their pages, a page's run at a time + const p = self.first + i; + const n = @min(self.pools - i, pools_a_page - p % pools_a_page); + const page: u64 = @intCast(self.ids[p / pools_a_page]); + try copy(&self.rig, self.out + i * dk * 2, pk + page * page_bytes + (p % pools_a_page) * dk * 2, n * dk * 2); + i += n; + } + try self.rig.s.synchronize(); + } + + pub fn count(self: *PoolKeys) usize { + return self.pools * dk; + } + + /// Pool p's weights and products as the kernel forms them, summed in `acc`; null acc: float64 throughout. + fn pool(self: *const PoolKeys, p: usize, d: usize, acc: ?c.Accum, order: g.Order, mag: *f64) f64 { + var l: [4]f64 = undefined; + for (&l, 0..) |*x, q| x.* = c.fromBf16(self.ig[(4 * p + q) * dk + d]) + c.fromBf16(self.ape[q * dk + d]); + const m = maxNan(maxNan(l[0], l[1]), maxNan(l[2], l[3])); + var e: [4]f64 = undefined; + for (&e, l) |*x, v| x.* = @exp(v - m); + const s = if (acc) |k| sum4(k, order, e) else e[0] + e[1] + e[2] + e[3]; + var t: [4]f64 = undefined; + mag.* = 0; + for (&t, e, 0..) |*x, w, q| { + const key = c.fromBf16(self.ik[(4 * p + q) * dk + d]); + x.* = if (acc != null) c.fromBf16(c.toBf16(@floatCast(c.fromBf16(c.toBf16(@floatCast(w / s))) * key))) else w / s * key; + mag.* += if (std.math.isFinite(x.*)) @abs(w / s * key) else 0; + } + return if (acc) |k| sum4(k, order, t) else t[0] + t[1] + t[2] + t[3]; + } + + pub fn reference(self: *PoolKeys, want: []f64, mag: []f64) !void { + for (0..self.pools) |i| for (0..dk) |d| { + want[i * dk + d] = self.pool(self.first + i, d, null, .sequential, &mag[i * dk + d]); + }; + } + + /// Each product rounds its weight and itself to bf16 (2^-8 each of its size); the sum and exponent add a little. + pub fn bounds(_: *PoolKeys, want: []const f64, mag: []const f64, allow: []f64) !void { + for (allow, want, mag) |*l, w, m| l.* = m * (0x1p-7 + 0x1p-14) + @abs(w) * 0x1p-8 * (1 + 0x1p-6) + 0x1p-133; + } + + pub fn simulate(self: *PoolKeys, acc: c.Accum, order: g.Order, out: []f64) !void { + var mag: f64 = 0; + for (0..self.pools) |i| for (0..dk) |d| { + out[i * dk + d] = c.fromBf16(c.toBf16(@floatCast(self.pool(self.first + i, d, acc, order, &mag)))); + }; + } +}; diff --git a/zig/tests/cuda/contract_tests.zig b/zig/tests/cuda/contract_tests.zig index d1ae2f6973..2b909aab29 100644 --- a/zig/tests/cuda/contract_tests.zig +++ b/zig/tests/cuda/contract_tests.zig @@ -32,6 +32,7 @@ const act_k = @import("contract_act.zig"); const gdn_k = @import("contract_gdn.zig"); const qk_k = @import("contract_qk.zig"); const wrows_k = @import("contract_window_rows.zig"); +const glm_k = @import("contract_glm.zig"); const embed_k = @import("contract_embed.zig"); const tree_k = @import("contract_tree_attn.zig"); @@ -65,7 +66,7 @@ pub fn image(name: []const u8) error{UnknownContractImage}![]const u8 { if (@import("contract_deepseek_images.zig").image(name)) |bytes| return bytes; if (std.mem.eql(u8, name, "window_rows")) return kk.window_rows; if (std.mem.eql(u8, name, "h3")) return &h3_image.bytes; - inline for (.{ "tree_round", "act_rows", "gdn_prep", "qk_rows", "norms", "moe", "experts", "qwen36_mtp", "nemotron_attention", "nemotron_route", "nemotron_latent", "qmm_group", "lane_gemv", "prompt16", "qmmf", "ck_experts", "rank_sum", "prefill_attention", "vit", "embed_rows", "route_rows", "flashnext_gdn", "flashnext_batch", "flashnext_hc", "flashnext_attn", "flashnext_mtp", "qwen_vision", "exl3_linear", "exl3_experts", "exl3_candidate", "experts3", "experts3_prefill", "drama3", "drama3_fp8", "drama3_codec", "glm_mm", "glm_dflash", "glm_glue", "nemotron_mamba", "sample", "qwen35", "qwen35_keep", "qwen35_kv8", "qwen35_draft", "ck_lane", "gdn_train", "train_q4", "train_attn", "flashnext_train", "train", "attn_train", "qwen27_train", "tree_attention", "block_attention", "window_attention", "moe_glue", "dense_rows", "qmm", "flashnext_fact", "flashnext_hcf", "flashnext_fexp", "experts_rows" }) |n| if (std.mem.eql(u8, name, n)) return @field(kk, n); + inline for (.{ "tree_round", "act_rows", "gdn_prep", "qk_rows", "norms", "moe", "experts", "qwen36_mtp", "nemotron_attention", "nemotron_route", "nemotron_latent", "qmm_group", "lane_gemv", "prompt16", "qmmf", "ck_experts", "rank_sum", "prefill_attention", "vit", "embed_rows", "route_rows", "flashnext_gdn", "flashnext_batch", "flashnext_hc", "flashnext_attn", "flashnext_mtp", "qwen_vision", "exl3_linear", "exl3_experts", "exl3_candidate", "experts3", "experts3_prefill", "drama3", "drama3_fp8", "drama3_codec", "glm_mm", "glm_dflash", "glm_glue", "glm_ops", "nemotron_mamba", "sample", "qwen35", "qwen35_keep", "qwen35_kv8", "qwen35_draft", "ck_lane", "gdn_train", "train_q4", "train_attn", "flashnext_train", "train", "attn_train", "qwen27_train", "tree_attention", "block_attention", "window_attention", "moe_glue", "dense_rows", "qmm", "flashnext_fact", "flashnext_hcf", "flashnext_fexp", "experts_rows" }) |n| if (std.mem.eql(u8, name, n)) return @field(kk, n); if (std.mem.eql(u8, name, "contract_controls")) return if (options.controls) &controls.bytes else &.{}; std.debug.print("FAIL kernel-contract: unknown image name: {s}\n", .{name}); return error.UnknownContractImage; @@ -425,6 +426,9 @@ pub fn run(gpu: check.Gpu) !void { for (&wrows, wrows_k.cfgs) |*x, f| x.* = .{ .rig = .{ .gpu = gpu, .s = s }, .f = f, .g = &window_fns, .ctl = mods[5], .arena = .init(a) }; defer for (&wrows) |*x| x.deinit(); for (&wrows) |*x| try kernels.append(a, x.kernel()); + var glm_cases = try glm_k.Cases.init(gpu, s); + defer glm_cases.deinit(); + try glm_cases.append(a, &kernels); var t: g.Tally = .{}; try @import("contract_dump.zig").run(a, kernels.items); const t0 = check.now(gpu.io); diff --git a/zig/tests/cuda/glm_paged_tests.zig b/zig/tests/cuda/glm_paged_tests.zig new file mode 100644 index 0000000000..29faf94876 --- /dev/null +++ b/zig/tests/cuda/glm_paged_tests.zig @@ -0,0 +1,112 @@ +//! GLM-5.3-Flash's KV pages on CUDA: a paged stream against its twin on a whole window, byte for byte. +const std = @import("std"); +const cuda = @import("cuda"); +const check = @import("check.zig"); +const lanes_tests = @import("glm_lanes_tests.zig"); +const Gpu = check.Gpu; + +/// Prompts at a page's edges (512 rows), a prompt chunk's (2,048) and past the dense pass (the sparse one reads pools). +const lengths = [_]usize{ 511, 512, 513, 2047, 2048, 2049, 5000 }; +/// A window of 8,192 and a verify window, as cuda_native.capacity makes it. +const capacity = 8256; +const paged_slot = 0; // the engine's own sequence, on pages from its init +const shuffled = 1; // takes and gives back pages between prompts, so slot 0's come out of the free list's order +const twin_slot = 3; // past the paged sequences: a whole window, as without TF_KV_PAGES + +/// `glm-paged-check`: pages taken out of order and NaN-filled; prompt, windows across page edges, keeps and drafts. +pub fn pagedCheck(gpu: Gpu, checkpoint: []const u8, rank: c_int, world: c_int, id_path: []const u8) !void { + const glm = @import("glm_cuda"); + var arena = std.heap.ArenaAllocator.init(gpu.gpa); + defer arena.deinit(); + const a = arena.allocator(); + var lib = try cuda.nccl.Library.open(); + defer lib.close(); + const comm = try lanes_tests.openComm(gpu, &lib, rank, world, id_path); + defer _ = lib.api.ncclCommDestroy(comm); + var problem: []const u8 = ""; + var set = glm.ranks.Settings.read(&glm.ranks.environment, &problem) catch |err| { + std.debug.print("REFUSE {s}\n", .{problem}); + return err; + }; + set.pages = true; // every rank's engine makes its own sequence on pages, as TF_KV_PAGES does + const e = try glm.engine.Engine.init(gpu.gpa, gpu.io, gpu.d, gpu.ctx, checkpoint, &lib, comm, @intCast(rank), @intCast(world), .{ .capacity = capacity, .mtp = true, .settings = set }); + defer e.deinit(); + var link = try glm.follow.Link.init(gpu.gpa, gpu.d, &lib, comm, e.stream, capacity); + defer link.deinit(); + if (rank != 0) { + try glm.follow.follow(e, &link); + check.pass("rank {d} made rank 0's pool and mapped its pages through every step", .{rank}); + return; + } + const x: glm.follow.Leader = .{ .e = e, .link = &link }; + defer x.end() catch {}; + const P = glm.pages; + const l = try P.layout(gpu.gpa, e, 0); + const count = 2 * P.tableLen(capacity) + 3; // two whole windows and a few pages over + try x.pool(2, count, l.pageBytes()); + e.pages = try P.Pages.init(gpu.gpa, e, l, 2, count, true); + const pages = e.pages.?; + pages.poison = true; // a row read before its write turns the reply to NaN + const V = e.share.vocab.len(); + const mine = try a.alloc(u16, 4 * V); + const twin = try a.alloc(u16, 4 * V); + var failed = false; + var unordered: usize = 0; + var compared: usize = 0; + for (lengths, 0..) |len, k| { + if (k % 2 == 0) { + try x.bind(shuffled); + _ = try pages.open(shuffled, (k + 1) * 512 - 100, 0); + } else x.release(shuffled) catch {}; + const ids = try a.alloc(u32, len); + for (ids, 0..) |*u, i| u.* = @intCast(100 + (i * 7919 + k * 31) % 5000); + const held = try pages.open(paged_slot, len, 64); + for (held[1..], held[0 .. held.len - 1]) |b, prev| unordered += @intFromBool(b != prev + 1); + var first: [2]u32 = undefined; + var drafts: [2][glm.engine.max_drafts]u32 = undefined; + for ([_]u32{ paged_slot, twin_slot }, 0..) |slot, j| { + try x.bind(slot); + if (slot == paged_slot) try x.beginOn(null, held) else try x.begin(null); + first[j] = try x.prefill(ids, null, null, null); + try gpu.d.check(gpu.d.api.cuMemcpyDtoH_v2(if (j == 0) mine.ptr else twin.ptr, e.bp.logits, V * 2), "cuMemcpyDtoH"); + try x.draft(true, &.{first[j]}, 2, &drafts[j]); + } + if (first[0] != first[1] or !std.mem.eql(u16, mine[0..V], twin[0..V]) or !std.mem.eql(u32, drafts[0][0..2], drafts[1][0..2])) { + std.debug.print("FAIL paged: a {d}-token prompt's first token {d}, logits or drafts differ from its twin's ({d})\n", .{ len, first[0], first[1] }); + failed = true; + } + for (0..6) |round| { // windows of 1 to 4 rows from the prompt's end, across its page's edge + const rows = 1 + round % 4; + var w: [4]u32 = undefined; + for (w[0..rows], 0..) |*u, r| u.* = @intCast(100 + (round * 31 + k * 7 + r * 13) % 5000); + var outs: [2][4]u32 = undefined; + for ([_]u32{ paged_slot, twin_slot }, 0..) |slot, j| { + try x.bind(slot); + try x.verify(w[0..rows], outs[j][0..rows], 0); + try gpu.d.check(gpu.d.api.cuMemcpyDtoH_v2(if (j == 0) mine.ptr else twin.ptr, e.bd.logits, rows * V * 2), "cuMemcpyDtoH"); + const keep = 1 + (round + k) % rows; + try x.keep(keep); + try x.draft(false, outs[j][0..keep], 2, &drafts[j]); + } + if (!std.mem.eql(u16, mine[0 .. rows * V], twin[0 .. rows * V]) or !std.mem.eql(u32, outs[0][0..rows], outs[1][0..rows]) or !std.mem.eql(u32, drafts[0][0..2], drafts[1][0..2])) { + std.debug.print("FAIL paged: a {d}-token prompt's window {d} ({d} rows) differs from its twin's\n", .{ len, round, rows }); + failed = true; + } + compared += rows; + } + try x.release(paged_slot); + try x.release(twin_slot); + } + x.release(shuffled) catch {}; + const c = pages.pool.census(); + if (!pages.pool.idle() or c.free != c.total) { + std.debug.print("FAIL paged: the pool is not idle after every stream ended (free {d} of {d}, promised {d})\n", .{ c.free, c.total, c.promised }); + failed = true; + } + if (unordered == 0) { + std.debug.print("FAIL paged: every stream's pages came in order; the check did not reach a shuffled table\n", .{}); + failed = true; + } + if (failed) return error.TestFailed; + check.pass("pages at TP={d}: on the engine's own sequence, {d} prompts at the pages' edges (511 to 5,000 tokens) and {d} window rows equal their twins' logits, tokens and drafts byte for byte, pages out of order ({d} jumps) and NaN before use; the pool idle after", .{ world, lengths.len, compared, unordered }); +} diff --git a/zig/tests/cuda/glm_ref_attn.zig b/zig/tests/cuda/glm_ref_attn.zig index d4e6911d58..49cc649067 100644 --- a/zig/tests/cuda/glm_ref_attn.zig +++ b/zig/tests/cuda/glm_ref_attn.zig @@ -74,7 +74,7 @@ fn attend(r: *Rig, set: glm.glue.Set, q: u64, cache: u64, pos: usize, rows: usiz const out = try r.zeros(rows * heads * lw * 2); const p0: i32 = @intCast(pos); const at = try r.dev(std.mem.asBytes(&p0)); - try latent.attention(set, r.s, q, cache, at, try partials(r, nch, rows), out, try r.zeros(4), rows, heads, lw, nch, scale, hb); + try latent.attention(set, r.s, q, cache, .{}, at, try partials(r, nch, rows), out, try r.zeros(4), rows, heads, lw, nch, scale, hb); return out; } @@ -142,19 +142,19 @@ pub fn sparse(r: *Rig, set: glm.glue.Set) !void { const dcnt = try r.dev(std.mem.sliceAsBytes(cnt)); const out = try r.zeros(rows * heads * lw * 2); const nch = latent.chunksFor(w); - try latent.sparseAttention(set, r.s, dq, c.dev, dtok, dcnt, w, try partials(r, nch, rows), out, rows, heads, lw, scale); + try latent.sparseAttention(set, r.s, dq, c.dev, .{}, dtok, dcnt, w, try partials(r, nch, rows), out, rows, heads, lw, scale); const ref64 = try reference(r, q, c.f64s, sets, scale); var name: [128]u8 = undefined; try ref.closeCols(r, try std.fmt.bufPrint(&name, "latent sparse attention {d} rows, up to {d} of {d} keys", .{ rows, w, keys }), out, false, ref64.want, ref64.tol, lw); const all = try r.back(out, rows * heads * lw * 2); for ([_]usize{ 0, 37, 69 }) |row| { const one = try r.zeros(heads * lw * 2); - try latent.sparseAttention(set, r.s, dq + row * heads * lw * 2, c.dev, dtok + row * w * 4, dcnt + row * 4, w, try partials(r, nch, 1), one, 1, heads, lw, scale); + try latent.sparseAttention(set, r.s, dq + row * heads * lw * 2, c.dev, .{}, dtok + row * w * 4, dcnt + row * 4, w, try partials(r, nch, 1), one, 1, heads, lw, scale); try sameRow(r, try std.fmt.bufPrint(&name, "latent sparse attention: row {d} alone equals its row in the {d}-row call", .{ row, rows }), all, one, row); } const none: i32 = 0; const left = try r.zeros(heads * lw * 2); - try latent.sparseAttention(set, r.s, dq, c.dev, dtok, try r.dev(std.mem.asBytes(&none)), w, try partials(r, nch, 1), left, 1, heads, lw, scale); + try latent.sparseAttention(set, r.s, dq, c.dev, .{}, dtok, try r.dev(std.mem.asBytes(&none)), w, try partials(r, nch, 1), left, 1, heads, lw, scale); for (try r.back(left, heads * lw * 2)) |b| try check.expect(b == 0xff, "latent sparse attention: a row with count 0 is left alone", .{}); check.pass("latent sparse attention over up to {d} tokens: within float64 tolerance, one-row windows equal, count 0 left alone", .{w}); } diff --git a/zig/tests/cuda/glm_ref_fp8.zig b/zig/tests/cuda/glm_ref_fp8.zig index 0809ef357a..afd982ce73 100644 --- a/zig/tests/cuda/glm_ref_fp8.zig +++ b/zig/tests/cuda/glm_ref_fp8.zig @@ -26,7 +26,7 @@ pub fn run(r: *Rig, set: glm.glue.Set) !void { const input = try r.dev(std.mem.sliceAsBytes(x)); const fp_cache = try r.zeros(try latent.cacheBytes(capacity, lw, true)); const scales = fp_cache + capacity * lw; - try latent.latentWriteFormat(set, r.s, input, stride, fp_cache, scales, try position(r, 0), keys, lw); + try latent.latentWriteFormat(set, r.s, input, stride, fp_cache, scales, .{}, try position(r, 0), keys, lw); const want_codes = try r.a.alloc(u8, keys * lw); const want_scales = try r.a.alloc(f32, keys); const decoded = try r.a.alloc(u16, capacity * lw); @@ -48,7 +48,7 @@ pub fn run(r: *Rig, set: glm.glue.Set) !void { const split = try r.zeros(try latent.cacheBytes(capacity, lw, true)); const ss = split + capacity * lw; var row: usize = 0; - while (row < keys) : (row += chunk) try latent.latentWriteFormat(set, r.s, input + row * stride * 2, stride, split, ss, try position(r, row), @min(chunk, keys - row), lw); + while (row < keys) : (row += chunk) try latent.latentWriteFormat(set, r.s, input + row * stride * 2, stride, split, ss, .{}, try position(r, row), @min(chunk, keys - row), lw); try check.sameBytes("GLM FP8 chunk-independent row codes", try r.back(split, keys * lw), want_codes); try check.sameBytes("GLM FP8 chunk-independent scales", try r.back(ss, keys * 4), std.mem.sliceAsBytes(want_scales)); } @@ -69,14 +69,14 @@ pub fn run(r: *Rig, set: glm.glue.Set) !void { for ([_]usize{ 16, 32 }) |hb| { const fp_out = try r.zeros(rows * heads * lw * 2); const bf_out = try r.zeros(rows * heads * lw * 2); - try latent.attentionFormat(set, r.s, dq, fp_cache, scales, at, try partials(r, nch, rows), fp_out, try r.zeros(4), rows, heads, lw, nch, 0.125, hb); - try latent.attention(set, r.s, dq, bf_cache, at, try partials(r, nch, rows), bf_out, try r.zeros(4), rows, heads, lw, nch, 0.125, hb); + try latent.attentionFormat(set, r.s, dq, fp_cache, scales, .{}, at, try partials(r, nch, rows), fp_out, try r.zeros(4), rows, heads, lw, nch, 0.125, hb); + try latent.attention(set, r.s, dq, bf_cache, .{}, at, try partials(r, nch, rows), bf_out, try r.zeros(4), rows, heads, lw, nch, 0.125, hb); const wide = try r.back(fp_out, rows * heads * lw * 2); try check.sameBytes("GLM FP8 reader equals reconstructed BF16 cache", wide, try r.back(bf_out, wide.len)); try ref.closeCols(r, "GLM FP8 reconstructed attention vs FP64", fp_out, false, want.want, want.tol, lw); for (0..rows) |row| { const one = try r.zeros(heads * lw * 2); - try latent.attentionFormat(set, r.s, dq + row * heads * lw * 2, fp_cache, scales, try position(r, pos + row), try partials(r, nch, 1), one, try r.zeros(4), 1, heads, lw, nch, 0.125, hb); + try latent.attentionFormat(set, r.s, dq + row * heads * lw * 2, fp_cache, scales, .{}, try position(r, pos + row), try partials(r, nch, 1), one, try r.zeros(4), 1, heads, lw, nch, 0.125, hb); try check.sameBytes("GLM FP8 concurrent row equals solo row", try r.back(one, heads * lw * 2), wide[row * heads * lw * 2 ..][0 .. heads * lw * 2]); } } @@ -86,8 +86,8 @@ pub fn run(r: *Rig, set: glm.glue.Set) !void { const dc = try r.dev(std.mem.asBytes(&counts)); const sparse_fp = try r.zeros(rows * heads * lw * 2); const sparse_bf = try r.zeros(rows * heads * lw * 2); - try latent.sparseAttentionFormat(set, r.s, dq, fp_cache, scales, dt, dc, 3, try partials(r, 1, rows), sparse_fp, rows, heads, lw, 0.125); - try latent.sparseAttention(set, r.s, dq, bf_cache, dt, dc, 3, try partials(r, 1, rows), sparse_bf, rows, heads, lw, 0.125); + try latent.sparseAttentionFormat(set, r.s, dq, fp_cache, scales, .{}, dt, dc, 3, try partials(r, 1, rows), sparse_fp, rows, heads, lw, 0.125); + try latent.sparseAttention(set, r.s, dq, bf_cache, .{}, dt, dc, 3, try partials(r, 1, rows), sparse_bf, rows, heads, lw, 0.125); try check.sameBytes("GLM FP8 sparse reader and count-zero preservation", try r.back(sparse_fp, rows * heads * lw * 2), try r.back(sparse_bf, rows * heads * lw * 2)); } check.pass("GLM FP8 synthetic format, masking, row/chunk identity and reconstructed BF16/FP64 attention", .{}); diff --git a/zig/tests/cuda/glm_ref_index.zig b/zig/tests/cuda/glm_ref_index.zig index b60dc3c9ea..f9d027cf7e 100644 --- a/zig/tests/cuda/glm_ref_index.zig +++ b/zig/tests/cuda/glm_ref_index.zig @@ -32,7 +32,7 @@ pub fn selection(r: *Rig, set: glm.glue.Set, ops: cuda.Module) !void { const s: sp.Scratch = .{ .scores = try r.zeros(rows * np * 4), .ls = try r.zeros(rows * sp.select_list * 4), .li = try r.zeros(rows * sp.select_list * 4), .pools = try r.zeros(rows * sp.topk_pools * 8), .at = try r.zeros(4) }; const tokens = try r.zeros(rows * sp.width * 4); const counts = try r.zeros(rows * 4); - try sp.selectTokens(set, ops, r.s, try r.dev(std.mem.sliceAsBytes(qi)), try r.dev(std.mem.sliceAsBytes(wts)), heads, try r.dev(std.mem.sliceAsBytes(pk)), try r.dev(std.mem.asBytes(&p0)), pos, rows, heads, d, cap, s, tokens, counts); + try sp.selectTokens(set, ops, r.s, try r.dev(std.mem.sliceAsBytes(qi)), try r.dev(std.mem.sliceAsBytes(wts)), heads, try r.dev(std.mem.sliceAsBytes(pk)), .{}, try r.dev(std.mem.asBytes(&p0)), pos, rows, heads, d, cap, s, tokens, counts); const raw = try r.back(s.scores, rows * np * 4); const got = std.mem.bytesAsSlice(f32, raw); diff --git a/zig/tests/cuda/main.zig b/zig/tests/cuda/main.zig index 845a98f5e2..fe3d659a26 100644 --- a/zig/tests/cuda/main.zig +++ b/zig/tests/cuda/main.zig @@ -58,6 +58,7 @@ const lfm2_kernel_tests = @import("lfm2_kernel_tests.zig"); const lfm2_vision_tests = @import("lfm2_vision_tests.zig"); const radio_tests = @import("radio_tests.zig"); const glm_lanes_tests = @import("glm_lanes_tests.zig"); +const glm_paged_tests = @import("glm_paged_tests.zig"); const glm_synth = @import("glm_synth.zig"); const tree_round_tests = @import("tree_round_tests.zig"); const tree_tests = @import("tree_tests.zig"); @@ -210,6 +211,7 @@ const usage = \\ glm-lanes-cover copy drafts, cancels, discards, eos and max_new on one lanes.Engine across requests \\ glm-lanes-concurrent concurrent streams on one lanes.Engine, each against its solo run \\ glm-shared-check shared verify windows against lone twins, logits and states byte for byte + \\ glm-paged-check a paged stream (pages out of order, NaN first) against a whole-window twin \\ kolibri-reference MODEL IDS Kolibri-1's layer 0 and head against float64 from the checkpoint (prompt and decode rows) \\ kolibri-layers MODEL IDS Kolibri-1's window edge (513 keys) and its fifth layer's full reach, on the checkpoint \\ kolibri-full MODEL IDS N Kolibri-1: eager rows, steps, drafted, sampled, shared rounds, cuts and kept states alike @@ -401,6 +403,7 @@ fn run(gpu: check.Gpu, cmd: []const u8, rest: []const [:0]const u8) !void { if (std.mem.eql(u8, cmd, "dense-rows")) return dense_rows_tests.run(gpu); if (std.mem.eql(u8, cmd, "train-pack")) return train_pack.run(gpu, try arg(rest, 0), try arg(rest, 1), if (rest.len > 2) try std.fmt.parseInt(usize, rest[2], 10) else 24); if (std.mem.eql(u8, cmd, "glm-lanes-cover")) return glm_lanes_tests.coverCheck(gpu, try arg(rest, 0), try arg(rest, 1), try std.fmt.parseInt(c_int, try arg(rest, 2), 10), try std.fmt.parseInt(c_int, try arg(rest, 3), 10), try arg(rest, 4)); + if (std.mem.eql(u8, cmd, "glm-paged-check")) return glm_paged_tests.pagedCheck(gpu, try arg(rest, 0), try std.fmt.parseInt(c_int, try arg(rest, 1), 10), try std.fmt.parseInt(c_int, try arg(rest, 2), 10), try arg(rest, 3)); if (std.mem.eql(u8, cmd, "glm-shared-check")) return glm_lanes_tests.sharedCheck(gpu, try arg(rest, 0), try arg(rest, 1), try std.fmt.parseInt(c_int, try arg(rest, 2), 10), try std.fmt.parseInt(c_int, try arg(rest, 3), 10), try arg(rest, 4)); if (std.mem.eql(u8, cmd, "glm-synth")) return glm_synth.run(gpu, try arg(rest, 0)); if (std.mem.eql(u8, cmd, "kolibri-reference")) return kolibri_layer_tests.reference(gpu, try arg(rest, 0), try arg(rest, 1));